Point cloud detection method and related device, scanner and storage medium

Abstract

The invention discloses a point cloud detection method, a related device, a scanner and a storage medium, and the method comprises the steps: carrying out the voxelization based on a real-time point cloud collected by the scanner, and obtaining a voxel grid; wherein the real-time point cloud comprises a plurality of three-dimensional points, and the voxel grid comprises a plurality of voxels; obtaining a voxel density score of the voxel based on the number value of the three-dimensional points in the voxel, and obtaining a normal visibility score of the voxel based on the estimation normal of the three-dimensional points in the voxel; fusing the voxel density score and the normal visibility score based on the voxel to obtain a comprehensive integrity score of the voxel; and at least prompting a target area suggested to be scanned on the real-time point cloud based on the comprehensive integrity score of each voxel. According to the scheme, the real-time performance of point cloud integrity judgment can be improved, the point cloud scanning efficiency is improved, and the complexity of scanning operation is reduced as much as possible.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and in particular to a point cloud detection method and related apparatus, scanner and storage medium. Background Technology

[0002] Existing 3D scanning equipment mainly collects point cloud data and, after completing the scanning of a certain area, uses algorithms such as Poisson reconstruction to convert the point cloud data into meshes to generate a mesh model, thereby observing the integrity of the point cloud data.

[0003] The aforementioned methods suffer from several drawbacks. First, they cannot assess the integrity of point cloud data in real-time because the integrity can only be determined after the mesh model is generated, resulting in poor real-time performance. Second, the conversion of point cloud data into a mesh model typically requires extensive computation, leading to lengthy processing times and low scanning efficiency. Furthermore, when point cloud data contains gaps, missing data, or discontinuities, users must repeatedly observe and perform rescanning, making the process cumbersome. Therefore, improving the real-time performance of point cloud integrity assessment, increasing point cloud scanning efficiency, and minimizing the complexity of scanning operations are urgent problems that need to be addressed. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a point cloud detection method and related apparatus, scanner, and storage medium, which can improve the real-time performance of point cloud integrity assessment, increase point cloud scanning efficiency, and minimize the complexity of scanning operations.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a point cloud detection method, comprising: voxelizing a real-time point cloud acquired by a scanner to obtain a voxel mesh; wherein the real-time point cloud contains a number of three-dimensional points, and the voxel mesh contains a number of voxels; obtaining a voxel density score based on the number of three-dimensional points within a voxel, and obtaining a voxel normal visibility score based on the estimated normals at the three-dimensional points within the voxel; fusing the voxel density score and the normal visibility score to obtain a voxel comprehensive integrity score; and, based on the comprehensive integrity scores of each voxel, indicating at least one target area on the real-time point cloud that should be scanned further.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a point cloud detection device, comprising: a point cloud voxel module, a voxel scoring module, a scoring fusion module, and a region prompting module. The point cloud voxel module is used to voxelize a real-time point cloud acquired by a scanner to obtain a voxel mesh; wherein the real-time point cloud contains a number of three-dimensional points, and the voxel mesh contains a number of voxels. The voxel scoring module is used to obtain a voxel density score based on the number of three-dimensional points within the voxel, and to obtain a voxel normal visibility score based on the estimated normals at the three-dimensional points within the voxel. The scoring fusion module is used to fuse the voxel density score and the normal visibility score to obtain a voxel comprehensive integrity score. The region prompting module is used to prompt at least one target area on the real-time point cloud that is recommended for further scanning based on the comprehensive integrity score of each voxel.

[0007] To address the aforementioned technical problems, a third aspect of this application provides a scanner that includes at least a memory and a processor coupled to each other. The memory stores at least program instructions, and the processor executes the program instructions to implement the point cloud detection method described in the first aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the point cloud detection method of the first aspect described above.

[0009] The above scheme performs voxelization on real-time point clouds acquired by a scanner to obtain a voxel mesh. The real-time point cloud contains several 3D points, and the voxel mesh contains several voxels. Based on the number of 3D points within each voxel, a voxel density score is obtained. Based on the estimated normals at the 3D points within each voxel, a voxel normal visibility score is obtained. The voxel density score and normal visibility score are then fused to obtain a comprehensive voxel integrity score. Based on the comprehensive integrity score of each voxel, at least the target area on the real-time point cloud that should be scanned is indicated. This is because by voxelizing the real-time point cloud and evaluating the voxel density and normal visibility of a single voxel... By scoring completeness across two dimensions—visibility and integrity—real-time point cloud integrity can be assessed without converting to a mesh model, improving the real-time performance of point cloud integrity assessment. Furthermore, the elimination of mesh model conversion reduces the computational burden of patching, enhancing point cloud scanning efficiency. Additionally, the comprehensive integrity score of each voxel directly suggests target areas for further scanning on the real-time point cloud, eliminating the need for users to repeatedly perform patching based on the suggested areas, thus minimizing scanning complexity. Therefore, this approach improves the real-time performance of point cloud integrity assessment, enhances point cloud scanning efficiency, and minimizes scanning complexity. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating an embodiment of the point cloud detection method of this application; Figure 2a This is a schematic diagram of an embodiment of voxelization and point counting in this application; Figure 2b This is a schematic diagram of a process for an embodiment of the normal visibility measurement of this application; Figure 2c This is a schematic diagram of an embodiment of the present application for displaying target regions on real-time point clouds; Figure 3 This is a schematic diagram of the framework of an embodiment of the point cloud detection device of this application; Figure 4 This is a schematic diagram of the frame of an embodiment of the scanner of this application; Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0011] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0012] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0013] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper indicates two or more objects.

[0014] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the point cloud detection method of this application. It should be noted that the process operations in this embodiment can be executed by an electronic device with computing capabilities or by related equipment including such an electronic device. The specific structure of the electronic device or related equipment including the electronic device is not limited here. Specifically, this embodiment may include the following steps: Step S11: Voxelize the real-time point cloud acquired by the scanner to obtain a voxel mesh.

[0015] In this embodiment, the real-time point cloud may contain several three-dimensional points, and the voxel mesh may contain several voxels. It should be noted that the scanner can acquire point clouds by combining a visible light camera with structured light scanning, or by combining a visible light camera with a depth camera, or by using a LiDAR. The specific method of point cloud acquisition by the scanner is not limited here, nor will it be listed in detail.

[0016] In one implementation scenario, the voxel edge length of a voxel mesh can be negatively correlated with the model resolution required for 3D modeling. That is, the higher the model resolution required for 3D modeling, the shorter the voxel edge length of the voxel mesh can be set, and vice versa. The specific value of the voxel edge length is not limited here.

[0017] In one implementation scenario, after determining the voxel edge length, the point cloud bounding box (the region defined by the minimum and maximum coordinates) can be divided into a 3D mesh (i.e., a voxel mesh) using the voxel edge length as the step size. For ease of understanding, the coordinates of each voxel can be represented as (i,j,k), which can correspond to the mesh index of the X, Y, and Z coordinate axes. Based on this, for each 3D point (x,y,z), the index of its corresponding voxel can be calculated:

[0018]

[0019]

[0020] In the above formula, x min y min z min These represent the minimum coordinates of the bounding box on the X-axis, Y-axis, and Z-axis, respectively, and v represents the voxel side length. Of course, the above example is merely one possible illustration of voxelization for real-time point clouds in practical applications; other possible implementations are not limited here, nor will they be listed in detail.

[0021] Step S12: Based on the number of three-dimensional points within the voxel, obtain the voxel density score, and based on the estimated normal at the three-dimensional points within the voxel, obtain the voxel normal visibility score.

[0022] In one implementation scenario, after voxelizing a real-time point cloud, the number of 3D points within each voxel in the voxel mesh can be counted, and the voxel density score can be determined accordingly. As one possible example, the voxel density score can be positively correlated with the number of 3D points within the voxel. That is, the larger the number of 3D points within the voxel, the higher the voxel density score; conversely, the smaller the number of 3D points within the voxel, the lower the voxel density score. As another possible example, a pre-set threshold can be used. When the number of 3D points within the voxel is below (or not higher than) the threshold, a first score can be assigned to the voxel density score; when the number of 3D points within the voxel is not lower than (or higher than) the threshold, a second score can be assigned, with the first score being lower than the second score. Of course, the above examples are merely a few possible examples of determining the voxel density score based on the number of 3D points within the voxel in practical applications. Other possible determination methods are not limited here, nor will they be listed in detail.

[0023] In one implementation scenario, if the number of 3D points within a voxel is not lower than a certain threshold, the voxel can be considered a non-empty voxel. The proportion of non-empty voxels in the voxel mesh can be used as the overall scan coverage, reflecting the real-time point cloud scan coverage from the perspective of point cloud density. It should be noted that the number threshold can be set according to actual application needs. For example, when high scan density is required, the number threshold can be set larger; conversely, when scan density requirements are relatively relaxed, the number threshold can be set appropriately smaller. The specific value of the number threshold is not limited here (e.g., between 5 and 10, or other possible values), and will not be listed in detail here. For easier understanding, please refer to [reference needed]. Figure 2a , Figure 2a This is a schematic diagram of an embodiment of voxelization and point counting in this application. Figure 2a As shown in the diagram on the far left, the pink dots represent 3D points in the real-time point cloud, the cuboids represent the bounding boxes of the real-time point cloud, and each small square in the middle diagram represents a voxel. Different color levels can be mapped to different values ​​for the number of 3D points contained in a voxel, such as... Figure 2a The rightmost diagram shows the flow rate from 0p / cm. 2 Up to 30p / cm 2 Different voxel densities can be assigned different colors, transitioning from warm to cool, to explicitly distinguish between them. Of course, Figure 2a The example shown is merely one possible instance in practical application. Other possible scenarios are not limited here, nor will they be listed one by one.

[0024] In one implementation scenario, as a possible approach, to obtain the normal visibility score of a voxel, a first surface can be fitted based on the 3D points within the voxel and the 3D points within each voxel's neighborhood. The normal vector of the first surface at any 3D point within the voxel is then obtained as the estimated normal. The angle between the estimated normal and the camera normal when the scanner acquires the 3D points within the voxel can then be obtained as the first angle. Furthermore, based on the first angle and the real-time point cloud distribution at the voxel, the normal visibility score of the voxel can be obtained. This method uses the voxel as the smallest unit for normal estimation, determines the estimated normal at the 3D points representing the voxel on the fitted surface through surface fitting, and combines the angle between the estimated normal and the camera normal, as well as the real-time point cloud distribution at the voxel, to jointly determine the normal visibility score of the voxel. This allows for normal visibility scoring from both visibility and continuity dimensions.

[0025] In a specific implementation scenario, for any voxel, a first surface can be obtained by combining the three-dimensional points within this voxel and its neighboring voxels, using methods such as least squares. Based on this, any three-dimensional point within the voxel can be selected to represent this voxel, or the three-dimensional point closest to the voxel's center can be selected to represent it; the specific method of selecting the three-dimensional point within the voxel is not limited here. After selecting the three-dimensional point within the voxel, the normal vector of the first surface at the selected three-dimensional point can be obtained as the estimated normal.

[0026] In a specific implementation scenario, the camera normal can be the normal vector of the camera imaging surface when the scanner acquires three-dimensional points within a voxel (such as the three-dimensional points selected within the voxel as mentioned above).

[0027] In a specific implementation scenario, after obtaining the first included angle and the point cloud distribution of the real-time point cloud at the voxel, the normal visibility score of the voxel can be determined by combining the two. Specifically, the point cloud density of the real-time point cloud at the voxel can be obtained based on the point cloud distribution of the real-time point cloud at the voxel. For example, the point cloud density of the real-time point cloud at the voxel can be measured by the number of 3D points contained within the voxel; or, for example, the point cloud density of the real-time point cloud at the voxel can be measured by the number of 3D points contained within the voxel and the number of 3D points contained within the voxel's neighboring voxels. Of course, the above methods are only a few possible ways to measure the density value of the real-time point cloud at the voxel, and the specific method for measuring the point cloud density of the real-time point cloud at the voxel is not limited here, nor will it be listed in detail. At the same time, the degree of deviation of the estimated normal from the camera normal can be obtained based on the first included angle. It should be noted that the larger the first included angle, the greater the degree of deviation, and vice versa. Based on this, a normal visibility score can be obtained based on point cloud density and deviation. It should be noted that the normal visibility score can be positively correlated with point cloud density. That is, the higher the point cloud density, the higher the normal visibility score, and vice versa. Furthermore, the normal visibility score can be negatively correlated with deviation. That is, the greater the deviation, the lower the normal visibility score, and vice versa. Therefore, in real-time point clouds, the voxels where the estimated normal is closer to the camera normal and the point cloud density is higher will have a higher normal visibility score. Conversely, in real-time point clouds, the voxels where the estimated normal deviates from the camera normal and the point cloud density is lower (usually areas where the scanner has difficulty scanning or accurately traversing when facing the target directly), will have a lower normal visibility score. The above method obtains the point cloud density at voxels based on the point cloud distribution of real-time point cloud, and obtains the degree of deviation of the estimated normal from the camera normal based on the first included angle. Then, based on the point cloud density and the degree of deviation, the normal visibility score is obtained, which can perform normal visibility scoring from two dimensions: normal deviation and point cloud density.

[0028] In another implementation scenario, as a different possible approach, distinct from the aforementioned implementation, to obtain the voxel's normal visibility score, a second surface can be fitted based on at least each 3D point within the voxel. Then, the normal vectors of the second surface at each 3D point within the voxel are obtained as estimated normals at the corresponding 3D points. This allows the acquisition of the angle between the estimated normal at each 3D point within the voxel and the camera normal when the scanner acquires the corresponding 3D point, serving as the second angle. Based on this second angle for each 3D point within the voxel, the voxel's normal visibility score can be obtained. This method uses the 3D points within the voxel as the smallest unit for normal estimation. By calculating the angle between the estimated normal at each 3D point within the voxel and the camera normal, the voxel's normal visibility score is determined, thus refining the evaluation granularity as much as possible during the determination of the normal visibility score.

[0029] In a specific implementation scenario, the specific method of surface fitting can be referred to the aforementioned descriptions, and will not be repeated here. It should be noted that, unlike the aforementioned implementation methods, in this implementation method, surface fitting can be performed only by referring to the three-dimensional points within the voxel to obtain the second surface. Of course, in order to make the second surface as close as possible to the real surface of the scanned target, surface fitting can also be performed using the three-dimensional points contained within the voxel and the three-dimensional points contained within the neighboring voxels, as in the aforementioned implementation methods, to obtain the second surface.

[0030] In a specific implementation scenario, after obtaining the estimated normals at each 3D point within a voxel, the angles between these normals and the camera normals when the scanner acquired the corresponding 3D points can be further calculated, serving as a second angle. Based on this, it can be determined whether to select a 3D point as a visible point based on the second angle corresponding to the 3D point within the voxel. For example, if the second angle corresponding to the 3D point within the voxel is less than (or not greater than) an angle threshold, the 3D point can be selected as a visible point; conversely, if the second angle corresponding to the 3D point within the voxel is greater than (or not less than) an angle threshold, the 3D point can not be selected as a visible point. Based on this, a normal visibility score can be obtained based on the proportion of visible points within the voxel to the 3D points contained in the voxel. It should be noted that the normal visibility score can be positively correlated with the proportion. That is, the larger the proportion of visible points within the voxel to the 3D points contained in the voxel, the larger the normal visibility score of the voxel; conversely, the smaller the proportion of visible points within the voxel to the 3D points contained in the voxel, the smaller the normal visibility score of the voxel. The above method determines whether to select a 3D point as a visible point based on the second included angle corresponding to the 3D point within the voxel. Then, based on the proportion of visible points within the voxel to each 3D point within the voxel, a normal visibility score is obtained. In this way, visible points can be identified through the second included angle, and blind spots caused by viewing angle limitations can be identified.

[0031] In a specific implementation scenario, please refer to [the relevant documentation] for easier understanding. Figure 2b , Figure 2b This is a schematic diagram illustrating an embodiment of the normal visibility measurement method of this application. Figure 2b As shown, the top left diagram represents the real-time point cloud (where pink dots represent 3D points), the green lines connecting the pink dots in the top right diagram represent the fitting plane, and the vertical lines of the green lines represent the normal vectors. In the bottom left diagram, the pink lines at the pink dots represent the estimated normals at the 3D points, and the black lines in the bottom right diagram represent the camera normals (the black-filled rectangles represent the camera). The angles between the estimated normals and camera normals at the three leftmost 3D points are significantly smaller than the angles between the estimated normals and camera normals at the other 3D points to their right. Clearly, the three leftmost 3D points can be considered visible points, while the other 3D points to their right can be considered "invisible points" (i.e., blind spots caused by cloud perspective limitations), potentially indicating insufficient scanning (i.e., further scanning may be needed). Of course, Figure 2b The example shown is only one possible scenario in practical application. Other possible scenarios are not limited here, nor will they be listed one by one.

[0032] Step S13: The voxel density score and normal visibility score based on voxels are fused to obtain the comprehensive voxel integrity score.

[0033] In one implementation scenario, as a possible approach, to fuse voxel density scores and normal visibility scores, averaging or weighted averaging operations can be performed on the voxel density scores and normal visibility scores to obtain a comprehensive voxel integrity score. It should be noted that when using weighted averaging for fusion, the weighting factors for both voxel density and normal visibility scores can be preset according to actual application needs. For example, when prioritizing point cloud density, the weighting factor for voxel density scores can be set larger than that for normal visibility scores (e.g., the former set to 0.6, the latter to 0.4); conversely, when prioritizing surface continuity, the weighting factor for voxel density scores can be set smaller than that for normal visibility scores (e.g., the former set to 0.4, the latter to 0.6). Of course, the above examples are only one possible way to set the weighting factors of voxel density score and normal visibility score when the fusion operation is achieved by weighted averaging. Other possible settings are not limited here, nor will they be listed one by one.

[0034] In another implementation scenario, as a possible approach, distinct from the aforementioned implementation, in order to fuse voxel density scores and normal visibility scores, the fusion operation can also be achieved by performing multi-level judgments on the voxel density scores and normal visibility scores to obtain a comprehensive voxel integrity score. For example, we can first combine voxel density scores to mark all low-density voxels (e.g., voxels with voxel density scores below the density score threshold) as high-risk. Then, for low-density voxels, we can adjust their weights based on their normal visibility scores (e.g., the larger the normal visibility score of a low-density voxel, the larger its weight factor compared to the voxel density score; conversely, the smaller the normal visibility score, the smaller its weight factor). Based on this, we can weight the normal visibility score and voxel density score based on the weight factors of the normal visibility score and the voxel density score, respectively, to obtain the comprehensive integrity score of the low-density voxels. In addition, for voxels other than low-density voxels, we can use fusion operations such as averaging or weighted averaging as described in the above embodiments to obtain the comprehensive integrity score of these voxels.

[0035] It should be noted that the above examples are only a few possible ways to fuse voxel density scores and normal visibility scores in practical applications. Other possible fusion methods are not limited here, nor will they be listed one by one.

[0036] Step S14: Based on the comprehensive integrity score of each voxel, suggest at least the target area to be scanned in the real-time point cloud.

[0037] In a common implementation scenario, as one possible approach, to suggest target areas for further scanning on a real-time point cloud, the overall integrity score of a voxel can be used to map 3D points within the voxel to colors that match the overall integrity score (i.e., this can be considered a 3D heatmap). It should be noted that when the overall integrity score is below a threshold, it can be mapped to a target color (e.g., red). 3D points mapped to the same color form connected regions, and the target region is the connected region mapped to the target color (e.g., red). This method, by mapping 3D points within the voxel to colors that match the overall integrity score in the real-time point cloud, allows 3D points within connected regions with relatively low overall integrity scores to be mapped to the same target color, thereby suggesting target areas for further scanning on the real-time point cloud. This allows for scanning suggestions to be provided through color.

[0038] In a specific implementation scenario, the mapping relationship between the overall completeness score and color can be such that a lower overall completeness score corresponds to a warmer color, while a higher overall completeness score corresponds to a cooler color. Of course, the above example is merely one possible illustration of the mapping relationship between overall completeness score and color; the mapping relationship between overall completeness score and color is not limited here, nor will it be listed in detail. Furthermore, the scoring threshold can be set according to the actual application needs. For example, when high modeling accuracy is required, the scoring threshold can be set appropriately larger; conversely, when modeling accuracy requirements are relatively relaxed, the scoring threshold can be set appropriately smaller. The specific value of the scoring threshold is not limited here, nor will it be listed in detail.

[0039] In a specific implementation scenario, please refer to the relevant documents. Figure 2c , Figure 2c This is a schematic diagram of an embodiment of this application that displays a target region on a real-time point cloud. For example... Figure 2c As shown, the real-time point cloud is obtained by scanning the head sculpture with a scanner. The warmer the color mapped to the 3D points in the real-time point cloud, the lower the overall integrity score of the voxel containing the 3D point; conversely, the cooler the color mapped to the 3D points in the real-time point cloud, the higher the overall integrity score of the voxel containing the 3D point. Figure 2c As shown, two target areas (marked with numbers 1 and 2) are highlighted in red on the real-time point cloud as areas recommended for further scanning. Of course, Figure 2c The example shown is just one possible instance of suggesting additional scanning of the target area in a real-world application. Other possible suggestion methods are not limited here, nor will they be listed one by one.

[0040] In another implementation scenario, as a possible alternative, distinct from the aforementioned implementation, to suggest target areas for further scanning on the real-time point cloud, several connected regions can be formed in the real-time point cloud based on the comprehensive integrity score of each voxel. It should be noted that connected regions within the same connected region whose comprehensive integrity scores are within the same score range, and whose comprehensive integrity scores fall within a target score range, are highlighted with a preset flag. The target score range is a score range whose upper limit is lower than the score threshold. This method, by forming several connected regions on the real-time point cloud based on the comprehensive integrity scores of each voxel, and highlighting connected regions with lower comprehensive integrity scores as target regions with preset flags, allows for supplementary scanning suggestions through these flags.

[0041] In a specific implementation scenario, multiple score ranges can be pre-defined. Based on this, for each voxel, its overall completeness score can be determined to fall within a specific score range. Ultimately, all 3D points within all voxels falling within the same score range can be connected into a region. Furthermore, a score threshold can be used to check the upper (or lower) limit of each score range. If the value is below the threshold, it can be considered the target range. In this way, all 3D points within all voxels whose overall completeness scores fall within the target range can be connected into a region, which becomes the target region. This target region is then highlighted on the real-time point cloud using pre-defined salient markers (e.g., arrows, numbers, circles, etc.). It should be noted that the method for setting the score threshold can be found in the aforementioned description and will not be repeated here.

[0042] In a specific implementation scenario, please refer to the following: Figure 2c ,like Figure 2c The image shows that preset highlighting "numbers" can be used to suggest and indicate target areas for additional scanning. For example... Figure 2c The numbers "1" and "2" are preset highlight markers, and the two numbers represent the target areas that are suggested for additional scanning.

[0043] It should be noted that the examples above are merely a few possible examples of prompting for additional cleaning in practical applications, and other possible prompting methods are not limited here. For example, the target area for additional cleaning can be highlighted, or the target area can be suggested for additional cleaning via voice prompts, etc., without limitation. In addition, the direction of suggested additional cleaning can be suggested (e.g., which part to clean first, then which part to clean next, etc.). These will not be listed in detail here. Of course, in practical applications, the above prompting methods are not limited to using only one method; they can also be used in combination (e.g., ...). Figure 2c The combination of "color" and "mark" is used to provide hints, and there are no restrictions on the combination method.

[0044] In one implementation scenario, as one possible approach, after suggesting a target area for rescanning, the process can return to the aforementioned step of voxelizing the real-time point cloud acquired by the scanner to obtain a voxel mesh, iterating repeatedly until the comprehensive integrity score of each voxel determines that no suggested target area exists. Alternatively, as another possible approach, after suggesting a target area for rescanning, the process can wait for a preset period before returning to the aforementioned step of voxelizing the real-time point cloud acquired by the scanner to obtain a voxel mesh, iterating repeatedly until the comprehensive integrity score of each voxel determines that no suggested target area exists. Yet another possible approach, after suggesting a target area for rescanning, the scanner's pose can be checked for changes. If a change is detected, the process can return to the aforementioned step of voxelizing the real-time point cloud acquired by the scanner to obtain a voxel mesh, iterating repeatedly until the comprehensive integrity score of each voxel determines that no suggested target area exists. If no change is detected, the process can continue waiting until a change is detected. Of course, the above examples are merely a few possible instances of the scanner continuing to operate after suggesting a target area in practical applications. Other possible operating methods are not limited here, nor will they be listed in detail.

[0045] The above scheme performs voxelization on real-time point clouds acquired by a scanner to obtain a voxel mesh. The real-time point cloud contains several 3D points, and the voxel mesh contains several voxels. Based on the number of 3D points within each voxel, a voxel density score is obtained. Based on the estimated normals at the 3D points within each voxel, a voxel normal visibility score is obtained. The voxel density score and normal visibility score are then fused to obtain a comprehensive voxel integrity score. Based on the comprehensive integrity score of each voxel, at least the target area on the real-time point cloud that should be scanned is indicated. This is because by voxelizing the real-time point cloud and evaluating the voxel density and normal visibility of a single voxel... By scoring completeness across two dimensions—visibility and integrity—real-time point cloud integrity can be assessed without converting to a mesh model, improving the real-time performance of point cloud integrity assessment. Furthermore, the elimination of mesh model conversion reduces the computational burden of patching, enhancing point cloud scanning efficiency. Additionally, the comprehensive integrity score of each voxel directly suggests target areas for further scanning on the real-time point cloud, eliminating the need for users to repeatedly perform patching based on the suggested areas, thus minimizing scanning complexity. Therefore, this approach improves the real-time performance of point cloud integrity assessment, enhances point cloud scanning efficiency, and minimizes scanning complexity.

[0046] Please see Figure 3 , Figure 3This is a schematic diagram of the framework of an embodiment of the point cloud detection device 30 of this application. The point cloud detection device 30 includes: a point cloud voxel module 31, a voxel scoring module 32, a scoring fusion module 33, and a region prompting module 34. The point cloud voxel module 31 is used to voxelize the real-time point cloud acquired by the scanner to obtain a voxel mesh; wherein, the real-time point cloud contains a number of three-dimensional points, and the voxel mesh contains a number of voxels; the voxel scoring module 32 is used to obtain a voxel density score based on the number of three-dimensional points within the voxel, and to obtain a voxel normal visibility score based on the estimated normal at the three-dimensional points within the voxel; the scoring fusion module 33 is used to fuse the voxel density score and the normal visibility score to obtain a voxel comprehensive integrity score; the region prompting module 34 is used to prompt at least the target area on the real-time point cloud that is recommended for additional scanning based on the comprehensive integrity score of each voxel.

[0047] In the above scheme, the point cloud detection device 30 performs voxelization on the real-time point cloud acquired by the scanner to obtain a voxel mesh. The real-time point cloud contains several 3D points, and the voxel mesh contains several voxels. Based on the number of 3D points within a voxel, a voxel density score is obtained, and based on the estimated normals at the 3D points within the voxel, a voxel normal visibility score is obtained. The voxel density score and normal visibility score are then fused to obtain a comprehensive voxel integrity score. Based on the comprehensive integrity scores of each voxel, at least the target area on the real-time point cloud that should be scanned is indicated. This is because by voxelizing the real-time point cloud and analyzing the voxel density of a single voxel... By scoring completeness along two dimensions—visibility and normal—real-time point cloud integrity can be assessed without converting to a mesh model, improving the real-time performance of point cloud integrity assessment. Furthermore, since conversion to a mesh model is unnecessary, the extensive computation required for patching is reduced, enhancing point cloud scanning efficiency. Additionally, the comprehensive completeness score of each voxel directly suggests target areas for further scanning on the real-time point cloud, eliminating the need for users to repeatedly perform patching based on the suggested target areas, thus minimizing the complexity of the scanning operation. Therefore, this approach improves the real-time performance of point cloud integrity assessment, enhances point cloud scanning efficiency, and minimizes the complexity of the scanning operation.

[0048] In some disclosed embodiments, the voxel scoring module 32 includes a first fitting submodule for fitting a first surface based on the three-dimensional points within the voxel and the three-dimensional points within each voxel in the neighborhood of the voxel; the voxel scoring module 32 includes a first estimation submodule for obtaining the normal vector of the first surface at any three-dimensional point within the voxel as an estimated normal; the voxel scoring module 32 includes a first included angle submodule for obtaining the angle between the estimated normal and the camera normal when the scanner acquires the three-dimensional points within the voxel as a first included angle; the voxel scoring module 32 includes a first scoring submodule for obtaining the normal visibility score of the voxel based on the first included angle and the point cloud distribution of the real-time point cloud at the voxel.

[0049] In some disclosed embodiments, the first scoring submodule includes a density estimation unit, used to obtain the point cloud density of the real-time point cloud at the voxel based on the point cloud distribution of the real-time point cloud at the voxel, and to obtain the degree of deviation of the estimated normal from the camera normal based on the first included angle; the first scoring submodule includes a normal visibility scoring unit, used to obtain a normal visibility score based on the point cloud density and the degree of deviation.

[0050] In some disclosed embodiments, the normal visibility score is positively correlated with the point cloud density; and / or, the normal visibility score is negatively correlated with the degree of deviation.

[0051] In some disclosed embodiments, the voxel scoring module 32 includes a second fitting submodule for fitting a second surface based at least on each three-dimensional point within the voxel; the voxel scoring module 32 includes a second estimation submodule for obtaining the normal vectors of the second surface at each three-dimensional point within the voxel, as estimated normals at the corresponding three-dimensional points; the voxel scoring module 32 includes a second included angle submodule for obtaining the angle between the estimated normals at the three-dimensional points within the voxel and the camera normals when the scanner acquires the corresponding three-dimensional points, as a second included angle; the voxel scoring module 32 includes a second scoring submodule for obtaining a voxel normal visibility score based on the second included angles corresponding to each three-dimensional point within the voxel.

[0052] In some disclosed embodiments, the second scoring submodule includes a visibility determination unit, used to determine whether to select a 3D point as a visible point based on a second included angle corresponding to the 3D point within the voxel; the second scoring submodule includes a scoring determination unit, used to obtain a normal visibility score based on the proportion of visible points within the voxel to each 3D point within the voxel.

[0053] In some disclosed embodiments, the normal visibility score is positively correlated with the scale value.

[0054] In some disclosed embodiments, the region prompting module 34 includes a first prompting submodule, used to map the three-dimensional points located within the voxels in the real-time point cloud to colors that match the comprehensive integrity score based on the voxel's overall integrity score; wherein, when the overall integrity score is lower than the score threshold, it is mapped to the target color, and three-dimensional points mapped to the same color form a connected region, and the target region is a connected region mapped with the target color; the region prompting module 34 includes a second prompting submodule, used to form several connected regions in the real-time point cloud based on the comprehensive integrity score of each voxel; wherein, the comprehensive integrity scores of each voxel located in the same connected region are within the same score value range, and the connected regions whose comprehensive integrity scores are within the target value range are indicated by a preset highlighting mark as target regions, and the target value range is a score value range whose upper limit is lower than the score threshold.

[0055] In some disclosed embodiments, the voxel side length of the voxel mesh is negatively correlated with the model resolution required for 3D modeling; and / or, the voxel density score is positively correlated with the number of 3D points within the voxel; and / or, when the number of 3D points within the voxel is not less than a number threshold, the voxel is considered a non-empty voxel, and the proportion of non-empty voxels in the voxel mesh is used as the overall scan coverage.

[0056] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the scanner of this application. The scanner 40 includes at least a memory 41 and a processor 42 coupled to each other. The memory 41 stores at least program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the above-described point cloud detection method embodiments. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here.

[0057] In one implementation scenario, scanner 40 may further include a scanning component (not shown) for scanning the target to acquire point cloud data. It should be noted that the scanning component may include a structured light unit and a visible light camera; alternatively, the scanning component may include a visible light camera and a depth camera; or, the scanning component may include a LiDAR. Of course, the above examples are merely a few possible implementations of the scanning component, and the specific construction of the scanning component is not limited here.

[0058] In one implementation scenario, the scanner 40 may also include a human-computer interaction component (not shown) for interacting with the user. It should be noted that the human-computer interaction component may include, but is not limited to, a display screen, buttons, a joystick, a speaker, a microphone, etc., and the specific construction of the human-computer interaction component is not limited here.

[0059] Specifically, processor 42 controls itself and memory 41 to implement the steps in any of the point cloud detection method embodiments described above. Processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.

[0060] The above scheme involves the scanner 40 performing voxelization on the real-time point cloud acquired by the scanner 40 to obtain a voxel mesh. The real-time point cloud contains several 3D points, and the voxel mesh contains several voxels. Based on the number of 3D points within each voxel, a voxel density score is obtained. Based on the estimated normals at the 3D points within each voxel, a voxel normal visibility score is obtained. The voxel density score and normal visibility score are then fused to obtain a comprehensive voxel integrity score. Based on the comprehensive integrity score of each voxel, at least the target area on the real-time point cloud that should be scanned is indicated. This is because by voxelizing the real-time point cloud and evaluating the voxel density and normal visibility of a single voxel... The use of two dimensions—normal visibility and integrity—to score the completeness of point clouds enables real-time point cloud integrity assessment without converting them to a mesh model. This improves the real-time performance of point cloud integrity assessment. Furthermore, since conversion to a mesh model is unnecessary, it reduces the significant computational costs associated with patching, thus improving point cloud scanning efficiency. Additionally, the comprehensive integrity score of each voxel directly suggests target areas for further scanning on the real-time point cloud, eliminating the need for users to repeatedly perform patching based on the suggested target areas, thereby minimizing the complexity of the scanning operation. Therefore, this approach improves the real-time performance of point cloud integrity assessment, enhances point cloud scanning efficiency, and minimizes the complexity of the scanning operation.

[0061] Please see Figure 5 , Figure 5This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor. The program instructions 51 are used to implement the steps in any of the above-described embodiments of the point cloud detection method.

[0062] The above scheme involves a computer-readable storage medium 50 voxelizing a real-time point cloud acquired by a scanner to obtain a voxel mesh. The real-time point cloud contains several 3D points, and the voxel mesh contains several voxels. Based on the number of 3D points within each voxel, a voxel density score is obtained. Based on the estimated normals at the 3D points within each voxel, a voxel normal visibility score is obtained. The voxel density score and normal visibility score are then fused to obtain a comprehensive voxel integrity score. Based on the comprehensive integrity scores of each voxel, at least one target area on the real-time point cloud that should be scanned is suggested. This is because voxelizing the real-time point cloud and analyzing the voxel density of individual voxels... The system uses two dimensions—degree and normal visibility—to score the completeness of point clouds, enabling real-time point cloud integrity assessment without converting them to a mesh model. This improves the real-time performance of point cloud integrity assessment. Furthermore, by eliminating the need for mesh model conversion, it reduces the significant computational burden of patching, thus improving point cloud scanning efficiency. Additionally, the comprehensive completeness score of each voxel directly suggests target areas for further scanning on the real-time point cloud, eliminating the need for users to repeatedly perform patching based on the suggested target areas, thereby minimizing the complexity of the scanning operation. Therefore, this approach improves the real-time performance of point cloud integrity assessment, enhances point cloud scanning efficiency, and minimizes the complexity of the scanning operation.

[0063] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0064] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0065] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0066] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0067] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0068] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0069] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A point cloud detection method, characterized in that, include: The real-time point cloud acquired by the scanner is voxelized to obtain a voxel mesh; wherein the real-time point cloud contains a number of three-dimensional points and the voxel mesh contains a number of voxels. Based on the number of the three-dimensional points within the voxel, the voxel density score is obtained, and based on the estimated normals at the three-dimensional points within the voxel, the voxel normal visibility score is obtained. The overall integrity score of the voxel is obtained by fusing the voxel density score and normal visibility score of the voxel. Based on the comprehensive integrity score of each voxel, at least the target area to be scanned on the real-time point cloud is suggested.

2. The method according to claim 1, characterized in that, The process of obtaining the normal visibility score of the voxel based on the estimated normal at the 3D point within the voxel includes: A first surface is obtained by fitting the three-dimensional points within the voxel and the three-dimensional points within each voxel in the neighborhood of the voxel. Obtain the normal vector of the first surface at any three-dimensional point within the voxel, as the estimated normal vector; The angle between the estimated normal and the camera normal when the scanner acquires the three-dimensional points within the voxel is obtained as the first angle. Based on the first included angle and the point cloud distribution of the real-time point cloud at the voxel, the normal visibility score of the voxel is obtained.

3. The method according to claim 2, characterized in that, The process of obtaining the normal visibility score of the voxel based on the first included angle and the point cloud distribution of the real-time point cloud at the voxel includes: Based on the point cloud distribution of the real-time point cloud at the voxel, the point cloud density of the real-time point cloud at the voxel is obtained, and based on the first included angle, the degree of deviation of the estimated normal from the camera normal is obtained. The normal visibility score is obtained based on the point cloud density and the degree of deviation.

4. The method according to claim 3, characterized in that, The normal visibility score is positively correlated with the point cloud density; And / or, the normal visibility score is negatively correlated with the degree of deviation.

5. The method according to claim 1, characterized in that, The process of obtaining the normal visibility score of the voxel based on the estimated normal at the 3D point within the voxel includes: The second surface is obtained by fitting at least based on each of the three-dimensional points within the voxel; Obtain the normal vectors of the second surface at each of the three-dimensional points within the voxel, and use them as estimated normals at the corresponding three-dimensional points; The angle between the estimated normal at the 3D point within the voxel and the camera normal when the scanner acquires the corresponding 3D point is obtained, and this angle is used as the second angle. The normal visibility score of the voxel is obtained based on the second included angle corresponding to each of the three-dimensional points within the voxel.

6. The method according to claim 5, characterized in that, The process of obtaining the normal visibility score of the voxel based on the second included angle corresponding to each of the three-dimensional points within the voxel includes: Based on the second included angle corresponding to the three-dimensional point within the voxel, determine whether to select the three-dimensional point as a visible point; The normal visibility score is obtained based on the proportion of the visible points within the voxel to the proportion of each of the three-dimensional points within the voxel.

7. The method according to claim 6, characterized in that, The normal visibility score is positively correlated with the ratio value.

8. The method according to claim 1, characterized in that, The comprehensive integrity score based on each voxel indicates at least one of the target areas for further scanning on the real-time point cloud: Based on the comprehensive integrity score of the voxels, the real-time point cloud maps the three-dimensional points located within the voxels to colors that match the comprehensive integrity score; wherein, when the comprehensive integrity score is lower than the score threshold, it is mapped to the target color, and three-dimensional points mapped to the same color form a connected region, and the target region is a connected region mapped to the target color. Based on the comprehensive integrity score of each voxel, several connected regions are formed in the real-time point cloud; wherein, the comprehensive integrity scores of each voxel located in the same connected region are within the same score value range, and the connected regions whose comprehensive integrity scores are within a target value range are marked with a preset highlighting mark as the target region, and the target value range is a score value range whose upper limit is lower than the score threshold.

9. The method according to any one of claims 1 to 8, characterized in that, The voxel side length of the voxel mesh is negatively correlated with the model resolution required for 3D modeling; And / or, the voxel density score of the voxel is positively correlated with the number of three-dimensional points within the voxel; And / or, if the number of the three-dimensional points within the voxel is not less than a number threshold, the voxel is considered a non-empty voxel, and the proportion of the non-empty voxels in the voxel grid is used as the overall scan coverage.

10. A point cloud detection device, characterized in that, include: The point cloud voxel module is used to voxelize real-time point clouds acquired by a scanner to obtain a voxel mesh; wherein, the real-time point cloud contains a number of three-dimensional points, and the voxel mesh contains a number of voxels. The voxel scoring module is used to obtain the voxel density score of the voxel based on the number of the three-dimensional points within the voxel, and to obtain the voxel normal visibility score based on the estimated normal at the three-dimensional points within the voxel. The scoring fusion module is used to fuse the voxel density score and normal visibility score of the voxel to obtain the comprehensive integrity score of the voxel. The region suggestion module is used to suggest at least the target region to be scanned in the real-time point cloud based on the comprehensive integrity score of each voxel.

11. A scanner, characterized in that, It includes at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the point cloud detection method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the point cloud detection method according to any one of claims 1 to 9.

Images