Point cloud processing method, device, equipment, and storage medium

Abstract

This application provides a point cloud processing method, device, equipment, and storage medium. The method includes the steps of: acquiring a current frame point cloud collected by a point cloud collection device and a global model, the global model being obtained by integrating past frame point clouds collected by the point cloud collection device; assigning weights to each 3D point in the global model, the weights of each 3D point representing the probability that the 3D point is a noise point or a target object; projecting the current frame point cloud onto the global model; casting several rays between the projected current frame point cloud and the point cloud collection device to determine target 3D points in the global model through which the rays pass; reducing the weights of the target 3D points; and, if the reduced weights of the target 3D points satisfy a noise point removal condition, removing the target 3D points. This allows for accurate noise point removal.

Description

[Technical Field]

[0001] The present application belongs to the technical field of point clouds, and in particular to a point cloud processing method, apparatus, electronic device and computer-readable storage medium.

[0002] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority based on a Chinese application bearing application number 202310208654.2 and entitled "Point Cloud Processing Method, Apparatus, Device and Storage Medium" filed with the China Patent Office on February 28, 2023, the entire contents of which are incorporated herein by reference. [Background technology]

[0003] 3D scanning primarily involves scanning the spatial shape, structure, and color of an object to obtain the spatial coordinates of the object's surface (e.g., point cloud data). Its key significance is that it can convert the object's three-dimensional information into digital signals that can be directly processed by a computer, providing a simple and quick method for digitizing real objects. However, during the 3D scanning process, invalid objects other than the target object may also be scanned, resulting in noise data (or noise points in the point cloud). This invalid noise data may affect subsequent processing, so the scanned noise data must be removed. Summary of the Invention

[0004] In view of the above, the present application provides a point cloud processing method, apparatus, electronic device and computer-readable storage medium.

[0005] The method of the present application can be used in 3D scanners such as oral scanners, face scanners, industrial scanners, professional scanners, handheld scanners, and fixed scanners, and can realize 3D reconstruction of objects and scenes such as oral cavity, teeth, face, human body, industrial products, industrial equipment, cultural relics, artworks, prosthetic limbs, medical instruments, and architecture.

[0006] Specifically, the present application is realized by the following technical solutions:

[0007] In a first aspect of an embodiment of the present application, there is provided a point cloud processing method, the point cloud processing method including the steps of: acquiring a current frame point cloud collected by a point cloud collecting device and a global model, the global model being obtained by integrating past frame point clouds collected by the point cloud collecting device, assigning weights to each 3D point in the global model, the weights representing the probability that the 3D point is a noise point (noise point or cluttered point) or a target object; projecting the current frame point cloud onto the global model, and then determining target 3D points in the global model through which the rays pass by casting several rays between the projected current frame point cloud and the point cloud collecting device; reducing the weights of the target 3D points; and deleting the target 3D points if the reduced weights of the target 3D points satisfy a noise point deletion condition.

[0008] Optionally, the method further includes a step of determining initial weights of three-dimensional points in the current frame point cloud, and after projecting the current frame point cloud onto a global model, further includes a step of updating weights of three-dimensional points of the current frame point cloud within the projection area of ​​the global model using the initial weights of the three-dimensional points in the current frame point cloud.

[0009] Optionally, in the current frame point cloud, the step of determining initial weights of 3D points in the current frame point cloud includes: for each 3D point in the current frame point cloud, obtaining a first adjustment coefficient based on the collected information of the 3D point; and using the first adjustment coefficient to adjust a reference weight to obtain an initial weight of the 3D point, wherein the first adjustment coefficient and the initial weight have a positive correlation.

[0010] Optionally, the collected information of each of the three-dimensional points includes at least one of the following: a distance between the three-dimensional point and an optimal depth of field of the point cloud collection device; a distance between the three-dimensional point and a center of the field of view of the point cloud collection device; a distance between the three-dimensional point and a ray projected by the point cloud collection device for obtaining the three-dimensional point by interpolation; or a difference between normal information of the three-dimensional point and normal information of an adjacent three-dimensional point; and a first adjustment coefficient of the three-dimensional point and the collected information of the three-dimensional point are negatively correlated.

[0011] Optionally, the step of determining initial weights of 3D points in the current frame point cloud includes: for each 3D point in the current frame point cloud, obtaining a first adjustment coefficient based on collected information of the 3D point; performing an object identification process on the current frame point cloud and then obtaining a second adjustment coefficient based on the object identification result corresponding to the 3D point; and adjusting a reference weight by the first adjustment coefficient and the second adjustment coefficient to obtain an initial weight of the 3D point, wherein the second adjustment coefficient and the initial weight are positively correlated, and the second adjustment coefficient of a 3D point whose object identification result is a target object is greater than the second adjustment coefficient of a 3D point whose object identification result is a noise point.

[0012] Optionally, a starting point of the light ray is one of a first position and a second position, and an ending point of the light ray is the other of the first position and the second position, wherein the first position comprises one of a position of the current frame point cloud after the projection or a result of combining a predetermined error distance with the position of the current frame point cloud after the projection, and the second position comprises one of a position of a lens of the point cloud collection device when the current frame point cloud was collected or a result of combining a predetermined distance with the position of the current frame point cloud after the projection, wherein the predetermined distance is determined based on a depth of field when the point cloud collection device collected the current frame point cloud.

[0013] Optionally, there is a one-to-one correspondence between the several rays and several 3D points in the current frame point cloud, and the step of reducing the weight of the target 3D point includes determining a 3D point in the current frame point cloud corresponding to a ray that has passed through the target 3D point, and reducing the weight of the target 3D point using the weight of the 3D point in the global model.

[0014] Optionally, a space in which the global model is located is divided into a number of voxels, and the step of determining a target 3D point in the global model through which the ray has passed comprises: The method includes detecting a target voxel through which the ray passes and determining a target 3D point in the target voxel that intersects with the ray.

[0015] Optionally, the method further includes a step of performing weight accumulation based on the initial weight of the deleted target 3D point in the point cloud of each frame when the deleted target 3D point is included in the point cloud of at least one frame subsequently collected by the point cloud collection device, and a step of reconstructing the target 3D point in the global model when the accumulated weight of the deleted target 3D point satisfies a predetermined reconstruction condition.

[0016] Optionally, the 3D points in said global model are displayed in a color gradation manner in ascending or descending order of weight, and the 3D points are displayed in different colors depending on the weight.

[0017] Optionally, the noise point removal condition is that the weight of the target three-dimensional point after reduction is less than a weight threshold, and the point cloud processing method includes the steps of acquiring a weight threshold input by a user into a user interface, or in response to a scan scene command input by the user, acquiring multiple recommended weight threshold values ​​from pre-stored data based on the scan scene indicated by the scan scene command, displaying the multiple recommended weight threshold values ​​on the user interface, and determining a selected value from the multiple recommended weight threshold values ​​in response to a user selection operation, wherein the pre-stored data includes multiple recommended weight threshold values ​​for different scan scenes.

[0018] In a second aspect of an embodiment of the present application, there is provided a point cloud processing method, which includes the steps of acquiring a global model collected by a point cloud collection device, assigning weights to each of the 3D points in the global model, the weights representing the probability that the 3D points are noise points or target objects, and deleting target 3D points in the global model whose weights satisfy a noise point deletion condition.

[0019] Optionally, weights attached to 3D points in the global model are adjustable based on a point cloud processing method according to any one of the forms of the first aspect.

[0020] In a third aspect of an embodiment of the present application, a point cloud processing apparatus is provided. The point cloud processing device includes an acquisition module, a target 3D point determination module, a weight reduction module, and a target 3D point removal module. The acquisition module is configured to acquire a current frame point cloud and a global model collected by a point cloud collection device. The global model is obtained by integrating past frame point clouds collected by the point cloud collection device. Each 3D point in the global model is assigned a weight, and the weight of each 3D point represents a probability that the 3D point is a noise point or a target object. The target 3D point determination module is configured to project the current frame point cloud onto the global model, and then determine target 3D points in the global model through which the rays have passed by casting several rays between the projected current frame point cloud and the point cloud collection device. The weight reduction module is configured to reduce the weights of the target 3D points. The target 3D point removal module is configured to remove the target 3D points if the reduced weights of the target 3D points satisfy a noise point removal condition.

[0021] In a fourth aspect of an embodiment of the present application, there is provided an electronic device, the electronic device including a memory, a processor, and executable instructions stored in the memory and executable by the processor; When the executable instructions are executed by the processor, the steps of the method according to any one of the first aspects are implemented.

[0022] In a fifth aspect of an embodiment of the present application, there is provided a computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, effect the steps of the method according to any one of the first aspects.

[0023] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0024] The drawings described herein illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the principles of the disclosure. [Brief explanation of the drawings]

[0025] [Figure 1A] FIG. 2 is a schematic diagram illustrating a result of projecting a point cloud of one frame collected by a point cloud collection device at time T onto a global model according to an exemplary embodiment of the present application. [Figure 1B] 1 is a diagram showing an area scanned in the oral cavity by a point cloud collection device at time T according to an exemplary embodiment of the present application. [Figure 2A] FIG. 1 is a schematic diagram illustrating a result of projecting one frame of point cloud collected by a point cloud collection device at time T+n onto a global model according to an exemplary embodiment of the present application. [Figure 2B] 1 is a diagram showing an area scanned in the oral cavity by a point cloud collection device at time T+n according to an exemplary embodiment of the present application. [Figure 3] 1 is a schematic flow chart of a point cloud processing method according to an exemplary embodiment of the present application; [Figure 4] 1 is an exemplary schematic diagram of an electronic device according to an exemplary embodiment of the present application; [Figure 5] FIG. 1 is a schematic configuration diagram of a point cloud processing apparatus according to an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0026] Exemplary embodiments are described in detail herein, examples of which are illustrated in the drawings. When referring to the drawings in the following description, the same numbers in different drawings represent the same or similar elements unless otherwise noted. The embodiments described in the following exemplary embodiments are not all embodiments consistent with the present application. These are merely examples of apparatus and methods consistent with certain aspects of the present application as detailed in the claims.

[0027] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit the scope of this application. As used in this application and in the claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. Also, as used herein, the term "and / or" includes any one or all possible combinations of the associated listed items.

[0028] Although terms such as "first," "second," and "third" may be used in this application to describe various pieces of information, these terms are not intended to be limiting. These terms merely distinguish between the same types of information. For example, first information could also be referred to as second information, and similarly, second information could be referred to as first information, without departing from the scope of this application. Additionally, the word "if" as used herein may be interpreted as "when," "when," "if," or "determined to," depending on the context.

[0029] 3D scanning primarily involves scanning the spatial shape, structure, and color of an object to obtain the spatial coordinates of the object's surface (e.g., point cloud data). Its key significance is that it can convert the object's three-dimensional information into digital signals that can be directly processed by a computer, providing a simple and quick method for digitizing real objects. However, during the 3D scanning process, invalid objects other than the target object may also be scanned and generated as noise data (or noise points in the point cloud data). This invalid noise data may affect subsequent processing, so the scanned noise data must be removed before further processing.

[0030] For example, in the field of oral scanning (intraoral scanning or extraoral scanning), a 3D scanner can be used to scan a user's teeth and construct a three-dimensional model of the teeth, which can be used for occlusion testing, undercut processing, etc. In the process of scanning a user's teeth using a 3D scanner, in addition to valid data such as teeth and gums, invalid noise data such as the tongue side, lip side, buccal side, and intraoral medical instruments may also be scanned, and this invalid noise data may affect the subsequent tooth processing process, so it is necessary to remove this invalid noise data.

[0031] To address the problems in the related art, an embodiment of the present application provides a point cloud processing method. The point cloud processing method can obtain a current frame point cloud collected by a point cloud collection device and a global model. The global model is obtained by integrating past frame point clouds collected by the point cloud collection device. Each 3D point in the global model is assigned a weight, and the weight of each 3D point represents the probability that the 3D point is a noise point or a target object, i.e., the weight represents the reliability of the 3D point. The higher the weight, the higher the reliability, and the greater the probability that the 3D point is a target object. The lower the weight, the lower the reliability, and the greater the probability that the 3D point is a noise point.

[0032] When collecting the current frame point cloud using a point cloud collection device, it is assumed that all the collected current frame point cloud is valid data and that no other point cloud data exists between the current frame point cloud and the point cloud collection device. That is, it is assumed that no other objects theoretically exist between the point cloud collection device and the measured object (i.e., the current frame point cloud). Based on this assumption, the current frame point cloud can be used to remove noise data in the global model. The current frame point cloud is projected onto the global model, and several rays are cast between the projected current frame point cloud and the point cloud collection device to determine the target 3D points in the global model through which the rays pass. The target 3D points may be noise points between the current frame point cloud and the point cloud collection device that were mistakenly collected at a previous time. In order to avoid erroneous deletion, in this embodiment, the weight of the target 3D point is reduced. In this way, if the weight of the target 3D point after reduction satisfies the noise point removal condition, the target 3D point is removed. If the weight of the target 3D point after reduction does not satisfy the noise point removal condition, the target 3D point is not removed. The point cloud collection device collects the next frame of point cloud and repeats the above process. In this way, the noise point data can be accurately removed through multiple verifications, and the accuracy of the removal can be improved.

[0033] The point cloud processing method according to the embodiments of the present application is performed by an electronic device, including but not limited to a 3D scanner, a smartphone / mobile phone, a tablet computer, a personal digital assistant (PDA), a laptop, a desktop computer, a media content player, a video game station / system, a virtual reality system, an augmented reality system, a wearable device (e.g., a watch, glasses, gloves, headgear (e.g., a hat, a helmet, a virtual reality headset, an augmented reality headset, a head-mounted device (HMD), a headband), a pendant, an armband, a leg band, a shoe, a vest), a remote control, or any other device with computing capabilities.

[0034] Illustratively, the electronic device includes a processor and a memory, executable instructions executable by the processor are stored in the memory, and when the executable instructions are executed by the processor, a point cloud processing method according to an embodiment of the present application is realized.Illustratively, the electronic device includes a computer program product incorporated therein, and when the computer program product is executed by the electronic device, a point cloud processing method according to an embodiment of the present application is realized.

[0035] For example, a point cloud collection device is installed in the electronic device, or the electronic device and the point cloud collection device are independently connected to each other for communication.

[0036] The point cloud collection device includes, but is not limited to, laser radar, millimeter wave radar, stereo vision camera, or structured light depth camera.

[0037] A laser radar is configured to emit a laser pulse train toward a target scene, receive the laser pulse train reflected from the target, and generate a three-dimensional point cloud based on the reflected laser pulse train. In one example, the laser radar can determine the reception time of the reflected laser pulse train, for example, by detecting the rise time and / or fall time of an electrical signal pulse. In this way, the laser radar determines the distance from the detected object to the laser radar by calculating the time of flight (TOF) using the reception time information and the emission time of the laser pulse train. The laser radar is an autonomous light-emitting sensor that does not rely on illumination from a light source, is relatively susceptible to interference from ambient light, can operate normally even in a closed, dark environment, can contribute to the generation of high-precision three-dimensional models, and has wide applicability. The point cloud collection principle of millimeter-wave radar is similar to that of laser radar, so a detailed description will be omitted here.

[0038] Stereo vision cameras use the parallax principle to capture two images of a target scene from different positions, and calculate the positional difference between corresponding points in the two images to obtain three-dimensional geometric information and generate a three-dimensional point cloud. Stereo vision cameras have low hardware requirements, which reduces costs accordingly. A common CMOS (Complementary Metal Oxide Semiconductor) camera is sufficient, and as long as the lighting is appropriate, they can be used in both indoor and outdoor environments, making them somewhat applicable.

[0039] A structured light depth camera projects light rays with specific structural characteristics onto a target scene, collects them, and then acquires a 3D point cloud by collecting different image phase information according to the depth region of the subject. The structured light depth camera is also an autonomous light-emitting sensor that does not depend on illumination from a light source, has relatively little interference from ambient light, and can operate normally even in a dark, enclosed environment, contributing to high-precision 3D models and having wide applicability.

[0040] For example, the point cloud collection device can be used in 3D scanners such as oral scanners, face scanners, industrial scanners, professional scanners, handheld scanners, and fixed scanners, and can realize 3D reconstruction of objects and scenes such as alveolar ridges, teeth, faces, human bodies, industrial products, industrial equipment, cultural relics, artworks, prosthetic limbs, medical instruments, and buildings.

[0041] In an exemplary application scenario, for example, in the field of oral cavity scanning, a point cloud collection device can construct a three-dimensional model of a user's teeth by scanning the inside of the user's oral cavity and collecting data related to the user's teeth. As shown in FIGS. 1A and 1B, FIG. 1A illustrates a result of projecting a point cloud of one frame collected by the point cloud collection device at time T onto a global model. For ease of understanding, the point cloud of the frame is illustrated in image format in FIG. 1A. The image may be a two-dimensional image or a depth image with depth information, and this embodiment is not limited thereto. FIG. 1B illustrates a range scanned by the point cloud collection device at time T within the oral cavity. As shown in FIGS. 1A and 1B, since the point cloud collection device also scanned a finger at time T, the collected point cloud of one frame includes noise points related to the finger. After the point cloud collection device projects the point cloud of one frame collected at time T onto the global model, noise data such as the finger appears within the projection area of ​​the global model (the rectangular frame in FIG. 1A).

[0042] The point cloud collection device continues scanning inside the user's oral cavity, and during the scanning process, it uses the point cloud processing method according to the embodiment of the present application to remove noise data in real time. For example, at time T+n, the point cloud collection device scans the same position as at time T. Referring to Figures 2A and 2B, Figure 2A is a diagram showing the result of projecting one frame of point cloud collected by the point cloud collection device at time T+n onto the global model, and the point cloud of the frame is shown in image format in Figure 2B. Figure 2B is a diagram showing the area scanned inside the oral cavity by the point cloud collection device at time T+n. As shown in Figures 2A and 2B, no finger was scanned by the point cloud collection device at time T+n. n is an integer greater than 0. Based on the assumption that there are theoretically no other objects between the point cloud collection device and the object to be measured (i.e., the current frame point cloud), the point cloud processing method according to the embodiment of the present application can reduce the weight of the noise points related to the finger collected at time T in the global model, and if the same position is scanned multiple times, the weight of the noise points related to the finger can be reduced multiple times. If the reduced weight meets the noise point removal condition, the noise points related to the finger collected at past times in the global model can be removed, and multiple verifications can be realized to accurately remove the noise points.

[0043] Hereinafter, a point cloud processing method according to an embodiment of the present application will be described by way of example.

[0044] The point cloud collection device according to the embodiment of the present application needs to scan the same position in the scene to be detected multiple times, so that it can select and delete noise data scanned at the same position at a previous time based on the scan result at the current time. Referring to Figure 3, Figure 3 shows a schematic flowchart of a point cloud processing method. The method is executed by an electronic device. The method includes the following steps:

[0045] S101: Obtain a current frame point cloud and a global model collected by a point cloud collection device. The global model is obtained by integrating past frame point clouds collected by the point cloud collection device. Each 3D point in the global model is assigned a weight, and the weight of each 3D point represents the probability that the 3D point is a noise point or a target object.

[0046] S102: After projecting the current frame point cloud onto the global model, cast some rays between the projected current frame point cloud and the point cloud collection device to determine the target 3D points in the global model through which the rays pass.

[0047] S103: The weight of the target 3D point is reduced.

[0048] S104: If the weight of the target 3D point after reduction satisfies a noise point removal condition, remove the target 3D point.

[0049] In this embodiment, it is assumed that the collected current frame point cloud (current frame) is all valid data and that no other point cloud data exists between the current frame point cloud and the point cloud collection device. That is, it is assumed that no other objects theoretically exist between the point cloud collection device and the measured object (i.e., the current frame point cloud). Based on this assumption, the current frame point cloud can be used to delete noise data collected in the past between the current frame point cloud and the point cloud collection device in the deleted global model. The current frame point cloud is projected onto the global model, and several rays are cast between the projected current frame point cloud and the point cloud collection device to determine the target 3D points in the global model through which the rays pass. The target 3D points may be noise points between the current frame point cloud and the point cloud collection device that were collected erroneously in the past. In order to avoid erroneous deletion, in this embodiment, the weight of the target 3D point is reduced, and the weight is reduced multiple times based on the fact that the ray passes through the target 3D point multiple times. If the weight of the target 3D point after reduction meets the noise point removal condition, the target 3D point is removed. In this way, the noise point data can be accurately removed through multiple verifications, and the accuracy of the removal can be improved.

[0050] In some embodiments, the deletion operation on the target 3D point can be implemented in multiple ways. Specific examples include, but are not limited to, the following: In one implementation, the target 3D point is separated from the global model, and only the global model after the target 3D point is separated is displayed on the display interface. In this implementation, the deletion operation selectively retains the display result, rather than completely deleting the target 3D point, i.e., only deleting the target 3D point from the display interface. In another implementation, the target 3D point is completely deleted directly from the source data. In another implementation, the target 3D point is separated from the global model, and the transparency of the separated target 3D point is increased, for example, displayed semi-transparently, thereby achieving the effect of prominently displaying the remaining global model on the display interface. In another implementation, the target 3D point is deleted from an intermediate product of the reconstructed model or the reconstructed model, so that the target 3D point is not displayed on the display interface.

[0051] For example, the noise point removal condition is that the weight of the three-dimensional point is less than a predetermined threshold, and the predetermined threshold can be specifically set according to the actual application scenario.

[0052] Exemplarily, the global model is obtained by integrating point clouds of past frames collected by the point cloud collection device. In one example, the global model includes several 3D points. In another example, the global model further includes triangular information configured by three-dimensional connections in addition to the several 3D points. In other words, the global model may be a mesh model.

[0053] In some embodiments, after obtaining the current frame point cloud collected by the point cloud collection device, the electronic device further needs to determine the initial weights of the 3D points in the current frame point cloud.

[0054] In one possible embodiment, it is assumed that all three-dimensional points in the current frame point cloud collected by the point cloud collection device are valid data, that is, the three-dimensional points in the current frame point cloud are assigned a predetermined initial weight, and all three-dimensional points in the current frame point cloud have the same initial weight.

[0055] In another possible embodiment, the initial weight of a 3D point is determined based on the reliability of depth information of the 3D point in the current frame point cloud, and the higher the reliability, the higher the initial weight. The reliability of the depth information of the 3D point is reflected by the collected information of the 3D point. Exemplarily, for each 3D point in the current frame point cloud, the electronic device obtains a first adjustment coefficient based on the collected information of the 3D point, and adjusts the base weight using the first adjustment coefficient to obtain the initial weight of the 3D point. The first adjustment coefficient and the initial weight have a positive correlation, i.e., the larger the first adjustment coefficient, the larger the initial weight, and the smaller the first adjustment coefficient, the smaller the initial weight. In one example, the initial weight of the 3D point is the product of the first adjustment coefficient and the base weight. In another example, the initial weight of the 3D point is the sum of the first adjustment coefficient and the base weight. In this embodiment, the specific adjustment method for adjusting the base weight using the first adjustment coefficient is not limited and can be specifically set according to actual application scenarios.

[0056] For example, the collected information of each of the 3D points may include, but is not limited to, at least one of the following: a distance between the 3D point and the optimal depth of field of the point cloud collection device, a distance between the 3D point and the center of the field of view of the point cloud collection device, a distance between the 3D point and a ray of light projected by the point cloud collection device for obtaining the 3D point by interpolation, or a difference between normal information of the 3D point and normal information of an adjacent 3D point. For example, if the point cloud collection device is a structured light camera, the projected light is structured light, and if the point cloud collection device is a laser radar, the projected light is laser light.

[0057] Through practice, it can be determined that: (1) the depth of a 3D point collected within the depth of field of a point cloud collection device is more reliable, so the smaller the distance between the 3D point and the optimal depth of field, the larger the first adjustment coefficient; and the larger the distance between the 3D point and the optimal depth of field, the smaller the first adjustment coefficient. (2) the depth of a 3D point collected at the center of the field of view of a point cloud collection device is more reliable, so the smaller the distance between the 3D point and the center of the field of view, the larger the first adjustment coefficient; and the larger the distance between the 3D point and the center of the field of view, the smaller the first adjustment coefficient. (3) The closer a 3D point is to the ray of light projected by the point cloud collection device to obtain the 3D point by interpolation, the more reliable the depth of the 3D point is, so the smaller the distance between the 3D point and the ray of light projected by the point cloud collection device to obtain the 3D point by interpolation, the larger the first adjustment coefficient, and vice versa. (4) The smaller the degree of depth step (sudden change in depth), the more reliable the depth of the 3D point is, so the smaller the difference between the normal information of the 3D point and the normal information of the adjacent 3D point, the larger the first adjustment coefficient.

[0058] In other words, the first adjustment coefficient of the three-dimensional point is negatively correlated with any one of the distance between the three-dimensional point and the depth of field, the distance between the three-dimensional point and the center of the field of view, the distance between the three-dimensional point and the ray projection position, and the difference between the normal information of the three-dimensional point and the normal information of the three-dimensional point adjacent to it.

[0059] Regarding (3) above, in one example, during 3D reconstruction, first, the object covered by the light beams projected by the point cloud collection device is reconstructed to obtain some sparse 3D points, and then, based on the sparse 3D points, dense 3D points are obtained by interpolation. The interpolation may be point-based interpolation or ray-based interpolation (for example, by supplementing the number of stripes, line segments, or dot matrix numbers of structured light).

[0060] If the light beam projected by the point cloud collection device is a structured light pattern or laser light, the structured light pattern includes, but is not limited to, a structured light stripe, a structured light line segment, or a structured light dot matrix. In this case, the sparse three-dimensional points can be understood to be points in the structured light pattern (e.g., points in a structured light stripe, points in a line segment, or points in a dot matrix) or points in the laser light, and correspond to the intersection points between the actual points on the object and the light beams projected by the point cloud collection device, i.e., the light beam projection positions.

[0061] It can be understood that "the distance between a three-dimensional point and a light ray among the light rays projected by the point cloud collection device for obtaining the three-dimensional point by interpolation" may be the distance between a three-dimensional point and a sparse three-dimensional point for obtaining the three-dimensional point by interpolation, or the distance between a three-dimensional point and a light ray at which a sparse three-dimensional point is located for obtaining the three-dimensional point by interpolation, or the distance between a three-dimensional point and a structured light stripe, structured light line segment, structured light dot matrix, or laser light for obtaining the three-dimensional point by interpolation.

[0062] In another possible embodiment, besides the reliability of the depth information of the 3D point in the current frame point cloud, the importance of the 3D point may also be taken into consideration, and the higher the importance of the 3D point, the higher the initial weight. The importance of the 3D point is reflected by the object identification result of the 3D point. If the 3D point is identified as a target object such as a tooth or gum, the 3D point is considered to be relatively important, and if the 3D point is identified as a noise point, the 3D point is considered to be unimportant.

[0063] For example, for each 3D point in the current frame point cloud, the electronic device obtains a first adjustment coefficient based on the collected information of the 3D point, performs an object identification process on the current frame point cloud, and then obtains a second adjustment coefficient based on the object identification result corresponding to the 3D point. Then, the electronic device adjusts the reference weight by the first adjustment coefficient and the second adjustment coefficient to obtain an initial weight for the 3D point. The second adjustment coefficient and the initial weight have a positive correlation, and the second adjustment coefficient for a 3D point whose object identification result is a target object is greater than the second adjustment coefficient for a 3D point whose object identification result is a noise point.

[0064] In one example, the initial weight of the 3D point is the product of the first adjustment coefficient, the second adjustment coefficient, and the base weight. In another example, the initial weight of the 3D point may be the sum of the first adjustment coefficient, the second adjustment coefficient, and the base weight. In this embodiment, the specific adjustment method for adjusting the base weight using the first adjustment coefficient and the second adjustment coefficient is not limited and can be specifically set according to actual application scenarios.

[0065] The target object may include one or more types, and the second adjustment coefficients of the 3D points representing different types of target objects may be the same or different, and can be specifically set according to the actual application scenario. For example, in the field of dental scanning, the target object includes teeth and gums, and the second adjustment coefficient of the 3D points whose object identification result is teeth is set to be larger than the second adjustment coefficient of the 3D points whose object identification result is gums.

[0066] In one example, the initial weight of the 3D point is the product of the first adjustment coefficient, the second adjustment coefficient, and the reference weight, where the value of the initial weight ranges from 0 to 1, the reference weight is 1, and the value ranges of the first adjustment coefficient and the second adjustment coefficient are from 0 to 1. For example, in the field of scanning teeth, when the target object includes teeth and gums, the second adjustment coefficient of the 3D points that are teeth is set to 1, the second adjustment coefficient of the 3D points that are gums is set to 0.5, and the second adjustment coefficient of the 3D points that are noise points is set to 0; of course, other values ​​may be used, and this embodiment is not limited thereto.

[0067] When performing object identification processing on the current frame point cloud, a pre-trained object identification model is used to process the current frame point cloud to obtain object identification results corresponding to the 3D points in the current frame point cloud. For example, the object identification model may be obtained by supervised training based on the point clouds of several frames and their object labels.

[0068] In some embodiments, the global model is obtained by integrating point clouds of past frames collected by the point cloud collection device. That is, each time the point cloud collection device collects a point cloud of a frame, the point cloud of the frame is integrated into the global model. The collection positions of the point cloud of the current frame and the point cloud of the past frame have an overlapping portion. That is, the current frame point cloud and the global model contain 3D points collected by the point cloud collection device from the same position. In this way, noise points in the global model can be filtered out based on the point cloud of the current frame.

[0069] Each of the 3D points in the global model is assigned a weight. In one example, the greater the weight of a 3D point in the global model, the greater the probability that the 3D point is a target object and the higher the confidence level of the 3D point, and the smaller the weight of a 3D point in the global model, the greater the probability that the 3D point is a noise point and the lower the confidence level of the 3D point.

[0070] After the current frame point cloud is projected onto the global model, an electronic device updates the weights of the 3D points of the current frame point cloud within the projection area of ​​the global model using the initial weights of the 3D points in the current frame point cloud. Illustratively, after a 3D point of the current frame point cloud is projected onto a certain projection position on the global model, if there is no 3D point at the projection position, the initial weight of the 3D point in the current frame point cloud is used as the weight of the 3D point at the projection position; if there is already a 3D point at the projection position, the initial weight of the 3D point in the current frame point cloud and the weight of the 3D point at the projection position are accumulated, and the accumulated weight is used as the updated weight of the 3D point at the projection position.

[0071] After the current frame point cloud is projected onto the global model, an electronic device casts several rays between the projected current frame point cloud and the point cloud collecting device to determine the target 3D points in the global model through which the rays pass. The electronic device may cast several rays between the projected current frame point cloud and the point cloud collecting device, or may cast several rays between a projection area corresponding to the projected current frame point cloud and the point cloud collecting device, which is not limited in this embodiment.

[0072] For example, the starting point of the ray is one of a first position and a second position, and the ending point of the ray is the other of the first position and the second position. The first position may be the position of the projected current frame point cloud, or may be the result of combining the position of the projected current frame point cloud with a predetermined error distance to prevent erroneous processing of the projected current frame point cloud. The specific value of the predetermined error distance can be specifically set according to actual application scenarios, and is not limited thereto in this embodiment.

[0073] The second position may be the position of the lens of the point cloud collection device when the current frame point cloud was collected, or may be the result of combining the position of the current frame point cloud after the projection with a predetermined distance to improve the efficiency of ray casting. The predetermined distance is determined based on the depth of field when the point cloud collection device collected the current frame point cloud, thereby preventing or reducing the casting of rays into areas where 3D points were not detected and improving the efficiency of ray casting.

[0074] In one example, the ray originates from a 3D point of the projected current frame point cloud and extends toward the position of the point cloud collection device. In another example, the ray originates from a coordinate of a 3D point of the projected current frame point cloud plus a predetermined error distance and extends toward the position of the point cloud collection device.

[0075] In some embodiments, after projecting some rays between the projected current frame point cloud and the point cloud collection device, an electronic device can determine the target 3D point in the global model through which the rays pass. Based on the assumption that there is no other object theoretically between the point cloud collection device and the measured object (i.e., the current frame point cloud), the target 3D point in the global model through which the rays pass may be a noise point between the current frame point cloud and the point cloud collection device that was erroneously collected at a past time.

[0076] In some possible embodiments, the electronics determines the target 3D point in the global model through which the ray passes by exhaustively detecting whether or not each 3D point in the global model intersects with any one of the rays.

[0077] In another possible embodiment, the space in which the global model is located is divided into several voxels. A voxel can be understood as the smallest unit of digital data in three-dimensional space, conceptually similar to a pixel, which is the smallest unit in two-dimensional space. A pixel is used for image data in two-dimensional computer images. A voxel represents a three-dimensional area using a scalar or vector. To reduce computational complexity and improve computational efficiency, in the process of determining a target 3D point, the electronic device first detects a target voxel among the several voxels through which the ray passes, and then determines a target 3D point in the target voxel that intersects with the ray. Because the area represented by one voxel is larger than one 3D point, the method of first determining the target voxel and then determining the target 3D point can significantly reduce the amount of computation and improve computational efficiency compared to the above-mentioned exhaustive method.

[0078] In some embodiments, after determining the target 3D point in the global model through which the ray passes, the electronics reduces the weight of the target 3D point, and if the reduced weight of the target 3D point satisfies the noise point removal condition, the target 3D point is removed. To avoid erroneous removal, in this embodiment, noise point detection is a continuous process, and if a 3D point is a noise point, the weight of the 3D point is reduced multiple times through multiple scans until the modified weight of the 3D point meets the noise point removal condition, thereby accurately removing the noise point through multiple verifications and improving the accuracy of the removal result. For example, the noise point removal condition is that the weight of the 3D point is less than a predetermined threshold.

[0079] In one possible embodiment, the electronics subtracts a predetermined value from the weight of the target 3D point if the smaller the weight, the greater the probability that the 3D point is a noise point, and further, if the reduced weight of the target 3D point is less than a weight threshold, the electronics deletes the target 3D point.

[0080] For example, at least one of the weight threshold and the predetermined value (i.e., reduction rate) can be set by the user according to actual needs, and the user may input a specific value based on his or her own experience, or may provide a recommended value that the user can select based on actual application scenarios.

[0081] In one example, the electronic device includes an input device (e.g., a user interface, a keyboard, a mouse, etc.), and the electronic device can receive the weight threshold and the predetermined value input by a user into the input device.

[0082] In another example, the electronic device responds to a scan scene command input by a user and retrieves multiple recommended weighting threshold values ​​and / or multiple recommended predetermined values ​​from pre-stored data based on the scan scene specified by the scan scene command, the pre-stored data including multiple recommended weighting threshold values ​​and / or multiple recommended predetermined values ​​for different scan scenes, and displays the multiple recommended weighting threshold values ​​and / or multiple recommended predetermined values ​​on a user interface for user adjustment, and determines a selected value from the multiple recommended weighting threshold values ​​and / or a selected value from multiple candidate predetermined values ​​in response to a user selection operation.

[0083] The magnitude of the weight threshold and the predetermined value influences the efficiency of the noise point removal and the accuracy of the noise point removal.

[0084] In one example, if the weight threshold is set relatively large and the predetermined value is set relatively small, the weight of the noise point cannot be reduced to the weight threshold in one go because the predetermined value is relatively small, so multiple comparisons are required to remove the noise point. For example, if the predetermined value is 0.3, the weight threshold is 0.1, and the weight of the noise point is 1, each time it is determined that a ray has passed through the noise point, the weight of the noise point is reduced by 0.3. In this case, at least three reductions are required to remove the noise point. The process of removing the noise point through multiple comparisons can contribute to improving the accuracy of the noise point removal, but it also reduces the efficiency of the noise point removal to a certain extent.

[0085] In another example, if the weight threshold is set relatively small and the predetermined value is set relatively large, the weight of the noise point can be reduced to below the weight threshold by one or a relatively small number of processing steps because the predetermined value is relatively large. In this way, the noise point can be directly deleted by one or a relatively small number of comparison steps, thereby improving the efficiency of noise point deletion.

[0086] Therefore, the user can set the weight threshold and the predetermined value according to the needs for the efficiency of noise point removal and the accuracy of noise point removal.

[0087] In another possible embodiment, the start point or end point of the ray is a 3D point in the current frame point cloud, that is, the above-mentioned projected rays and the above-mentioned 3D points in the current frame point cloud have a one-to-one correspondence. In this case, to more accurately adjust the weight of the target 3D point, the electronic device determines the 3D point in the current frame point cloud corresponding to the ray that passes through the target 3D point, and uses the weight of the 3D point in the global model to reduce the weight of the target 3D point. In this embodiment, the weight of the target 3D point is appropriately reduced according to the weight of the 3D point in the current frame point cloud that corresponds to the ray, thereby achieving accurate adjustment of the weight of the target 3D points passed by different rays.

[0088] In one example, if a larger weight indicates a higher probability that the 3D point is a target object, and a smaller weight indicates a higher probability that the 3D point is a noise point, the electronic device determines a 3D point in the current frame point cloud corresponding to a ray passing through the target 3D point, and then reduces the weight of the target 3D point using the weight of the 3D point in the global model. The larger the weight of the 3D point, the more the weight of the target 3D point is reduced. For example, the weight of the target 3D point may be subtracted from the weight of the target 3D point by the weight of the global model, or the weight of the target 3D point may be subtracted by a value corresponding to the weight of the 3D point in the global model, where the value and the weight of the 3D point are positively correlated. Furthermore, if the reduced weight of the target 3D point is less than a weight threshold, the electronic device deletes the target 3D point. The weight threshold can be set by a user according to actual needs. The user may input a specific value based on their own experience, or a recommended value may be provided for the user to select based on actual application scenarios.

[0089] That is, the greater the weight of a 3D point in the current frame point cloud corresponding to a ray that passed through a target 3D point, the greater the probability that the 3D point is a target object, and the greater the reduction in the weight of the target 3D point. Conversely, the smaller the weight of a 3D point in the current frame point cloud corresponding to a ray that passed through a target 3D point, the smaller the probability that the 3D point is a target object, and the smaller the reduction in the weight of the target 3D point.

[0090] In some embodiments, different 3D points in the global model have different weights, and therefore different colors are displayed for the different 3D points, allowing a user to grasp in real time the degree of probability that the different 3D points are target objects or noise points. For example, the 3D points in the global model may be displayed in a color gradation manner in ascending or descending order of weight. The display colors of the 3D points vary depending on the weight.

[0091] In some embodiments, a user can select a lock area in the global model according to actual needs, and the lock area indicates that any 3D points within the lock area are valid data required by the user, and no weight modification or deletion is required. After the current frame point cloud is projected onto the global model, if the projected current frame point cloud is within the lock area, no subsequent operations are required.

[0092] In some embodiments, since the deleted target 3D point may be mistakenly determined as a noise point even though it is not a noise point, resulting in erroneous deletion, if the deleted target 3D point is included in at least one frame of point cloud subsequently collected by the point cloud collection device, the electronic device performs weight accumulation based on the initial weight of the deleted target 3D point in the point cloud of each frame, and if the accumulated weight of the deleted target 3D point satisfies a predetermined reconstruction condition, reconstructs the target 3D point in the global model. In other words, if the deleted target 3D point is included in both the collection and scanning of the point cloud collection device, there is a very high probability that the deleted target 3D point is erroneously deleted, and in this case, the deleted target 3D point can be restored using the weight accumulation method.

[0093] In one example, if the accumulated weight of the deleted target 3D point exceeds a weight threshold, the electronic device reconstructs the deleted target 3D point in the global model. By accumulating the initial weight, the deleted target 3D point can be restored in the global model through multiple verifications.

[0094] The solutions according to the above embodiments can be combined with each other as long as there is no contradiction, and are not listed one by one in the embodiments of the present disclosure.

[0095] In some embodiments, a point cloud processing method obtains a global model collected by a point cloud collection device, and assigns a weight to each of the three-dimensional points in the global model, where the weight of each three-dimensional point represents the probability that the corresponding three-dimensional point is a noise point or a target object. If a target three-dimensional point exists in the global model and the weight satisfies a noise point removal condition, the target three-dimensional point is removed.

[0096] The weights may be adjusted as in the above example, adjusted by an AI intelligent system (e.g., reducing the weight of a 3D point if it is identified as a non-target object), adjusted based on patient history data, adjusted based on the scan scene (e.g., dental impression scan or patient intraoral scan), or adjusted based on user rules.

[0097] FIG. 4 is a schematic diagram of a device according to an exemplary embodiment. As shown in FIG. 4, the device includes a processor 402, an internal bus 404, a network interface 406, an internal memory 408, and a non-volatile memory 410. The device may also include other hardware required for its operation. One or more embodiments herein may be implemented in software, for example, by having the processor 402 load a corresponding computer program from the non-volatile memory 410 into the internal memory 408 and execute it. Of course, in addition to software implementation, one or more embodiments herein may also be implemented in other ways, such as by a logical device or a combination of software and hardware. That is, the execution entity of the following process flow is not limited to each logical unit, but may also be hardware or a logical device.

[0098] As shown in Fig. 5, the point cloud processing device can be used in the device shown in Fig. 4 to realize the technical solution in this specification. The point cloud processing device includes an acquisition module 501, a target 3D point determination module 502, a weight reduction module 503, and a target 3D point deletion module 504.

[0099] The acquisition module 501 is configured to acquire a current frame point cloud collected by a point cloud collection device and a global model. The global model is obtained by integrating past frame point clouds collected by the point cloud collection device. Each 3D point in the global model is assigned a weight, and the weight of each 3D point represents the probability that the corresponding 3D point is a noise point or a target object.

[0100] The target 3D point determination module 502 is configured to project the current frame point cloud onto the global model, and then cast several rays between the projected current frame point cloud and the point cloud collection device to determine the target 3D points in the global model through which the rays pass.

[0101] The weight reduction module 503 is configured to reduce the weight of the target 3D points.

[0102] The target 3D point removal module 504 is configured to remove the target 3D point if the reduced weight of the target 3D point satisfies a noise point removal condition.

[0103] In some embodiments, the system further includes a weight determination module in addition to the acquisition module 501, which is configured to determine initial weights of 3D points in the current frame point cloud, and the weight determination module is further configured to update weights of 3D points of the current frame point cloud within the projection region of the global model using the initial weights of the 3D points in the current frame point cloud after projecting the current frame point cloud onto a global model.

[0104] In some embodiments, the weight determination module is specifically configured to obtain, for each 3D point in the current frame point cloud, a first adjustment coefficient based on the collected information of the 3D point, and adjust a reference weight using the first adjustment coefficient to obtain an initial weight for the 3D point, wherein the first adjustment coefficient and the initial weight have a positive correlation.

[0105] The collected information of each of the three-dimensional points includes at least one of the following: the distance between the three-dimensional point and the optimal depth of field of the point cloud collection device; the distance between the three-dimensional point and the center of the field of view of the point cloud collection device; the distance between the three-dimensional point and a ray of light projected by the point cloud collection device to obtain the three-dimensional point by interpolation; or the difference between the normal information of the three-dimensional point and the normal information of an adjacent three-dimensional point.

[0106] The first adjustment coefficient of the three-dimensional point and the collected information of the three-dimensional point are negatively correlated.

[0107] In some embodiments, the weight determination module is specifically configured to: obtain, for each 3D point in the current frame point cloud, a first adjustment coefficient based on collected information of the 3D point; perform object identification processing on the current frame point cloud, then obtain a second adjustment coefficient based on the object identification result corresponding to the 3D point; adjust the reference weight by the first adjustment coefficient and the second adjustment coefficient to obtain an initial weight for the 3D point; the second adjustment coefficient and the initial weight have a positive correlation, and the second adjustment coefficient for a 3D point whose object identification result is a target object is greater than the second adjustment coefficient for a 3D point whose object identification result is a noise point.

[0108] In some embodiments, a starting point of the light beam is one of a first location and a second location, and an ending point of the light beam is the other of the first location and the second location.

[0109] The first position includes one of the projected current frame point cloud position or the result of combining the projected current frame point cloud position with a predetermined error distance.

[0110] The second position includes one of a position of a lens of the point cloud collection device when the current frame point cloud was collected, or a result of combining the position of the current frame point cloud after projection with a predetermined distance, where the predetermined distance is determined based on a depth of field when the point cloud collection device collected the current frame point cloud.

[0111] In some embodiments, the several rays and the several 3D points in the current frame point cloud have a one-to-one correspondence, and the weight reduction module 503 is specifically configured to determine a 3D point in the current frame point cloud corresponding to a ray passing through the target 3D point, and reduce the weight of the target 3D point by using the weight of the 3D point in the global model.

[0112] In some embodiments, the space where the global model is located is divided into several voxels, and the target 3D point determination module 502 is specifically configured to detect a target voxel through which the ray passes, and determine a target 3D point in the target voxel that intersects with the ray.

[0113] In some embodiments, the method further includes a reconstruction module, which is configured to, when the deleted target 3D point is included in the point cloud of at least one frame subsequently collected by the point cloud collection device, perform weight accumulation based on the initial weight of the deleted target 3D point in the point cloud of each frame, and, when the accumulated weight of the deleted target 3D point satisfies a predetermined reconstruction condition, reconstruct the target 3D point in the global model.

[0114] In some embodiments, the 3D points in the global model are displayed in a color gradation manner in ascending or descending order of weight, with the 3D points being displayed in different colors depending on the weight.

[0115] In some embodiments, the noise point removal condition is that the weight of the target 3D point after reduction is less than a weight threshold. The device further includes an interaction module configured to: acquire a weight threshold input by a user on a user interface or in response to a scan scene command input by the user, acquire a plurality of recommended weight threshold values ​​from pre-stored data based on the scan scene indicated by the scan scene command, display the plurality of recommended weight threshold values ​​on the user interface, and determine a selected value from the plurality of recommended weight threshold values ​​in response to a user selection operation, wherein the pre-stored data includes a plurality of recommended weight threshold values ​​for different scan scenes.

[0116] The process for realizing the functions and actions of each module in the above device can be specifically referred to the process for realizing the corresponding step in the above method, so the description will be omitted here.

[0117] Accordingly, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and executable instructions stored in the memory and executable by the processor.

[0118] When the executable instructions are executed by the processor, the steps of the above method are accomplished.

[0119] Exemplarily, the processor may include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0120] Illustratively, the memory includes at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a memory card (e.g., SD or DX memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, or an optical disk.

[0121] Accordingly, an embodiment of the present application further provides a computer program product, which includes a computer program, which, when executed by a processor, realizes the above method.

[0122] In an exemplary embodiment, a non-transitory computer-readable storage medium containing instructions is further provided, for example, a memory containing instructions, which can be executed by a processor of the device to implement the method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0123] The non-transitory computer-readable storage medium enables a terminal to perform the above method when instructions in the storage medium are executed by a processor of the terminal.

[0124] Embodiments of the subject matter and functional operations described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein can also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier, executed by or controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions can be encoded into an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which encodes and transmits information to a suitable receiver device for execution by the data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random access memory device, or a serial access memory device, or one or more combinations thereof.

[0125] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows may be performed by, for example, a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and an apparatus may be implemented as special purpose logic circuitry.

[0126] A computer suitable for executing a computer program includes, for example, a general-purpose and / or special-purpose microprocessor, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random-access memory. The basic components of a computer include a central processing unit, which performs or executes instructions, and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or the computer is operably coupled to the mass storage device to receive data from or transmit data to the mass storage device, or a combination of the two. However, a computer is not required to include such devices. A computer may also be incorporated into other devices, such as a mobile phone, a personal digital assistant (PDA), a portable audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash memory or USB flash drive.

[0127] Computer-readable media suitable for the storage of computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, CD-ROM, and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0128] Although the present specification contains many specific implementation details, these should not be construed as limiting the scope of the invention or the scope of protection sought, but are primarily intended to describe the features of specific embodiments of particular inventions. In this specification, some specific features described in multiple embodiments may be implemented in combination in a single embodiment. Also, various features described in a single embodiment may be implemented individually or in appropriate subcombinations in multiple embodiments. Furthermore, even if features function in specific combinations as described above and are initially intended to be protected as such, one or more features may be removed from the combination sought to be protected, and the combination sought to be protected may refer to a subcombination or a variation thereof.

[0129] Similarly, although operations are depicted in a particular order in the figures, this should not be construed as requiring that the operations be performed in the particular order or sequence shown, or that all of the illustrated operations be performed to achieve a desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above examples should not be construed as requiring such separation in all examples, and it should be understood that the described program components and systems may typically be integrated into a single software product or packaged into multiple software products.

[0130] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results. Furthermore, the processes depicted in the figures do not necessarily require performance in the particular order shown or by performing the sequence to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0131] The above is only a preferred embodiment of the present application, and does not limit the present application. As long as it does not deviate from the essence and spirit of the present application, all modifications, equivalent replacements, improvements, etc., fall within the protection scope of the present invention. [Industrial Applicability]

[0132] The point cloud processing method, device, equipment, and storage medium according to the present application can accurately delete noise point data through multiple verifications, improve the accuracy of deletion, and prevent erroneous deletion of data, making them industrially applicable.

Claims

1. A step of obtaining a current frame point cloud and a global model collected by a point cloud collecting device, the global model being obtained by integrating past frame point clouds collected by the point cloud collecting device, each 3D point in the global model being weighted, the weight of each 3D point representing the probability that the 3D point is a noise point or a target object; After projecting the current frame point cloud onto the global model, cast some rays between the projected current frame point cloud and the point cloud collection device to determine target 3D points in the global model through which the rays pass; reducing the weight of the target 3D points; removing the target 3D point if the weight after reduction of the target 3D point satisfies a noise point removal condition; Point cloud processing methods.

2. determining initial weights of the 3D points in the current frame point cloud; After projecting the current frame point cloud onto the global model, and updating weights of 3D points in the current frame point cloud within a projection region of the global model using initial weights of 3D points in the current frame point cloud. The point cloud processing method according to claim 1 .

3. In the current frame point cloud, determining initial weights of 3D points in the current frame point cloud includes: For each 3D point in the current frame point cloud, obtain a first adjustment coefficient based on the collected information of the 3D point; and adjusting the reference weights using the first adjustment coefficient to obtain initial weights for the three-dimensional points, wherein the first adjustment coefficients and the initial weights are positively correlated. The point cloud processing method according to claim 2 .

4. The collected information of each of the three-dimensional points includes at least one of the following: a distance between the three-dimensional point and the optimal depth of field of the point cloud collection device; a distance between the three-dimensional point and the center of the field of view of the point cloud collection device; a distance between the three-dimensional point and a ray for obtaining the three-dimensional point by interpolation among the rays projected by the point cloud collection device; or a difference between normal information of the three-dimensional point and normal information of the three-dimensional point adjacent thereto; The first adjustment coefficient of the three-dimensional point and the collected information of the three-dimensional point are negatively correlated; The point cloud processing method according to claim 3 .

5. The step of determining the initial weights of the 3D points in the current frame point cloud includes: For each 3D point in the current frame point cloud, obtain a first adjustment coefficient based on the collected information of the 3D point; performing an object identification process on the current frame point cloud, and then obtaining a second adjustment coefficient based on the object identification result corresponding to the 3D point; adjusting the reference weights by the first adjustment coefficient and the second adjustment coefficient to obtain initial weights for the 3D points, wherein the second adjustment coefficients and the initial weights have a positive correlation, and the second adjustment coefficients for the 3D points whose object identification results indicate that they are target objects are greater than the second adjustment coefficients for the 3D points whose object identification results indicate that they are noise points; The point cloud processing method according to claim 1 .

6. a starting point of the light beam is one of a first position and a second position, and an ending point of the light beam is the other of the first position and the second position; the first position comprises one of the position of the projected current frame point cloud or a result of combining the position of the projected current frame point cloud with a predetermined error distance; the second position includes one of a position of a lens of the point cloud collection device when the current frame point cloud was collected, or a result of combining the position of the projected current frame point cloud with a predetermined distance, and the predetermined distance is determined based on a depth of field when the point cloud collection device collected the current frame point cloud. The point cloud processing method according to claim 1 .

7. the number of rays corresponds one-to-one to the number of three-dimensional points in the current frame point cloud; The step of reducing the weight of the target 3D points includes: determining a 3D point in the current frame point cloud corresponding to a ray that has passed through the target 3D point, and reducing the weight of the target 3D point using the weight of the 3D point in the global model; The point cloud processing method according to any one of claims 1 to 6.

8. The space in which the global model is located is divided into several voxels, determining a target 3D point in the global model through which the ray passes, detecting a target voxel through which the ray passes and determining a target 3D point in the target voxel that intersects with the ray; The point cloud processing method according to any one of claims 1 to 6.

9. When the deleted target 3D point is included in the point cloud of at least one frame collected later by the point cloud collection device, performing weight accumulation based on the initial weight of the deleted target 3D point in the point cloud of each frame; If the accumulated weight of the deleted target 3D point satisfies a predetermined reconstruction condition, reconstructing the target 3D point in the global model. The point cloud processing method according to any one of claims 1 to 6.

10. The 3D points in the global model are displayed in a color gradation manner in order of increasing or decreasing weight, and the display color of the 3D points varies depending on the weight. The point cloud processing method according to any one of claims 1 to 6.

11. The noise point removal condition is that the weight of the target 3D point after reduction is less than a weight threshold; The point cloud processing method includes: obtaining a weight threshold value input by a user through a user interface, or in response to a scan scene command input by a user, obtaining a plurality of recommended values ​​of the weight threshold value from pre-stored data based on a scan scene indicated by the scan scene command; displaying a plurality of recommended weight threshold values ​​in a user interface; and determining a selected value from a plurality of recommended weighting threshold values ​​in response to a user's selection operation, wherein the pre-stored data includes a plurality of recommended weighting threshold values ​​for different scan scenes. The point cloud processing method according to any one of claims 1 to 6.

12. Acquiring a global model collected by a point cloud collection device, and assigning a weight to each of the three-dimensional points in the global model, the weight representing the probability that the three-dimensional point is a noise point or a target object; If there is a target 3D point in the global model whose weight satisfies a noise point removal condition, removing the target 3D point. Point cloud processing methods.

13. The method includes an acquisition module, a target 3D point determination module, a weight reduction module, and a target 3D point deletion module; The acquisition module is configured to acquire a current frame point cloud and a global model collected by a point cloud collection device, the global model being obtained by integrating past frame point clouds collected by the point cloud collection device, and each 3D point in the global model is assigned a weight, and the weight of each 3D point represents a probability that the 3D point is a noise point or a target object; The target 3D point determination module is configured to project the current frame point cloud onto the global model, and then cast some rays between the projected current frame point cloud and the point cloud collection device to determine the target 3D points in the global model through which the rays pass; the weight reduction module is configured to reduce weights of the target 3D points; the target 3D point removal module is configured to remove the target 3D point if the reduced weight of the target 3D point satisfies a noise point removal condition; Point cloud processing equipment.

14. a memory, a processor, and executable instructions stored in the memory and executable by the processor; When the executable instructions are executed by the processor, each step of the point cloud processing method according to any one of claims 1 to 12 is realized. electronic equipment.

15. A computer-readable storage medium having computer instructions stored therein, the computer instructions being executed by a processor to realize the steps of the point cloud processing method according to any one of claims 1 to 12. A computer-readable storage medium.

Images