Point cloud processing method, device, equipment and storage medium

Abstract

The present application provides a point cloud processing method, device, equipment and storage medium. The method includes: obtaining the current frame point cloud collected by the point cloud acquisition device and the global model; wherein the global model is obtained by fusion of the historical frame point cloud collected by the point cloud acquisition device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a noise point or a target object; after projecting the current frame point cloud into the global model, projecting a number of rays between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional point in the global model passed by the ray; reducing the weight of the target three-dimensional point; if the reduced weight of the target three-dimensional point meets the noise deletion condition, delete the target three-dimensional point. Accurately delete noise points.

Description

Technical Field

[0001] The present application relates to the field of point cloud technology, and in particular to a point cloud processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] 3D scanning is mainly used to scan the spatial shape, structure and color of an object to obtain the spatial coordinates of the object surface, such as point cloud data. Its significance lies in the ability to convert the three-dimensional information of an object into a digital signal that can be directly processed by a computer, providing a very convenient and fast means for the digitization of physical objects. However, during the 3D scanning process, invalid debris outside the target object may also be scanned in, forming miscellaneous data (or miscellaneous points in the point cloud). These invalid miscellaneous data may affect the subsequent processing process, so the scanned miscellaneous data needs to be deleted. Summary of the invention

[0003] In view of this, the present application provides a point cloud processing method, device, electronic device and computer-readable storage medium.

[0004] Specifically, the present application is implemented through the following technical solutions:

[0005] According to a first aspect of an embodiment of the present application, a point cloud processing method is provided, comprising:

[0006] Acquire the current frame point cloud collected by the point cloud collection device and the global model; wherein the global model is obtained by fusing the historical frame point clouds collected by the point cloud collection device; any three-dimensional point in the global model carries a weight, and the weight represents the possibility that the three-dimensional point is a clutter point or a target object;

[0007] After projecting the current frame point cloud into the global model, projecting a plurality of rays between the projected current frame point cloud and the point cloud acquisition device to determine a target three-dimensional point in the global model passed by the rays;

[0008] reducing the weight of the target three-dimensional point;

[0009] If the reduced weight of the target three-dimensional point satisfies the noise point deletion condition, the target three-dimensional point is deleted.

[0010] Optionally, it also includes: determining the initial weights of the three-dimensional points in the point cloud of the current frame;

[0011] After projecting the current frame point cloud into the global model, the method further includes:

[0012] The weights of the three-dimensional points in the current frame point cloud are updated within the projection area of ​​the global model using the initial weights of the three-dimensional points in the current frame point cloud.

[0013] Optionally, in the current frame point cloud, determining the initial weights of the three-dimensional points in the current frame point cloud includes:

[0014] For each three-dimensional point in the current frame point cloud, obtaining a first adjustment coefficient according to acquisition information of the three-dimensional point;

[0015] The reference weight is adjusted using the first adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the first adjustment coefficient is positively correlated with the initial weight.

[0016] Optionally, the acquisition information of each of the three-dimensional points includes at least one of the following: the distance between the three-dimensional point and the best depth of field of the point cloud acquisition device, the distance between the three-dimensional point and the center of the field of view of the point cloud acquisition device, the distance between the three-dimensional point and a light ray projected by the point cloud acquisition device to interpolate the three-dimensional point, or the difference between the normal information of the three-dimensional point and the normal information of its adjacent three-dimensional point;

[0017] The first adjustment coefficient of the three-dimensional point is negatively correlated with the collection information of the three-dimensional point.

[0018] Optionally, determining the initial weights of the three-dimensional points in the point cloud of the current frame includes:

[0019] For each three-dimensional point in the current frame point cloud, obtaining a first adjustment coefficient according to acquisition information of the three-dimensional point;

[0020] And, after performing object recognition processing on the current frame point cloud, obtaining a second adjustment coefficient according to the object recognition result corresponding to the three-dimensional point;

[0021] The reference weight is adjusted according to the first adjustment coefficient and the second adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the second adjustment coefficient is positively correlated with the initial weight, and the second adjustment coefficient of the three-dimensional point whose object recognition result is the target object is greater than the second adjustment coefficient of the three-dimensional point whose object recognition result is a noise point.

[0022] Optionally, the starting point of the light is one of a first position and a second position, and the ending point of the light is the other of the first position and the second position;

[0023] Wherein, the first position includes one of the following: the position of the projected current frame point cloud, or the result of the projected current frame point cloud position combined with a preset error distance;

[0024] The second position includes one of the following: the position of the lens of the point cloud acquisition device when acquiring the current frame point cloud, or the position of the projected current frame point cloud combined with a preset distance; the preset distance is determined according to the depth of field of the point cloud acquisition device when acquiring the current frame point cloud.

[0025] Optionally, the plurality of rays correspond one-to-one to a plurality of three-dimensional points in the point cloud of the current frame respectively;

[0026] The step of reducing the weight of the target three-dimensional point comprises:

[0027] A 3D point in the current frame point cloud corresponding to a light ray passing through the target 3D point is determined, and a weight of the target 3D point is reduced by using the weight of the 3D point in the global model.

[0028] Optionally, the space where the global model is located is divided into a number of voxels;

[0029] The step of determining the target three-dimensional point in the updated global model passed by the light comprises:

[0030] The target voxel passed by the light ray is detected, and the target three-dimensional point intersecting with the light ray in the target voxel is determined.

[0031] Optionally, also include:

[0032] If at least one frame of point cloud subsequently collected by the point cloud collection device contains the deleted target three-dimensional point, weight accumulation is performed based on the initial weight of the deleted target three-dimensional point in each frame of point cloud;

[0033] When the accumulated weight of the deleted target three-dimensional point meets a preset reconstruction condition, the target three-dimensional point is reconstructed in the global model.

[0034] Optionally, the three-dimensional points in the global model are displayed in a color gradient manner in the order of weight from large to small or from small to large; the display colors of the three-dimensional points indicated by different weights are different.

[0035] Optionally, the noise removal condition indicates that the reduced weight of the target three-dimensional point is less than a weight threshold;

[0036] The method further comprises:

[0037] Obtain the weight threshold value entered by the user in the interactive interface; or

[0038] In response to a scanning scene instruction input by a user, a plurality of recommended values ​​of a weight threshold are obtained from pre-stored data according to the scanning scene indicated by the scanning scene instruction; wherein the pre-stored data includes a plurality of recommended values ​​of the weight threshold under different scanning scenes;

[0039] Display multiple recommended values ​​of weight thresholds in the interactive interface;

[0040] In response to a selection operation by the user, a selected value is determined from a plurality of recommended values ​​of the weight threshold.

[0041] According to a second aspect of an embodiment of the present application, a point cloud processing method is provided, comprising:

[0042] Acquire a global model collected by a point cloud collection device; wherein any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object;

[0043] If there is a target three-dimensional point in the global model whose weight satisfies the noise deletion condition, the target three-dimensional point is deleted.

[0044] According to a third aspect of an embodiment of the present application, a point cloud processing device is provided, including:

[0045] An acquisition module is used to acquire the current frame point cloud collected by the point cloud acquisition device and a global model; wherein the global model is obtained by fusing the historical frame point clouds collected by the point cloud acquisition device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object;

[0046] A target three-dimensional point determination module is used to project a plurality of rays between the projected current frame point cloud and the point cloud acquisition device after projecting the current frame point cloud into the global model, so as to determine the target three-dimensional point in the global model passed by the rays;

[0047] A weight reduction module, used to reduce the weight of the target three-dimensional point;

[0048] The target three-dimensional point deletion module is used to delete the target three-dimensional point if the reduced weight of the target three-dimensional point meets the noise deletion condition.

[0049] According to a fourth aspect of an embodiment of the present application, there is provided an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0050] Wherein, when the processor executes the executable instructions, the steps in the method as described in any one of the first aspects are implemented.

[0051] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] In the disclosed embodiment, it can be assumed that all the collected current frame point cloud is valid data, and there is no other point cloud data between the current frame point cloud and the point cloud acquisition device. In other words, there is no other object between the point cloud acquisition device and the object to be measured (i.e., the current frame point cloud) in theory. Based on this assumption, the current frame point cloud can be used to delete the noise data between the current frame point cloud and the point cloud acquisition device collected at a historical moment in the global model; the current frame point cloud can be projected into the global model, and then a number of rays are projected between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional point in the global model that is passed by the rays. The target three-dimensional point may be a noise point between the current frame point cloud and the point cloud acquisition device that was mistakenly collected at a historical moment. In order to avoid mistaken deletion, the present embodiment reduces the weight of the target three-dimensional point. The weight can be reduced multiple times based on the situation that the target three-dimensional point is passed by the rays multiple times until the reduced weight of the target three-dimensional point meets the noise deletion condition, and the target three-dimensional point is deleted, so that the noise data can be accurately deleted after multiple verifications, thereby improving the accuracy of the deletion result.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0056] Figure 1A It is a schematic diagram of the projection result of a frame of point cloud collected by a point cloud collection device at time T on a global model shown in an exemplary embodiment of the present application.

[0057] Figure 1B It is the scanning range of the point cloud acquisition device in the oral cavity at time T shown in an exemplary embodiment of the present application.

[0058] Figure 2A It is a schematic diagram of the projection result of a frame of point cloud collected by a point cloud collection device at time T+n on the global model shown in an exemplary embodiment of the present application.

[0059] Figure 2B It is the scanning range of the point cloud acquisition device in the oral cavity at time T+n shown in an exemplary embodiment of the present application.

[0060] Figure 3 It is a flowchart of a point cloud processing method shown in an exemplary embodiment of the present application.

[0061] Figure 4 It is an exemplary schematic diagram of an electronic device shown in an exemplary embodiment of the present application.

[0062] Figure 5 It is a structural schematic diagram of a point cloud processing device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0064] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0065] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0066] 3D scanning is mainly used to scan the spatial shape, structure and color of an object to obtain the spatial coordinates of the object surface, such as point cloud data. Its significance lies in the ability to convert the three-dimensional information of an object into a digital signal that can be directly processed by a computer, providing a very convenient and fast means for the digitization of physical objects. However, during the 3D scanning process, invalid debris outside the target object may also be scanned in, forming miscellaneous data (or miscellaneous points in the point cloud data). These invalid miscellaneous data may affect the subsequent processing process, so the scanned miscellaneous data needs to be deleted before subsequent processing.

[0067] For example, in the field of oral scanning, a 3D scanner can be used to scan the user's teeth to build a 3D tooth model, which can be used for occlusion detection, undercut treatment, etc. In the process of scanning the user's teeth with a 3D scanner, in addition to valid data such as teeth and gums, invalid data such as the tongue, lip, cheek, and intraoral medical devices may also be scanned, and these invalid data may affect the subsequent tooth treatment process. Therefore, these invalid data need to be deleted.

[0068] In response to the problems in the related art, an embodiment of the present application provides a point cloud processing method, which can obtain a current frame point cloud collected by a point cloud acquisition device and a global model; wherein the global model is obtained by fusion of historical frame point clouds collected by the point cloud acquisition device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a debris point or a target object. In other words, the weight represents the confidence of the three-dimensional point; the higher the weight, the higher the confidence, and the greater the possibility that the three-dimensional point is a target object; the lower the weight, the lower the confidence, and the greater the possibility that the three-dimensional point is a debris point.

[0069] When the point cloud acquisition device acquires the current frame point cloud, it can be assumed that the acquired current frame point cloud is all valid data, and there is no other point cloud data between the current frame point cloud and the point cloud acquisition device. In other words, there is no other object between the point cloud acquisition device and the measured object (i.e., the current frame point cloud) in theory. Based on this assumption, the current frame point cloud can be used to delete the miscellaneous data in the global model; the current frame point cloud can be projected into the global model, and then a number of rays are projected between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional point in the global model that is passed by the rays. The target three-dimensional point may be a miscellaneous point between the current frame point cloud and the point cloud acquisition device that was mistakenly acquired at a historical moment. In order to avoid mistaken deletion, this embodiment reduces the weight of the target three-dimensional point, and then if the reduced weight of the target three-dimensional point meets the miscellaneous point deletion condition, the target three-dimensional point is deleted, and if the reduced weight of the target three-dimensional point does not meet the miscellaneous point deletion condition, the target three-dimensional point is not deleted, and the point cloud acquisition device acquires the next frame point cloud and continues the above processing, so that the miscellaneous point data can be accurately deleted in the case of multiple verifications, thereby improving the accuracy of the deletion result.

[0070] The point cloud processing method provided in the embodiments of the present application can be executed by electronic devices, including but not limited to smart phones / mobile phones, tablet computers, personal digital assistants (PDAs), laptop computers, desktop computers, media content players, video game stations / systems, virtual reality systems, augmented reality systems, wearable devices (e.g., watches, glasses, gloves, headwear (e.g., hats, helmets, virtual reality headsets, augmented reality headsets, head-mounted devices (HMDs), headbands), pendants, armbands, leg rings, shoes, vests), remote controls, or any other device with computing capabilities.

[0071] Exemplarily, the electronic device includes a processor and a memory, the memory stores executable instructions that can be run on the processor, and the processor implements the point cloud processing method provided in the embodiment of the present application when executing the executable instructions. Exemplarily, the electronic device integrates a computer program product, and the electronic device implements the point cloud processing method provided in the embodiment of the present application when executing the computer program product.

[0072] Exemplarily, the electronic device may be provided with a point cloud acquisition device; or the electronic device and the point cloud acquisition device are independent of each other and are communicatively connected.

[0073] Among them, the point cloud acquisition device includes but is not limited to laser radar, millimeter wave radar, binocular vision sensor or structured light depth camera.

[0074] The laser radar is used to emit a laser pulse sequence to the target scene, then receive the laser pulse sequence reflected from the target, and generate a three-dimensional point cloud based on the reflected laser pulse sequence. In one example, the laser radar can determine the reception time of the reflected laser pulse sequence, for example, by detecting the rising edge time and / or falling edge time of the electrical signal pulse to determine the reception time of the laser pulse sequence. In this way, the laser radar can calculate TOF (Time of flight) using the reception time information and emission time of the laser pulse sequence, thereby determining the distance from the detected object to the laser radar. The laser radar is an autonomous luminous sensor that does not rely on light source illumination and is less affected by ambient light. It can work normally even in a lightless closed environment, so as to subsequently generate a high-precision three-dimensional model, and has a wide range of applicability. The point cloud acquisition principle of the millimeter wave radar is similar to that of the laser radar, which will not be repeated here.

[0075] The binocular vision sensor obtains two images of the target scene from different positions based on the parallax principle, and obtains 3D geometric information by calculating the position deviation between the corresponding points of the two images, thereby generating a 3D point cloud. The binocular vision sensor has low hardware requirements, and accordingly, it can also reduce costs. It only needs an ordinary CMOS (Complementary Metal Oxide Semiconductor) camera. As long as the light is suitable, it can be used in both indoor and outdoor environments, so it also has certain applicability.

[0076] The structured light depth camera projects light with certain structural features into the target scene for collection. This light with a certain structure will collect different image phase information due to the different depth areas of the object being photographed, and then convert it into depth information to obtain a three-dimensional point cloud. The structured light depth camera is also a self-luminous sensor that does not rely on light source illumination and is less affected by ambient light. It can work normally even in a dark and closed environment, so as to generate high-precision three-dimensional models later, and has a wide range of applicability.

[0077] In an exemplary application scenario, such as in the field of oral scanning, the point cloud acquisition device can scan inside the user's oral cavity to collect data related to the user's teeth, thereby establishing a three-dimensional model of the teeth; Figure 1A and Figure 1B As shown, Figure 1A The projection result of a frame of point cloud collected by the point cloud collection device at time T in the global model is shown. For easy understanding, the frame of point cloud is Figure 1A The image may be a two-dimensional image or a depth image carrying depth information, and this embodiment does not impose any restriction on this; Figure 1B The scanning range of the point cloud acquisition device in the oral cavity at time T is shown. Figure 1A and Figure 1B It can be seen that the point cloud acquisition device at time T scans the finger, so the collected point cloud contains noise related to the finger. After projecting the point cloud collected by the point cloud acquisition device at time T onto the global model, the projection area of ​​the global model ( Figure 1A Miscellaneous data such as fingers appear in the box).

[0078] The point cloud acquisition device continues to scan inside the user's oral cavity, and the point cloud processing method provided in the embodiment of the present application can be applied during the scanning process to delete the miscellaneous data in real time; for example, at time T+N, the point cloud acquisition device scans a position that intersects with time T; please refer to Figure 2A and Figure 2B ,like Figure 2AThe projection result of a frame of point cloud collected by the point cloud collection device at time T+N in the global model is shown. Figure 2B In the form of images; Figure 2B The scanning range of the point cloud acquisition device in the oral cavity at time T+n is shown. Figure 2A and Figure 2B It can be seen that the point cloud acquisition device did not scan the finger at time T+n, where n is an integer greater than 0. Based on the assumption that there are no other objects in theory between the point cloud acquisition device and the object to be measured (i.e., the point cloud of the current frame), after applying the point cloud processing method provided in the embodiment of the present application, the weight of the noise points related to the finger collected at time T in the global model can be reduced; the weight of the noise points related to the finger can be reduced multiple times when the same position is scanned multiple times, and finally, when the reduced weight meets the noise point deletion condition, the noise points related to the finger collected at the historical moment in the global model are deleted; multiple verifications are performed to accurately delete the noise points.

[0079] Next, the point cloud processing method provided in the embodiment of the present application is exemplarily described:

[0080] It should be noted that the point cloud acquisition device in the embodiment of the present application needs to scan the same position in the detection scene multiple times, so that the miscellaneous data scanned at the same position at the historical moment can be filtered out and deleted based on the scanning result at the current moment. Figure 3 , Figure 3 A schematic flow chart of a point cloud processing method is shown. The method can be executed by an electronic device, and the method includes:

[0081] In S101, the current frame point cloud collected by the point cloud collection device and the global model are obtained; wherein the global model is obtained by fusion of the historical frame point clouds collected by the point cloud collection device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object.

[0082] In S102, after the current frame point cloud is projected into the global model, a plurality of light rays are projected between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional points in the global model passed by the light rays.

[0083] In S103, the weight of the target three-dimensional point is reduced.

[0084] In S104, if the reduced weight of the target three-dimensional point satisfies a noise point deletion condition, the target three-dimensional point is deleted.

[0085] In this embodiment, it can be assumed that all the collected current frame point cloud is valid data, and there is no other point cloud data between the current frame point cloud and the point cloud acquisition device. In other words, there is no other object between the point cloud acquisition device and the object to be measured (i.e., the current frame point cloud) in theory. Based on this assumption, the current frame point cloud can be used to delete the noise data between the current frame point cloud and the point cloud acquisition device collected at a historical moment in the global model; the current frame point cloud can be projected into the global model, and then a number of rays are projected between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional point in the global model that is passed by the rays. The target three-dimensional point may be a noise point between the current frame point cloud and the point cloud acquisition device that was mistakenly collected at a historical moment. In order to avoid mistaken deletion, this embodiment reduces the weight of the target three-dimensional point. The weight can be reduced multiple times based on the situation that the target three-dimensional point is passed by the rays multiple times until the reduced weight of the target three-dimensional point meets the noise deletion condition, and then the target three-dimensional point is deleted, so that the noise data can be accurately deleted after multiple verifications, thereby improving the accuracy of the deletion result.

[0086] Exemplarily, the noise deletion condition indicates that the weight of the three-dimensional point is less than a preset threshold, and the preset threshold can be specifically set according to an actual application scenario.

[0087] Exemplarily, the global model is obtained based on the fusion of historical frame point clouds collected by the point cloud collection device. In one example, the global model includes a plurality of three-dimensional points. In another example, in addition to the plurality of three-dimensional points, the global model may also include triangle information formed by three-dimensional connections; that is, the global model may be a mesh model.

[0088] In some embodiments, after acquiring the current frame point cloud collected by the point cloud collection device, the electronic device needs to further determine the initial weights of the three-dimensional points in the current frame point cloud.

[0089] In one possible implementation, it can be assumed that the three-dimensional points in the current frame point cloud collected by the point cloud acquisition device are all valid data, that is, the three-dimensional points in the current frame point cloud can be assigned preset initial weights, and the initial weights of all three-dimensional points in the current frame point cloud are the same.

[0090] In another possible implementation, the initial weight of the 3D point can be determined based on the reliability of the depth information of the 3D point in the current frame point cloud, and the greater the reliability, the greater the initial weight. The reliability of the depth information of the 3D point can be reflected by the acquisition information of the 3D point. Exemplarily, for each 3D point in the current frame point cloud, the electronic device can obtain a first adjustment coefficient according to the acquisition information of the 3D point, and then adjust the reference weight using the first adjustment coefficient to obtain the initial weight of the 3D point; wherein the first adjustment coefficient is positively correlated with the initial weight, that is, the larger the first adjustment coefficient, the larger the initial weight, and vice versa. In one example, the initial weight of the 3D point is equal to the product of the first adjustment coefficient and the reference weight. In another example, the initial weight of the 3D point is equal to the sum of the first adjustment coefficient and the reference weight. The specific adjustment method of adjusting the reference weight using the first adjustment coefficient in this embodiment is not limited in any way and can be specifically set according to the actual application scenario.

[0091] Exemplarily, the acquisition information of each 3D point includes at least one of the following: the distance between the 3D point and the best depth of field of the point cloud acquisition device, the distance between the 3D point and the center of the field of view of the point cloud acquisition device, the distance between the 3D point and the light interpolating the 3D point in the light projected by the point cloud acquisition device, or the difference between the normal information of the 3D point and the normal information of its adjacent 3D points; but not limited to this. In one example, if the point cloud acquisition device is a structured light camera, the projected light is structured light; if the point cloud acquisition device is a laser radar, the projected light is a laser line.

[0092] Based on practical demonstration, it can be determined that: (1) the depth of the three-dimensional point collected within the depth of field of the point cloud acquisition device is more reliable, so the smaller the distance between the three-dimensional point and the optimal depth of field, the larger the first adjustment coefficient, and vice versa; (2) the depth of the three-dimensional point collected at the center of the field of view of the point cloud acquisition device is more reliable, so the smaller the distance between the three-dimensional point and the center of the field of view, the larger the first adjustment coefficient, and vice versa; (3) the closer the three-dimensional point is to the light interpolating the three-dimensional point in the light projected by the point cloud acquisition device, the more reliable the depth of the three-dimensional point is, so the smaller the distance between the three-dimensional point and the light interpolating the three-dimensional point in the light projected by the point cloud acquisition device, the larger the first adjustment coefficient, and vice versa; (4) the depth of the three-dimensional point with a smaller depth jump is more reliable, so the smaller the difference between the normal information of the three-dimensional point and the normal information of its adjacent three-dimensional points, the larger the first adjustment coefficient, and vice versa.

[0093] That is to say, the first adjustment coefficient of the three-dimensional point is negatively correlated with any one of the following: the distance between the three-dimensional point and the depth of field, the distance between the three-dimensional point and the center of the field of view, the distance between the three-dimensional point and the light projection position, and the difference between the normal information of the three-dimensional point and the normal information of its adjacent three-dimensional point.

[0094] In another possible implementation, in addition to considering the reliability of the depth information of the 3D point in the current frame point cloud, the importance of the 3D point can be further considered. The higher the importance of the 3D point, the higher the initial weight. The importance of the 3D point can be reflected by the object recognition result of the 3D point. If the 3D point is recognized as a target object, such as a tooth or gum, it means that the 3D point is more important; if the 3D point is recognized as a miscellaneous point, it means that the 3D point is not important.

[0095] Exemplarily, for each three-dimensional point in the current frame point cloud, the electronic device can obtain a first adjustment coefficient based on the acquisition information of the three-dimensional point; and, after performing object recognition processing on the current frame point cloud, obtain a second adjustment coefficient based on the object recognition result corresponding to the three-dimensional point; then adjust the reference weight based on the first adjustment coefficient and the second adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the second adjustment coefficient is positively correlated with the initial weight, and the second adjustment coefficient of the three-dimensional point whose object recognition result is the target object is greater than the second adjustment coefficient of the three-dimensional point whose object recognition result is a noise point.

[0096] In one example, the initial weight of the three-dimensional point may be the product of the first adjustment coefficient, the second adjustment coefficient, and the reference weight. In another example, the initial weight of the three-dimensional point may also be the sum of the first adjustment coefficient, the second adjustment coefficient, and the reference weight. In this embodiment, the specific adjustment method of adjusting the reference weight using the first adjustment coefficient and the second adjustment coefficient is not limited in any way and can be specifically set according to the actual application scenario.

[0097] The target object may include one or more than one, and the second adjustment coefficients of the three-dimensional points indicating different types of target objects may be the same or different, and may be specifically set according to the actual application scenario. In one example, assuming that in the field of tooth scanning, the target object includes teeth and gums, the second adjustment coefficient of the three-dimensional point whose object recognition result is teeth may be set to be greater than the second adjustment coefficient of the three-dimensional point whose object recognition result is gums.

[0098] In an example, taking the example that the initial weight of the three-dimensional point can be the product of the first adjustment coefficient, the second adjustment coefficient and the baseline weight, the value of the initial weight is between 0-1, the value of the baseline weight is 1, and the values ​​of the first adjustment coefficient and the second adjustment coefficient are between 0 and 1; assuming that in the field of dental scanning, the target objects include teeth and gums, wherein the second adjustment coefficient of the three-dimensional point being a tooth can be set to 1, the second adjustment coefficient of the three-dimensional point being a gum can be set to 0.5, and the second adjustment coefficient of the three-dimensional point being a miscellaneous point can be set to 0; of course, other values ​​can also be used, and this embodiment does not impose any restrictions on this.

[0099] When performing object recognition processing on the current frame point cloud, a pre-trained object recognition model can be used to process the current frame point cloud, thereby obtaining an object recognition result corresponding to a three-dimensional point in the current frame point cloud. Exemplarily, the object recognition model can be obtained through supervised training based on several frame point clouds and their object labels.

[0100] In some embodiments, the global model is obtained by fusing the historical frame point clouds collected by the point cloud collection device, that is, each time the point cloud collection device collects a frame of point cloud, the frame of point cloud can be fused into the global model. Wherein, there is an overlap between the collection position of the current frame point cloud and the collection position of the historical frame point cloud, that is, there are three-dimensional points collected by the point cloud collection device from the same position in the current frame point cloud and the global model, so that the noise points in the global model can be screened out based on the current frame point cloud.

[0101] Any 3D point in the global model carries a weight. In one example, the greater the weight of a 3D point in the global model, the greater the possibility that the 3D point is a target object and the more reliable the 3D point is, and the smaller the weight, the greater the possibility that the 3D point is a noise point and the more unreliable the 3D point is.

[0102] After projecting the current frame point cloud into the global model, the electronic device may use the initial weights of the three-dimensional points in the current frame point cloud to update the weights of the three-dimensional points in the projection area of ​​the current frame point cloud in the global model. Exemplarily, after projecting a certain three-dimensional point of the current frame point cloud to a certain projection position in the global model, if there is no three-dimensional point at the projection position, the initial weight of the three-dimensional point in the current frame point cloud may be used as the weight of the three-dimensional point at the projection position; if there is a three-dimensional point at the projection position, the initial weight of the three-dimensional point in the current frame point cloud and the weight of the three-dimensional point at the projection position may be accumulated, and the accumulated weight may be used as the updated weight of the three-dimensional point at the projection position.

[0103] After projecting the current frame point cloud into the global model, the electronic device may project a number of rays between the projected current frame point cloud and the point cloud acquisition device to determine the target three-dimensional point in the global model passed by the rays. The electronic device may project a number of rays between the projected current frame point cloud and the point cloud acquisition device; or may project a number of rays between the projection area corresponding to the projected current frame point cloud and the point cloud acquisition device; this embodiment does not impose any restrictions on this.

[0104] Exemplarily, the starting point of the light is one of the first position and the second position, and the end point of the light is the other of the first position and the second position. The first position may be the position of the projected current frame point cloud, or in order to avoid the situation where the projected current frame point cloud is misprocessed, the first position may be the result of the projected current frame point cloud position combined with a preset error distance. The specific value of the preset error distance can be specifically set according to the actual application scenario, and this embodiment does not impose any restrictions on this.

[0105] The second position may be the position of the lens of the point cloud acquisition device when acquiring the current frame point cloud; or in order to improve the efficiency of light projection, the second position may be the result of combining the position of the projected current frame point cloud with a preset distance, and the preset distance is determined according to the depth of field of the point cloud acquisition device when acquiring the current frame point cloud, thereby avoiding or reducing the projection of light to areas where no three-dimensional points are acquired, thereby improving the efficiency of light projection.

[0106] In one example, the light starts from the three-dimensional point of the projected current frame point cloud and extends in the direction of the location of the point cloud acquisition device. In another example, the initial point of the light can be the result of adding a preset error distance to the coordinates of the three-dimensional point of the projected current frame point cloud, and then extends in the direction of the location of the point cloud acquisition device.

[0107] In some embodiments, after a plurality of rays are projected between the projected current frame point cloud and the point cloud acquisition device, the electronic device can determine the target three-dimensional point in the global model that is passed by the rays. Based on the assumption that there are no other objects in theory between the point cloud acquisition device and the object to be measured (i.e., the current frame point cloud), the target three-dimensional point in the global model that is passed by the rays may be a noise point between the current frame point cloud and the point cloud acquisition device that was mistakenly collected at a historical moment.

[0108] In some possible implementations, the electronic device may detect in an exhaustive manner whether each 3D point in the global model intersects with any of the plurality of light rays, thereby determining a target 3D point in the global model passed by the light ray.

[0109] In another possible implementation, the space where the global model is located can be divided into a number of voxels. Voxel can be understood as the smallest unit of digital data in the three-dimensional space segmentation, which is conceptually similar to the smallest unit of two-dimensional space - pixel, which is used in the image data of two-dimensional computer images. Voxel represents a three-dimensional area with a constant scalar or vector. In order to reduce computational complexity and improve computational efficiency, in the process of determining the target three-dimensional point, the electronic device can first detect the target voxel passed by the light in the several voxels, and then further determine the target three-dimensional point in the target voxel that intersects with the light; since the area represented by a single voxel is larger than a single three-dimensional point, the method of first determining the target voxel and then determining the target three-dimensional point can significantly reduce the amount of computation and improve computational efficiency compared to the above-mentioned exhaustive method.

[0110] In some embodiments, after determining the target 3D point in the global model that is passed by the light, the electronic device can reduce the weight of the target 3D point, and then delete the target 3D point if the reduced weight of the target 3D point meets the noise deletion condition; in order to avoid accidental deletion, the detection of noise in this embodiment is a continuous process. If a certain 3D point is a noise point, the weight of the 3D point can be reduced multiple times through multiple scanning processes until the modified weight of the 3D point meets the noise deletion condition, so as to accurately delete the noise point under multiple verifications and improve the accuracy of the deletion result. Exemplarily, the noise deletion condition indicates that the weight of the 3D point is less than a preset threshold.

[0111] In a possible implementation, when a smaller weight indicates a greater possibility that the three-dimensional point is a noise point, the electronic device may subtract a preset value from the weight of the target three-dimensional point; further, if the reduced weight of the target three-dimensional point is less than a weight threshold, the electronic device may delete the target three-dimensional point.

[0112] Exemplarily, at least one of the weight threshold and the preset value (i.e., the reduction ratio) can be set by the user according to actual needs. The user can input a specific value based on his or her own experience, or some recommended values ​​can be given for the user to choose based on the actual application scenario.

[0113] In one example, the electronic device is provided with an input device (such as an interactive interface, a keyboard, a mouse, etc.), and the electronic device can receive the weight threshold and the preset value input by the user in the input device.

[0114] In another example, the electronic device responds to a scanning scene instruction input by the user, and obtains multiple recommended values ​​of the weight threshold and / or multiple recommended values ​​of the preset value from pre-stored data according to the scanning scene indicated by the scanning scene instruction; wherein the pre-stored data includes multiple recommended values ​​of the weight threshold and / or multiple recommended values ​​of the preset value under different scanning scenes; then the electronic device displays the multiple recommended values ​​of the weight threshold and / or multiple recommended values ​​of the preset value in the interactive interface for user adjustment; and then, in response to the user's selection operation, it can determine the selected value from the multiple recommended values ​​of the weight threshold, and / or determine the selected value from the multiple candidate values ​​of the preset value.

[0115] Among them, the values ​​of the weight threshold and the preset value will affect the efficiency and accuracy of noise removal.

[0116] In an example, if the weight threshold is set higher and the preset value is set lower, the weight of the noise point cannot be reduced to less than the weight threshold at one time due to the smaller preset value, and multiple comparisons are required to delete the noise point. For example, if the preset value is 0.3, the weight threshold is 0.1, and the weight of the noise point is 1, then each time it is determined that the noise point passes through the light, the weight of the noise point is reduced by 0.3 each time, and it needs to be reduced at least 3 times before the noise point can be deleted. The process of multiple comparisons and deletions is conducive to improving the accuracy of noise deletion, but it reduces the efficiency of noise deletion to a certain extent.

[0117] In another example, if the weight threshold is set low and the preset value is set high, the weight of the noise point may be reduced to less than the weight threshold after one or fewer times because the preset value is large, and the noise point can be directly deleted through a single or fewer comparison process, thereby improving the efficiency of noise point deletion.

[0118] Therefore, users can set weight thresholds and preset values ​​that meet their needs based on their needs for noise removal efficiency and accuracy.

[0119] In another possible implementation, the starting point or the ending point of the light may be a 3D point in the current frame point cloud, that is, the projected rays correspond one-to-one to the 3D points in the current frame point cloud; in order to further accurately modify the weight of the target 3D point, the electronic device may determine the 3D point in the current frame point cloud corresponding to the light passing through the target 3D point, and use the weight of the 3D point in the global model to reduce the weight of the target 3D point. This embodiment adaptively reduces the weight of the target 3D point according to the weight of the 3D point in the current frame point cloud corresponding to the light, thereby accurately modifying the weight of the target 3D point passed by different rays.

[0120] In one example, when the greater the weight, the greater the possibility that the 3D point is the target object, and the smaller the weight, the greater the possibility that the 3D point is a noise point, the electronic device can use the weight of the 3D point in the global model to reduce the weight of the target 3D point after determining the 3D point in the current frame point cloud corresponding to the light passing through the target 3D point. The greater the weight of the 3D point, the more the weight of the target 3D point is reduced; for example, the weight of the 3D point in the global model can be subtracted from the weight of the target 3D point; for another example, the value corresponding to the weight of the 3D point in the global model can be subtracted from the weight of the target 3D point, and the value is positively correlated with the weight of the 3D point. Further, when the reduced weight of the target 3D point is less than the weight threshold, the electronic device can delete the target 3D point. Among them, the weight threshold can be set by the user according to actual needs, and the user can enter a specific value based on his own experience, or some recommended values ​​can be given for the user to choose according to the actual application scenario.

[0121] In other words, the greater the weight of the 3D point in the current frame point cloud corresponding to the light passing through the target 3D point, the greater the possibility that the 3D point is the target object, and the more the weight of the target 3D point is reduced; conversely, the smaller the weight of the 3D point in the current frame point cloud corresponding to the light passing through the target 3D point, the smaller the possibility that the 3D point is the target object, and the less the weight of the target 3D point is reduced.

[0122] In some embodiments, the weights of different 3D points in the global model are different, and the display colors of the different 3D points are also different, so that the user can understand in real time the possibility of different 3D points being target objects or noise points. Exemplarily, the 3D points in the global model can be displayed in a color gradient in the order of weight from large to small or from small to large; the display colors of 3D points indicated by different weights are different.

[0123] In some embodiments, the user can select a lock region in the global model according to actual needs, and the lock region indicates that any three-dimensional point in the lock region is valid data that the user wants to obtain, and there is no need to modify the weight or delete it. After projecting the current frame point cloud into the global model, if the projected current frame point cloud is within the lock region, no subsequent operation is required.

[0124] In some embodiments, considering that there may be a deleted target 3D point that is not a noise point but is mistakenly judged as a noise point, resulting in a situation of mistaken deletion, therefore, if at least one frame of point cloud subsequently collected by the point cloud acquisition device contains the deleted target 3D point, the electronic device can perform weight accumulation based on the initial weight of the deleted target 3D point in each frame of point cloud; then, if the accumulated weight of the deleted target 3D point meets the preset reconstruction condition, the target 3D point is reconstructed in the global model. In other words, if the deleted target 3D point is included in the collection and scanning of the point cloud acquisition device, it means that the deleted target 3D point is likely to be mistakenly deleted, and the deleted target 3D point can be restored by weight accumulation.

[0125] In one example, the electronic device can reconstruct the target 3D point in the global model when the accumulated weight of the deleted target 3D point is greater than a weight threshold. By accumulating the initial weight, the deleted target 3D point can be restored in the global model through multiple verifications.

[0126] It is not difficult to understand that the solutions described in the above embodiments can be combined when there is no conflict, and they are not listed one by one in the embodiments of the present disclosure.

[0127] In some embodiments, a point cloud processing method includes: obtaining a global model collected by a point cloud acquisition device; wherein any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a noise point or a target object; if there is a target three-dimensional point in the global model whose weight meets the noise deletion condition, the target three-dimensional point is deleted.

[0128] Among them, the above weights can be adjusted according to the above embodiments, and can also be adjusted according to the Ai intelligent system (such as reducing the weight of the three-dimensional point when the three-dimensional point is identified as a non-target object). It can also be adjusted according to the patient's historical case data, and can also be adjusted according to the scanning scene, such as scanning a dental model or scanning a patient's oral cavity, and can also be adjusted according to user-set rules.

[0129] Figure 4 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 4At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and may also include hardware required for other services. One or more embodiments of this specification may be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0130] Please refer to Figure 5 , the point cloud processing device can be applied to Figure 4 The device shown in the figure is used to implement the technical solution of this specification. The point cloud processing device may include:

[0131] The acquisition module 501 is used to acquire the current frame point cloud collected by the point cloud acquisition device and the global model; wherein the global model is obtained by fusion of the historical frame point clouds collected by the point cloud acquisition device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object.

[0132] The target three-dimensional point determination module 502 is used to project a plurality of light rays between the projected current frame point cloud and the point cloud acquisition device after projecting the current frame point cloud into the global model, so as to determine the target three-dimensional points in the global model passed by the light rays.

[0133] The weight reduction module 503 is used to reduce the weight of the target three-dimensional point.

[0134] The target 3D point deletion module 504 is configured to delete the target 3D point if the reduced weight of the target 3D point satisfies a noise point deletion condition.

[0135] In some embodiments, after the acquisition module 501, a weight determination module is further included to determine the initial weights of the three-dimensional points in the current frame point cloud. The weight determination module is also used to update the weights of the three-dimensional points in the current frame point cloud within the projection area of ​​the global model using the initial weights of the three-dimensional points in the current frame point cloud after projecting the current frame point cloud into the global model.

[0136] In some embodiments, the weight determination module is specifically used to obtain a first adjustment coefficient for each three-dimensional point in the current frame point cloud according to the acquisition information of the three-dimensional point;

[0137] The reference weight is adjusted using the first adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the first adjustment coefficient is positively correlated with the initial weight.

[0138] The acquisition information of each of the three-dimensional points includes at least one of the following: the distance between the three-dimensional point and the best depth of field of the point cloud acquisition device, the distance between the three-dimensional point and the center of the field of view of the point cloud acquisition device, the distance between the three-dimensional point and a light ray projected by the point cloud acquisition device to interpolate the three-dimensional point, or the difference between the normal information of the three-dimensional point and the normal information of its adjacent three-dimensional point;

[0139] The first adjustment coefficient of the three-dimensional point is negatively correlated with the collection information of the three-dimensional point.

[0140] In some embodiments, the weight determination module is specifically used to: for each three-dimensional point in the current frame point cloud, obtain a first adjustment coefficient according to the acquisition information of the three-dimensional point;

[0141] And, after performing object recognition processing on the current frame point cloud, obtaining a second adjustment coefficient according to the object recognition result corresponding to the three-dimensional point;

[0142] The reference weight is adjusted according to the first adjustment coefficient and the second adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the second adjustment coefficient is positively correlated with the initial weight, and the second adjustment coefficient of the three-dimensional point whose object recognition result is the target object is greater than the second adjustment coefficient of the three-dimensional point whose object recognition result is a noise point.

[0143] In some embodiments, the starting point of the light ray is one of a first position and a second position, and the ending point of the light ray is the other of the first position and the second position;

[0144] Wherein, the first position includes one of the following: the position of the projected current frame point cloud, or the result of the projected current frame point cloud position combined with a preset error distance;

[0145] The second position includes one of the following: the position of the lens of the point cloud acquisition device when acquiring the current frame point cloud, or the position of the projected current frame point cloud combined with a preset distance; the preset distance is determined according to the depth of field of the point cloud acquisition device when acquiring the current frame point cloud.

[0146] In some embodiments, the plurality of light rays correspond one-to-one to a plurality of three-dimensional points in the current frame point cloud respectively; the weight reduction module 503 is specifically used to: determine the three-dimensional point in the current frame point cloud corresponding to the light ray passing through the target three-dimensional point, and use the weight of the three-dimensional point in the global model to reduce the weight of the target three-dimensional point.

[0147] In some embodiments, the space where the global model is located is divided into a number of voxels; the target three-dimensional point determination module 502 is specifically used to detect the target voxels passed by the light and determine the target three-dimensional points in the target voxels that intersect with the light.

[0148] In some embodiments, a reconstruction module is also included, which is used to accumulate weights based on the initial weights of the deleted target three-dimensional points in each frame of point cloud if at least one frame of point cloud subsequently acquired by the point cloud acquisition device contains deleted target three-dimensional points; and to reconstruct the target three-dimensional points in the global model if the accumulated weights of the deleted target three-dimensional points meet preset reconstruction conditions.

[0149] In some embodiments, the three-dimensional points in the global model are displayed in a color gradient manner in the order of weight from large to small or from small to large; the display colors of the three-dimensional points indicated by different weights are different.

[0150] In some embodiments, the noise removal condition indicates that the reduced weight of the target three-dimensional point is less than a weight threshold. The device further includes: an interactive module for obtaining a weight threshold input by a user in an interactive interface; or in response to a scanning scene instruction input by a user, obtaining multiple recommended values ​​of the weight threshold from pre-stored data according to the scanning scene indicated by the scanning scene instruction; wherein the pre-stored data includes multiple recommended values ​​of the weight threshold under different scanning scenes; displaying the multiple recommended values ​​of the weight threshold in the interactive interface; and determining a selected value from the multiple recommended values ​​of the weight threshold in response to a selection operation by the user.

[0151] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0152] Accordingly, an embodiment of the present application further provides an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0153] Wherein, the processor implements the steps in the above method when executing the executable instructions.

[0154] Exemplarily, the processor includes but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0155] Exemplarily, the memory may include at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0156] Accordingly, an embodiment of the present application further provides a computer program product, including a computer program, which is used to implement the above method when executed by a processor.

[0157] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0158] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the above method.

[0159] The embodiments of the subject matter and functional operations described in this specification may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules in computer program instructions encoded on a tangible non-temporary program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0160] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits), and the apparatus can also be implemented as special purpose logic circuits.

[0161] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or to transmit data to it, or both. However, the computer does not necessarily have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0162] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0163] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0164] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0165] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0166] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A point cloud processing method, characterized in that: include: Acquire the current frame point cloud collected by the point cloud collection device and the global model; wherein the global model is obtained by fusing the historical frame point clouds collected by the point cloud collection device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object; After projecting the current frame point cloud into the global model, projecting a plurality of rays between the projected current frame point cloud and the point cloud acquisition device to determine a target three-dimensional point in the global model passed by the rays; reducing the weight of the target three-dimensional point; If the reduced weight of the target three-dimensional point satisfies the noise point deletion condition, the target three-dimensional point is deleted.

2. The method according to claim 1, characterized in that Also includes: Determining initial weights of three-dimensional points in the point cloud of the current frame; After projecting the current frame point cloud into the global model, the method further includes: The weights of the three-dimensional points in the current frame point cloud are updated within the projection area of ​​the global model using the initial weights of the three-dimensional points in the current frame point cloud.

3. The method according to claim 2, characterized in that In the current frame point cloud, determining the initial weights of the three-dimensional points in the current frame point cloud includes: For each three-dimensional point in the current frame point cloud, obtaining a first adjustment coefficient according to acquisition information of the three-dimensional point; The reference weight is adjusted using the first adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the first adjustment coefficient is positively correlated with the initial weight.

4. The method according to claim 3, characterized in that The acquisition information of each of the three-dimensional points includes at least one of the following: the distance between the three-dimensional point and the best depth of field of the point cloud acquisition device, the distance between the three-dimensional point and the center of the field of view of the point cloud acquisition device, the distance between the three-dimensional point and a light ray projected by the point cloud acquisition device to interpolate the three-dimensional point, or the difference between the normal information of the three-dimensional point and the normal information of its adjacent three-dimensional point; The first adjustment coefficient of the three-dimensional point is negatively correlated with the collection information of the three-dimensional point.

5. The method according to claim 1, characterized in that The determining of the initial weights of the three-dimensional points in the point cloud of the current frame includes: For each three-dimensional point in the current frame point cloud, obtaining a first adjustment coefficient according to acquisition information of the three-dimensional point; And, after performing object recognition processing on the current frame point cloud, obtaining a second adjustment coefficient according to the object recognition result corresponding to the three-dimensional point; The reference weight is adjusted according to the first adjustment coefficient and the second adjustment coefficient to obtain the initial weight of the three-dimensional point; wherein the second adjustment coefficient is positively correlated with the initial weight, and the second adjustment coefficient of the three-dimensional point whose object recognition result is the target object is greater than the second adjustment coefficient of the three-dimensional point whose object recognition result is a noise point.

6. The method according to claim 1, characterized in that The starting point of the light ray is one of a first position and a second position, and the ending point of the light ray is the other of the first position and the second position; Wherein, the first position includes one of the following: the position of the projected current frame point cloud, or the result of the projected current frame point cloud position combined with a preset error distance; The second position includes one of the following: the position of the lens of the point cloud acquisition device when acquiring the current frame point cloud, or the position of the projected current frame point cloud combined with a preset distance; the preset distance is determined according to the depth of field of the point cloud acquisition device when acquiring the current frame point cloud.

7. The method according to any one of claims 1 to 6, characterized in that: The plurality of light rays correspond one-to-one to a plurality of three-dimensional points in the point cloud of the current frame respectively; The step of reducing the weight of the target three-dimensional point comprises: A 3D point in the current frame point cloud corresponding to a light ray passing through the target 3D point is determined, and a weight of the target 3D point is reduced by using the weight of the 3D point in the global model.

8. The method according to any one of claims 1 to 6, characterized in that: The space where the global model is located is divided into a number of voxels; The step of determining the target three-dimensional point in the updated global model passed by the light comprises: The target voxel passed by the light ray is detected, and the target three-dimensional point intersecting with the light ray in the target voxel is determined.

9. The method according to any one of claims 1 to 6, characterized in that: Also includes: If at least one frame of point cloud subsequently collected by the point cloud collection device contains the deleted target three-dimensional point, weight accumulation is performed based on the initial weight of the deleted target three-dimensional point in each frame of point cloud; When the accumulated weight of the deleted target three-dimensional point meets a preset reconstruction condition, the target three-dimensional point is reconstructed in the global model.

10. The method according to any one of claims 1 to 6, characterized in that: The three-dimensional points in the global model are displayed in a color gradient manner in the order of weight from large to small or from small to large; the display colors of the three-dimensional points indicated by different weights are different.

11. The method according to any one of claims 1 to 6, characterized in that: The noise removal condition indicates that the reduced weight of the target three-dimensional point is less than a weight threshold; The method further comprises: Obtain the weight threshold value entered by the user in the interactive interface; or In response to a scanning scene instruction input by a user, a plurality of recommended values ​​of a weight threshold are obtained from pre-stored data according to the scanning scene indicated by the scanning scene instruction; wherein the pre-stored data includes a plurality of recommended values ​​of the weight threshold under different scanning scenes; Display multiple recommended values ​​of weight thresholds in the interactive interface; In response to a selection operation by the user, a selected value is determined from a plurality of recommended values ​​of the weight threshold.

12. A point cloud processing method, characterized in that: include: Acquire a global model collected by a point cloud collection device; wherein any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object; the weight carried by the three-dimensional point in the global model can be adjusted based on the point cloud processing method according to any one of claims 1 to 11; If there is a target three-dimensional point in the global model whose weight satisfies the noise deletion condition, the target three-dimensional point is deleted.

13. A point cloud processing device, characterized in that: include: An acquisition module is used to acquire the current frame point cloud collected by the point cloud acquisition device and a global model; wherein the global model is obtained by fusing the historical frame point clouds collected by the point cloud acquisition device; any three-dimensional point in the global model carries a weight, and the weight of any three-dimensional point represents the possibility that the three-dimensional point is a clutter point or a target object; A target three-dimensional point determination module is used to project a plurality of rays between the projected current frame point cloud and the point cloud acquisition device after projecting the current frame point cloud into the global model, so as to determine the target three-dimensional point in the global model passed by the rays; A weight reduction module, used to reduce the weight of the target three-dimensional point; The target three-dimensional point deletion module is used to delete the target three-dimensional point if the reduced weight of the target three-dimensional point meets the noise deletion condition.

14. An electronic device, characterized in that: including a memory, a processor, and executable instructions stored in the memory and executable on the processor; Wherein, when the processor executes the executable instructions, the steps in the method as described in any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images