Point cloud data processing method and device, electronic device and storage medium

Abstract

The point cloud data processing method and device, electronic device and storage medium provided in the present application relate to the field of data processing technology. In the present application, first, the point cloud data set to be processed is divided into multiple point cloud data subsets; secondly, for each target point cloud data subset in the multiple point cloud data subsets, the visibility information of the feature point area corresponding to the target point cloud data subset is determined; then, if it is determined based on the visibility information that there is at least one point cloud data subset to be processed, then for each point cloud data subset to be processed, the visibility information of each corresponding feature point is determined based on the point data in the point cloud data subset to be processed, wherein the visibility information of the feature point area corresponding to the point cloud data subset to be processed meets the target visibility condition. Based on the above method, the problem of poor processing effect of point cloud data processing in the prior art can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a point cloud data processing method and device, an electronic device, and a storage medium. Background Art

[0002] In application scenarios such as expression-driven, virtual digital human, and virtual 3D special effects, in order to reconstruct a high-precision face, it is necessary to determine the visibility of the point cloud (the visibility of each 3D point). The visibility of the point cloud generally refers to whether the 3D point is visible to the camera. For example, if the 3D point is facing the camera, it is visible to the camera. If it is blocked by other points, the 3D point is invisible to the camera. The inventors have found that in the prior art, in the process of determining the visibility of the point cloud, the processing of the point cloud data has a poor processing effect. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a point cloud data processing method and device, an electronic device and a storage medium, so as to improve the problem of poor processing effect of point cloud data in the prior art.

[0004] To achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0005] A point cloud data processing method, comprising:

[0006] Dividing the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each of the point cloud data subsets includes point data of a plurality of feature points;

[0007] For each target point cloud data subset in the plurality of point cloud data subsets, determine visibility information of a feature point area corresponding to the target point cloud data subset, wherein the target point cloud data subset is one or more of the plurality of point cloud data subsets;

[0008] If it is determined based on the visibility information that there is at least one subset of point cloud data to be processed, then for each of the subsets of point cloud data to be processed, the visibility information of each corresponding feature point is determined based on the point data in the subset of point cloud data to be processed, wherein the subset of point cloud data to be processed belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the subset of point cloud data to be processed meets the target visibility condition.

[0009] In a preferred embodiment of the present application, in the above point cloud data processing method, the step of dividing the point cloud data set to be processed into a plurality of point cloud data subsets includes:

[0010] Determine a plurality of feature point regions based on the historical point cloud data set corresponding to the plurality of feature points, wherein the historical point cloud data set includes the historical point data of each of the feature points;

[0011] For each of the feature point regions, a point cloud data subset corresponding to the feature point region is formed based on the point data of each feature point in the point cloud data set to be processed that belongs to the feature point region.

[0012] In a preferred embodiment of the present application, in the above point cloud data processing method, the step of determining a plurality of feature point areas based on the historical point cloud data set corresponding to the plurality of feature points includes:

[0013] Acquire a historical point cloud data set corresponding to the plurality of feature points;

[0014] Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine a visibility representation value corresponding to each of the feature points and a spatial distance representation value corresponding to each two of the feature points, and determine a cluster distance value between each two of the feature points based on the visibility representation value and the spatial distance representation value;

[0015] The plurality of feature points are clustered based on the cluster distance value between every two of the feature points to obtain a plurality of corresponding feature point regions.

[0016] In a preferred choice of the embodiment of the present application, in the above-mentioned point cloud data processing method, the step of determining the visibility representation value corresponding to each feature point and the spatial distance representation value corresponding to each two feature points based on the historical point data set of each feature point, and determining the clustering distance value between each two feature points based on the visibility representation value and the spatial distance representation value includes:

[0017] Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine the normal vector corresponding to each of the feature points, and based on the angle between the corresponding normal vector and the optical axis of the camera, respectively determine the visibility representation value corresponding to each of the feature points;

[0018] Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine the spatial distance between every two of the feature points, and based on the spatial distance and the number of feature point regions to be clustered, respectively determine the spatial distance representation value corresponding to each two of the feature points;

[0019] For every two feature points among the plurality of feature points, a cluster distance value between the two feature points is determined based on the visibility representation values ​​corresponding to the two feature points and the spatial distance representation value corresponding to the two feature points.

[0020] In a preferred embodiment of the present application, in the above point cloud data processing method, the step of determining the visibility information of the feature point area corresponding to each target point cloud data subset in the multiple point cloud data subsets includes:

[0021] For each point cloud data subset of the plurality of point cloud data subsets, determining whether point data of feature points at the edge of a feature point region corresponding to the point cloud data subset meets a preset target data condition, and determining the point cloud data subset as a target point cloud data subset when the point data of feature points at the edge of the feature point region corresponding to the point cloud data subset does not meet the target data condition;

[0022] For each of the target point cloud data subsets, visibility information of a feature point region corresponding to the target point cloud data subset is determined.

[0023] In a preferred embodiment of the present application, in the above point cloud data processing method, the step of determining the visibility information of the feature point area corresponding to each of the target point cloud data subsets includes:

[0024] For each of the target point cloud data subsets, determining at least one corresponding feature point plane in the feature point region corresponding to the target point cloud data subset, wherein each of the feature point planes includes a plurality of feature points in the feature point region;

[0025] For each of the target point cloud data subsets, a normal vector of each of the feature point planes corresponding to the target point cloud data subset is determined, and visibility information of the feature point area corresponding to the target point cloud data subset is determined based on the normal vector of each of the feature point planes.

[0026] In a preferred embodiment of the present application, in the above point cloud data processing method, if it is determined based on the visibility information that there is at least one point cloud data subset to be processed, then for each of the point cloud data subsets to be processed, the step of determining the visibility information of each corresponding feature point based on the point data in the point cloud data subset to be processed includes:

[0027] For each of the target point cloud data subsets, if the visibility information of the feature point region corresponding to the target point cloud data subset indicates that the feature point region is invisible, then the target point cloud data subset is determined as the point cloud data subset to be processed;

[0028] For each of the target point cloud data subsets, if the visibility information of the feature point area corresponding to the target point cloud data subset indicates that the feature point area is visible, determining whether an adjacent target point cloud data subset of the target point cloud data subset belongs to the point cloud data subset to be processed, and if the adjacent target point cloud data subset belongs to the point cloud data subset to be processed, determining the target point cloud data subset as the point cloud data subset to be processed;

[0029] For each of the to-be-processed point cloud data subsets, visibility information of each corresponding feature point is determined based on the point data in the to-be-processed point cloud data subset.

[0030] The present application also provides a point cloud data processing device, comprising:

[0031] A data subset division module, used for dividing the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each of the point cloud data subsets includes point data of a plurality of feature points;

[0032] A region visibility determination module, configured to determine, for each target point cloud data subset in the plurality of point cloud data subsets, visibility information of a feature point region corresponding to the target point cloud data subset, wherein the target point cloud data subset is one or more point cloud data subsets in the plurality of point cloud data subsets;

[0033] A feature point visibility determination module is used to determine the visibility information of each corresponding feature point for each of the point cloud data subsets to be processed based on the point data in the point cloud data subset to be processed when it is determined that there is at least one point cloud data subset to be processed based on the visibility information, wherein the point cloud data subset to be processed belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the point cloud data subset to be processed meets the target visibility condition.

[0034] Based on the above, an embodiment of the present application further provides an electronic device, including:

[0035] Memory for storing computer programs;

[0036] The processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned point cloud data processing method.

[0037] On the basis of the above, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is run, each step included in the above-mentioned point cloud data processing method is executed.

[0038] The point cloud data processing method and device, electronic device and storage medium provided by the present application can first divide the point cloud data set to be processed into multiple point cloud data subsets, so that the visibility information of the feature point area corresponding to each target point cloud data subset in the multiple point cloud data subsets can be determined first, and then, when there is a point cloud data subset to be processed whose visibility information of the corresponding feature point area meets the target visibility condition in the target point cloud data subset, the visibility information of each feature point in the point cloud data subset to be processed is determined respectively, that is, when the visibility information of the feature point area meets the target visibility condition, the visibility of each feature point in the feature point area is determined respectively. Based on this, compared with the conventional technical solution of directly traversing each feature point to determine the visibility of each feature point respectively, the technical solution provided by the present application is adopted. Since a visibility determination mechanism of the feature point area is configured before determining the visibility of each feature point, the number of feature points that need to be directly determined based on the corresponding point data can be reduced to a certain extent. Therefore, the amount of data for point cloud data processing can be reduced and the efficiency of point cloud data processing can be improved, thereby improving the problem of poor point cloud data processing effect in the prior art.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0041] Figure 2 A flowchart of a point cloud data processing method provided in an embodiment of the present application.

[0042] Figure 3 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S110.

[0043] Figure 4 for Figure 3 Schematic diagram of the flow chart of the sub-steps included in step S111.

[0044] Figure 5 A schematic diagram of feature points and corresponding triangular facets provided in an embodiment of the present application.

[0045] Figure 6 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S120.

[0046] Figure 7 for Figure 6 Schematic diagram of the flow chart of the sub-steps included in step S122.

[0047] Figure 8 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S130.

[0048] Fig. 9 A block diagram of a point cloud data processing device provided in an embodiment of the present application.

[0049] Icons: 10 - electronic device; 12 - memory; 14 - processor; 100 - point cloud data processing device; 110 - data subset division module; 120 - region visibility determination module; 130 - feature point visibility determination module. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0051] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0052] like Figure 1 As shown, an embodiment of the present application provides an electronic device 10 which may include a memory 12 , a processor 14 and a point cloud data processing device 100 .

[0053] The memory 12 and the processor 14 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The point cloud data processing device 100 includes at least one software function module that can be stored in the memory 12 in the form of software or firmware. The processor 14 is used to execute an executable computer program stored in the memory 12, for example, the software function module and computer program included in the point cloud data processing device 100, so as to implement the point cloud data processing method provided in the embodiment of the present application.

[0054] Optionally, the memory 12 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0055] Furthermore, the processor 14 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0056] Understandably, Figure 1 The structure shown is for illustration only. The electronic device 10 may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices (such as terminal devices).

[0057] It is understandable that, in an alternative example, the electronic device may be a server with data processing capabilities.

[0058] Combination Figure 2 The embodiment of the present application further provides a point cloud data processing method applicable to the electronic device 10 , wherein the method steps defined in the process related to the point cloud data processing method can be implemented by the electronic device 10 .

[0059] The following will Figure 2 The specific process shown is explained in detail.

[0060] Step S110: dividing the point cloud data set to be processed into a plurality of point cloud data subsets.

[0061] In an embodiment of the present application, the electronic device may divide the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each of the point cloud data subsets includes point data of a plurality of feature points, and the point data may be used to characterize feature information of the corresponding feature points, such as position coordinates.

[0062] Step S120 : for each target point cloud data subset in the plurality of point cloud data subsets, determine visibility information of a feature point region corresponding to the target point cloud data subset.

[0063] In the embodiment of the present application, after obtaining the plurality of point cloud data subsets based on step S110, the electronic device may determine the visibility information of the feature point area corresponding to the target point cloud data subset for each target point cloud data subset in the plurality of point cloud data subsets. The target point cloud data subset is one or more of the plurality of point cloud data subsets, and the feature point area corresponding to the target point cloud data subset may refer to an area composed of the corresponding plurality of feature points.

[0064] Step S130: If it is determined based on the visibility information that there is at least one point cloud data subset to be processed, then for each of the point cloud data subsets to be processed, the visibility information of each corresponding feature point is determined based on the point data in the point cloud data subset to be processed.

[0065] In the embodiment of the present application, after obtaining the visibility information of the feature point area corresponding to each of the target point cloud data subsets based on step S120, if it is determined based on the visibility information that there is at least one point cloud data subset to be processed, the electronic device can determine the visibility information of each corresponding feature point for each of the point cloud data subsets to be processed based on the point data in the point cloud data subset to be processed. Wherein, the point cloud data subset to be processed belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the point cloud data subset to be processed meets the target visibility condition, such as the visibility information of the feature point area is invisible or other.

[0066] Based on the steps included in the above method, the point cloud data set to be processed can be first divided into multiple point cloud data subsets, so that the visibility information of the feature point area corresponding to each target point cloud data subset in the multiple point cloud data subsets can be determined first, and then, in the target point cloud data subset, when there is a point cloud data subset to be processed whose visibility information of the corresponding feature point area meets the target visibility condition, the visibility information of each feature point in the point cloud data subset to be processed is determined respectively, that is, when the visibility information of the feature point area meets the target visibility condition, the visibility of each feature point in the feature point area is determined respectively. Based on this, compared with the conventional technical solution of directly traversing each feature point to determine the visibility of each feature point respectively, the technical solution provided by the present application is adopted. Since a visibility determination mechanism of the feature point area is configured before determining the visibility of each feature point, the number of feature points that need to be directly determined based on the corresponding point data can be reduced to a certain extent. Therefore, the amount of data for point cloud data processing can be reduced and the efficiency of point cloud data processing can be improved, thereby improving the problem of poor point cloud data processing effect in the prior art.

[0067] First, it should be noted that for step S110, the specific method of dividing the to-be-processed point cloud data set into the multiple point cloud data subsets based on step S110 is not limited.

[0068] For example, in an alternative example, the point cloud data set to be processed can be divided into the multiple point cloud data subsets based on manual labeling. For example, the point cloud data set to be processed can be divided into the multiple point cloud data subsets based on the division operation performed by the corresponding user.

[0069] For example, in another alternative example, in order to improve the efficiency of dividing the to-be-processed point cloud data set into the plurality of point cloud data subsets and reduce labor costs, etc., Figure 3 Step S110 may include step S111 and step S112, the specific contents of which are described below.

[0070] Step S111: determining a plurality of feature point regions based on the historical point cloud data set corresponding to the plurality of feature points.

[0071] In an embodiment of the present application, after obtaining the point cloud data set to be processed, multiple feature point regions can be determined based on the historical point cloud data set corresponding to multiple feature points in the point cloud data set to be processed, wherein the historical point cloud data set includes the historical point data of each of the feature points.

[0072] Step S112: for each of the feature point regions, based on the point data of each feature point in the point cloud data set to be processed belonging to the feature point region, a point cloud data subset corresponding to the feature point region is formed.

[0073] In an embodiment of the present application, after the multiple feature point areas are determined based on step S111, a point cloud data subset corresponding to the feature point area can be formed for each of the feature point areas based on the point data of each feature point belonging to the feature point area in the point cloud data set to be processed.

[0074] In this way, a one-to-one correspondence can be formed between the feature point region and the point cloud data subset, so that the point data of each feature point in a feature point region belongs to the point cloud data subset corresponding to the feature point region.

[0075] Optionally, for step S111 in the above example, the specific method of determining the multiple feature point areas is not limited and can be selected according to actual application requirements.

[0076] For example, in an alternative example, the position distance information between each of the feature points can be first determined based on the historical point cloud data set corresponding to the multiple feature points, and then the multiple feature points can be clustered based on the position distance information to determine multiple feature point areas.

[0077] For example, in another alternative example, in order to improve the accuracy of clustering, and considering that the multiple feature points have a relatively complex three-dimensional spatial relationship, such as the multiple feature points are face feature points, therefore, combined with Figure 4 , step S111 may further include step S111a, step S111b and step S111c, the specific contents of which are described as follows.

[0078] Step S111a, obtaining a historical point cloud data set corresponding to the plurality of feature points.

[0079] In an embodiment of the present application, a historical point cloud data set corresponding to multiple feature points in the point cloud data set to be processed can be obtained first. For example, the point cloud data to be processed can be a frame of three-dimensional face image currently scanned, and the historical point cloud data set can be a frame of three-dimensional face image scanned historically, such as the first frame of three-dimensional face image scanned.

[0080] Step S111b, based on the historical point data of each feature point included in the historical point cloud data set, determine the visibility representation value corresponding to each feature point and the spatial distance representation value corresponding to each two feature points, and determine the clustering distance value between each two feature points based on the visibility representation value and the spatial distance representation value.

[0081] In an embodiment of the present application, after the historical point cloud data set is acquired based on step S111a, on the one hand, the visibility characterization value corresponding to each feature point can be determined based on the historical point data of each feature point included in the historical point cloud data set. On the other hand, the spatial distance characterization value corresponding to each two feature points can be determined based on the historical point data of each feature point included in the historical point cloud data set. Then, the cluster distance value between each two feature points is determined based on the visibility characterization value and the spatial distance characterization value, that is, the cluster distance value is obtained by integrating the information of the two dimensions to improve the clustering accuracy.

[0082] Step S111c: clustering the plurality of feature points based on the cluster distance value between every two of the feature points to obtain a plurality of corresponding feature point regions.

[0083] In the embodiment of the present application, after the cluster distance value between each two feature points is determined based on step S111b, the plurality of feature points may be clustered based on the cluster distance value between each two feature points to obtain a plurality of corresponding feature point regions. For example, the plurality of feature points may be clustered based on a nearest neighbor algorithm (KNN, k-Nearest Neighbor).

[0084] Optionally, for step S111b in the above example, the specific method of determining the cluster distance value between every two feature points is not limited and can be selected according to actual application requirements.

[0085] For example, in an alternative example, step S111b may include sub-step 1, sub-step 2 and sub-step 3, the specific contents of which are described as follows.

[0086] Sub-step 1, based on the historical point data of each feature point included in the historical point cloud data set, determine the normal vector corresponding to each feature point respectively, and based on the angle between the corresponding normal vector and the camera optical axis, determine the visibility representation value corresponding to each feature point respectively.

[0087] In an embodiment of the present application, the normal vector corresponding to each feature point can be determined based on the historical point data of each feature point included in the historical point cloud data set, and then, for each feature point, the visibility characterization value corresponding to the feature point can be determined based on the angle between the normal vector corresponding to the feature point and the camera optical axis (also called the positive direction of the z-axis).

[0088] Sub-step 2, based on the historical point data of each feature point included in the historical point cloud data set, respectively determine the spatial distance between each two feature points, and based on the spatial distance and the number of feature point areas to be clustered, respectively determine the corresponding spatial distance characterization value between each two feature points.

[0089] In the embodiment of the present application, the spatial distance between each two feature points, such as the three-dimensional spatial distance, can be determined based on the historical point data of each feature point included in the historical point cloud data set. Then, for each two feature points, the spatial distance characterization value corresponding to the two feature points can be determined based on the spatial distance between the two feature points and the number of feature point regions to be clustered.

[0090] Sub-step 3: for every two feature points among the plurality of feature points, based on the visibility representation values ​​corresponding to the two feature points and the spatial distance representation value corresponding to the two feature points, determine a cluster distance value between the two feature points.

[0091] In an embodiment of the present application, after obtaining the visibility characterization value and the spatial distance characterization value based on the above-mentioned sub-steps 1 and 2, the clustering distance value between the two feature points can be determined for each two feature points among the multiple feature points based on the visibility characterization value corresponding to the two feature points and the spatial distance characterization value corresponding to the two feature points.

[0092] It can be understood that, for sub-step 1 in the above example, in an alternative example, a triangular face (a plane determined by three feature points, such as Figure 5 As shown, each triangle in the figure is a different triangular facet, and the vertices of the triangles are feature points), and the normal vector of the feature point is determined based on the normal vector of each triangular facet corresponding to the feature point, such as the average value of the normal vector of each triangular facet corresponding to the feature point. Then, for each of the feature points, the arccos (inverse cosine) value of the angle between the normal vector corresponding to the feature point and the optical axis of the camera can be calculated as the visibility representation value corresponding to the feature point, such as 0-1.

[0093] It can be understood that for sub-step 2 in the above example, in an alternative example, for every two feature points, the spatial distance between the two feature points can be divided by the number of feature point regions to be clustered to obtain the corresponding spatial distance representation value between the two feature points.

[0094] Among them, the number of feature point areas to be clustered can be a pre-configured threshold value, and the threshold value can also be adjusted. In this way, after determining the spatial distance representation value based on the current value, the cluster distance value is determined based on the subsequent steps, and then clustering is performed based on the cluster distance value. If the inter-class difference and / or intra-class difference obtained by clustering is large, the threshold value can be increased, and then the spatial distance representation value is re-determined, and clustering is performed again until the latest inter-class difference and / or intra-class difference is small (for example, less than the set difference value). In addition, in order to achieve normalization of the spatial distance representation value, the spatial distance representation value can also be divided by a standard value. For example, when processing facial feature points, the representation value can be the distance between the outer corners of the two eyes, or the distance between the contours of the two ears, as well as other distances that can represent the width of the face.

[0095] It can be understood that, for sub-step 3 in the above example, in an alternative example, for each two feature points among the multiple feature points, the absolute value of the difference between the visibility characterization values ​​corresponding to the two feature points can be calculated first to obtain the absolute difference corresponding to the two feature points, and then the absolute difference can be fused with the spatial distance characterization values ​​corresponding to the two feature points, such as performing a weighted sum calculation to obtain the corresponding weighted sum value, and the weighted sum value is used as the clustering distance value between the two feature points.

[0096] Among them, when performing weighted sum calculation, the inventors have found through research that, as an alternative example, in order to improve the clustering accuracy, the weight coefficient corresponding to the absolute difference value may be greater than the weight coefficient corresponding to the spatial distance characterization value. For example, the weight coefficient corresponding to the absolute difference value may be 0.7, and the weight coefficient corresponding to the spatial distance characterization value may be 0.3.

[0097] What needs to be further explained about the above-mentioned step S110 is that step S111 included in step S110 can be executed only once. Thus, after obtaining the point cloud data set to be processed each time, the currently obtained point cloud data set to be processed can be divided based on the multiple feature point areas obtained by historically executing step S111 to obtain multiple corresponding point cloud data subsets. Thus, when executing step S110 subsequently, the data processing amount can be effectively reduced and the data processing efficiency can be improved.

[0098] Secondly, it should be noted that for step S120, the specific method of determining the visibility information of the feature point area corresponding to the target point cloud data subset among the multiple point cloud data subsets based on step S120 is not limited and can be selected according to actual application requirements.

[0099] For example, in an alternative example, each point cloud data subset in the multiple point cloud data subsets can be directly used as a target point cloud data subset, and then the visibility information of the feature point area corresponding to each target point cloud data subset can be determined respectively.

[0100] For example, in another alternative example, in order to further improve the effect of point cloud data processing, such as comprehensively considering the requirements of processing efficiency and accuracy, combined with Figure 6 Step S120 may include step S121 and step S122, the specific contents of which are described as follows.

[0101] Step S121, for each point cloud data subset among the multiple point cloud data subsets, determine whether the point data of the feature points at the edge in the feature point area corresponding to the point cloud data subset meets the preset target data conditions, and when the point data of the feature points at the edge in the feature point area corresponding to the point cloud data subset does not meet the target data conditions, determine the point cloud data subset as the target point cloud data subset.

[0102] In an embodiment of the present application, after obtaining the multiple point cloud data subsets based on step S110, it can be determined for each point cloud data subset in the multiple point cloud data subsets whether the point data of the feature points at the edge in the feature point area corresponding to the point cloud data subset meets the preset target data conditions, and when the point data of the feature points at the edge in the feature point area corresponding to the point cloud data subset does not meet the target data conditions, the point cloud data subset is determined as the target point cloud data subset.

[0103] Step S122: for each of the target point cloud data subsets, determine visibility information of the feature point area corresponding to the target point cloud data subset.

[0104] In the embodiment of the present application, after the target point cloud data subset is determined based on step S121, visibility information of the feature point area corresponding to the target point cloud data subset may be determined for each of the target point cloud data subsets.

[0105] Optionally, for step S121 in the above example, the specific method of determining whether the point cloud data subset belongs to the target point cloud data subset is not limited and can be selected according to actual application requirements.

[0106] For example, in an alternative example, for each of the point cloud data subsets, the visibility representation value of each feature point on the edge of the feature point area corresponding to the point cloud data subset can be determined (as described above), and then the variance of the visibility representation value of each feature point is calculated to obtain the corresponding representation value variance. Thereafter, the representation value variance is compared with a preset threshold. If the representation value variance is less than or equal to the preset threshold, it can be determined that the point cloud data subset belongs to the target point cloud data subset. If the representation value variance is greater than the preset threshold, it can be determined that the point cloud data subset does not belong to the target point cloud data subset.

[0107] In an alternative example, the preset threshold may be 0.2.

[0108] For example, in another alternative example, for each of the point cloud data subsets, it can be determined whether the feature point area corresponding to the point cloud data subset belongs to the target area. If the feature point area corresponding to the point cloud data subset belongs to the target area, it is determined that the point cloud data subset does not belong to the target point cloud data subset. If the feature point area corresponding to the point cloud data subset does not belong to the target area, the visibility characterization value of each feature point on the edge of the feature point area corresponding to the point cloud data subset is first determined, and then the variance of the visibility characterization value of each feature point is calculated to obtain the corresponding characterization value variance. Thereafter, the characterization value variance is compared with a preset threshold. If the characterization value variance is less than or equal to the preset threshold, it can be determined that the point cloud data subset belongs to the target point cloud data subset. If the characterization value variance is greater than the preset threshold, it can be determined that the point cloud data subset does not belong to the target point cloud data subset.

[0109] Among them, in an alternative example, the target area may be a mouth area, etc. Thus, by excluding the mouth area, that is, since the variation range of the mouth area is generally large, it is not necessary to determine the visibility information of the mouth area. Thus, the reliability of the visibility information of the determined feature point area in reflecting the visibility of the corresponding feature point area can be improved.

[0110] Optionally, for step S122 in the above example, the specific method of determining the visibility information of the feature point area corresponding to the target point cloud data subset is not limited.

[0111] For example, in an alternative example, in order to improve the reliability of the visibility information of the determined feature point area, Figure 7 , step S122 may include step S122a and step S122b.

[0112] Step S122a: for each of the target point cloud data subsets, determining at least one corresponding feature point plane in the feature point region corresponding to the target point cloud data subset.

[0113] In the embodiment of the present application, after the target point cloud data subset is determined based on step S121, at least one corresponding feature point plane can be determined in the feature point region corresponding to each target point cloud data subset. Each of the feature point planes includes multiple feature points in the feature point region.

[0114] Step S122b, for each of the target point cloud data subsets, determine the normal vector of each of the feature point planes corresponding to the target point cloud data subset, and determine the visibility information of the feature point area corresponding to the target point cloud data subset based on the normal vector of each of the feature point planes.

[0115] In an embodiment of the present application, after determining at least one feature point plane corresponding to the target point cloud data subset based on step S122a, the normal vector of each feature point plane corresponding to the target point cloud data subset can be determined for each target point cloud data subset, and the visibility information of the feature point area corresponding to the target point cloud data subset can be determined based on the normal vector of each feature point plane.

[0116] It can be understood that for step S122a in the above example, in an alternative example, for each of the target point cloud data subsets, a feature point plane can be determined in the feature point area corresponding to the target point cloud data subset, such as forming a feature point plane based on the leftmost, topmost and rightmost points in the feature point area, and the feature point plane can be understood as a larger triangular face (as described above).

[0117] It can be understood that, for step S122b in the above example, in an alternative example, for each of the target point cloud data subsets, the average value of the normal vector of each of the feature point planes corresponding to the target point cloud data subset can be calculated first (if there is only one feature point plane, the average value of the normal vector is the normal vector itself), and then, based on the angle between the average value of the normal vector and the optical axis of the camera, the visibility information of the feature point area corresponding to the target point cloud data subset is determined; if the angle is greater than a preset angle, the visibility information of the feature point area is invisible; if the angle is less than or equal to the preset angle, the visibility information of the feature point area is visible.

[0118] Thirdly, it should be noted that for step S130, the specific method of determining the visibility information of each feature point corresponding to the point cloud data subset to be processed based on step S130 is not limited and can be selected according to actual application requirements.

[0119] For example, in an alternative example, the target point cloud data subset corresponding to the invisible feature point area represented by the visibility information can be directly determined as the point cloud data subset to be processed, and the target point cloud data subset corresponding to the visible feature point area represented by the visibility information can be directly determined as the point cloud data subset not to be processed. Then, only the visibility information of each feature point corresponding to the point cloud data subset to be processed is further determined, and for the point cloud data subset not to be processed, the visibility information of each feature point corresponding to the point cloud data subset not to be processed can be directly determined as visible, that is, the visibility information of the feature point area corresponding to the point cloud data subset not to be processed is directly used to represent the visibility information of each corresponding feature point, so as to reduce the amount of data processing.

[0120] For example, in another alternative example, in order to improve the reliability of the visibility information of the determined feature points, Figure 8 Step S130 may include step S131, step S132 and step S133, the specific contents of which are described as follows.

[0121] Step S131 : for each of the target point cloud data subsets, if the visibility information of the feature point region corresponding to the target point cloud data subset indicates that the feature point region is invisible, the target point cloud data subset is determined as the point cloud data subset to be processed.

[0122] In an embodiment of the present application, after obtaining the visibility information of the feature point area corresponding to each target point cloud data subset based on step S120, for each of the target point cloud data subsets, if the visibility information of the feature point area corresponding to the target point cloud data subset indicates that the feature point area is invisible, then the target point cloud data subset is determined as the point cloud data subset to be processed.

[0123] Step S132, for each of the target point cloud data subsets, if the visibility information of the feature point area corresponding to the target point cloud data subset indicates that the feature point area is visible, determine whether the adjacent target point cloud data subset of the target point cloud data subset belongs to the point cloud data subset to be processed, and when the adjacent target point cloud data subset belongs to the point cloud data subset to be processed, determine the target point cloud data subset as the point cloud data subset to be processed.

[0124] In an embodiment of the present application, after obtaining the visibility information of the feature point area corresponding to each target point cloud data subset based on step S120, for each of the target point cloud data subsets, if the visibility information of the feature point area corresponding to the target point cloud data subset indicates that the feature point area is visible, it is determined whether an adjacent target point cloud data subset of the target point cloud data subset belongs to the point cloud data subset to be processed, and when the adjacent target point cloud data subset belongs to the point cloud data subset to be processed, the target point cloud data subset is determined as the point cloud data subset to be processed.

[0125] Step S133 : for each of the point cloud data subsets to be processed, visibility information of each corresponding feature point is determined based on the point data in the point cloud data subset to be processed.

[0126] In an embodiment of the present application, after the subset of point cloud data to be processed is determined based on step S131 and step S132, the visibility information of each corresponding feature point can be determined for each subset of point cloud data to be processed based on the point data in the subset of point cloud data to be processed. For example, for each feature point, the average value of the normal vector of each triangular facet formed by the feature point and adjacent feature points is determined based on the point data of the feature point, and then, whether the feature point is visible is determined based on the size of the angle between the average value of the normal vector and the optical axis of the camera.

[0127] Based on this, in the above example, for each feature point corresponding to the point cloud data subset to be processed, the visibility of the feature point can be determined based on the corresponding point data to improve the accuracy of the determined visibility. For each feature point corresponding to the non-point cloud data subset to be processed, the visibility of the corresponding feature point area can be directly determined as the visibility of each corresponding feature point to improve the efficiency of data processing. In this way, the visibility of the feature points corresponding to all target point cloud data subsets can be determined. For other point cloud data subsets other than the target point cloud data subset, such as the aforementioned mouth and other areas with large changes or areas with large variances in the visibility characterization values ​​of feature points at the edges of the areas, the visibility of the feature points can be determined based on the corresponding point data (as mentioned above, determined based on the normal vectors of the corresponding triangular facets) to ensure the accuracy of the determined visibility.

[0128] Combination Fig. 9 The embodiment of the present application further provides a point cloud data processing device 100 applicable to the electronic device 10. The point cloud data processing device 100 may include a data subset partitioning module 110, a region visibility determination module 120 and a feature point visibility determination module 130.

[0129] The data subset division module 110 is used to divide the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each point cloud data subset includes point data of a plurality of feature points. In this embodiment, the data subset division module 110 can be used to perform Figure 2 As shown in step S110, the relevant contents of the data subset partitioning module 110 can refer to the above description of step S110.

[0130] The region visibility determination module 120 is used to determine the visibility information of the feature point region corresponding to each target point cloud data subset in the plurality of point cloud data subsets, wherein the target point cloud data subset is one or more point cloud data subsets in the plurality of point cloud data subsets. In this embodiment, the region visibility determination module 120 can be used to perform Figure 2 As shown in step S120, the relevant contents about the area visibility determination module 120 can refer to the above description of step S120.

[0131] The feature point visibility determination module 130 is used to determine the visibility information of each corresponding feature point based on the point data in each of the point cloud data subsets to be processed when it is determined that there is at least one to-be-processed point cloud data subset based on the visibility information, wherein the to-be-processed point cloud data subset belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the to-be-processed point cloud data subset meets the target visibility condition. In this embodiment, the feature point visibility determination module 130 can be used to perform Figure 2 As shown in step S130 , the relevant contents of the feature point visibility determination module 130 can refer to the above description of step S130 .

[0132] In the embodiment of the present application, corresponding to the above-mentioned point cloud data processing method applied to the electronic device 10, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the above-mentioned point cloud data processing method is executed. Among them, each step executed when the aforementioned computer program is run will not be described one by one here, and reference can be made to the above explanation of the point cloud data processing method.

[0133] In summary, the point cloud data processing method and device, electronic device and storage medium provided by the present application can first divide the point cloud data set to be processed into multiple point cloud data subsets, so that the visibility information of the feature point area corresponding to each target point cloud data subset in the multiple point cloud data subsets can be determined first, and then, in the target point cloud data subset, when there is a point cloud data subset to be processed whose visibility information of the corresponding feature point area meets the target visibility condition, the visibility information of each feature point in the point cloud data subset to be processed is determined respectively, that is, when the visibility information of the feature point area meets the target visibility condition, the visibility of each feature point in the feature point area is determined respectively. Based on this, compared with the conventional technical solution of directly traversing each feature point to determine the visibility of each feature point respectively, the technical solution provided by the present application is adopted. Since a visibility determination mechanism of the feature point area is configured before determining the visibility of each feature point, the number of feature points that need to be directly determined based on the corresponding point data can be reduced to a certain extent. Therefore, the amount of data for point cloud data processing can be reduced and the efficiency of point cloud data processing can be improved, thereby improving the problem of poor point cloud data processing effect in the prior art.

[0134] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0136] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0137] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud data processing method, characterized in that: include: Dividing the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each of the point cloud data subsets includes point data of a plurality of feature points; For each target point cloud data subset in the plurality of point cloud data subsets, determine visibility information of a feature point area corresponding to the target point cloud data subset, wherein the target point cloud data subset is one or more of the plurality of point cloud data subsets; If it is determined based on the visibility information that there is at least one subset of point cloud data to be processed, then for each of the subsets of point cloud data to be processed, the visibility information of each corresponding feature point is determined based on the point data in the subset of point cloud data to be processed, wherein the subset of point cloud data to be processed belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the subset of point cloud data to be processed satisfies the target visibility condition; The step of determining visibility information of a feature point area corresponding to each target point cloud data subset in the plurality of point cloud data subsets comprises: For each point cloud data subset of the plurality of point cloud data subsets, determining whether point data of feature points at the edge of a feature point region corresponding to the point cloud data subset meets a preset target data condition, and determining the point cloud data subset as a target point cloud data subset when the point data of feature points at the edge of the feature point region corresponding to the point cloud data subset does not meet the target data condition; For each of the target point cloud data subsets, visibility information of a feature point region corresponding to the target point cloud data subset is determined.

2. The point cloud data processing method according to claim 1, characterized in that: The step of dividing the point cloud data set to be processed into a plurality of point cloud data subsets comprises: Determine a plurality of feature point regions based on the historical point cloud data set corresponding to the plurality of feature points, wherein the historical point cloud data set includes the historical point data of each of the feature points; For each of the feature point regions, a point cloud data subset corresponding to the feature point region is formed based on the point data of each feature point in the point cloud data set to be processed that belongs to the feature point region.

3. The point cloud data processing method according to claim 2, characterized in that: The step of determining a plurality of feature point regions based on the historical point cloud data set corresponding to the plurality of feature points comprises: Acquire a historical point cloud data set corresponding to the plurality of feature points; Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine a visibility representation value corresponding to each of the feature points and a spatial distance representation value corresponding to each two of the feature points, and determine a cluster distance value between each two of the feature points based on the visibility representation value and the spatial distance representation value; The plurality of feature points are clustered based on the cluster distance value between every two of the feature points to obtain a plurality of corresponding feature point regions.

4. The point cloud data processing method according to claim 3, characterized in that: The step of determining the visibility representation value corresponding to each feature point and the spatial distance representation value corresponding to each two feature points based on the historical point data of each feature point included in the historical point cloud data set, and determining the cluster distance value between each two feature points based on the visibility representation value and the spatial distance representation value, comprises: Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine the normal vector corresponding to each of the feature points, and based on the angle between the corresponding normal vector and the optical axis of the camera, respectively determine the visibility representation value corresponding to each of the feature points; Based on the historical point data of each of the feature points included in the historical point cloud data set, respectively determine the spatial distance between every two of the feature points, and based on the spatial distance and the number of feature point regions to be clustered, respectively determine the spatial distance representation value corresponding to each two of the feature points; For every two feature points among the plurality of feature points, a cluster distance value between the two feature points is determined based on the visibility representation values ​​corresponding to the two feature points and the spatial distance representation value corresponding to the two feature points.

5. The point cloud data processing method according to claim 1, characterized in that: The step of determining visibility information of a feature point area corresponding to each of the target point cloud data subsets comprises: For each of the target point cloud data subsets, determining at least one corresponding feature point plane in the feature point region corresponding to the target point cloud data subset, wherein each of the feature point planes includes a plurality of feature points in the feature point region; For each of the target point cloud data subsets, a normal vector of each of the feature point planes corresponding to the target point cloud data subset is determined, and visibility information of the feature point area corresponding to the target point cloud data subset is determined based on the normal vector of each of the feature point planes.

6. The point cloud data processing method according to any one of claims 1 to 4, characterized in that: If it is determined based on the visibility information that there is at least one subset of point cloud data to be processed, then for each subset of point cloud data to be processed, the step of determining the visibility information of each corresponding feature point based on the point data in the subset of point cloud data to be processed comprises: For each of the target point cloud data subsets, if the visibility information of the feature point region corresponding to the target point cloud data subset indicates that the feature point region is invisible, then the target point cloud data subset is determined as the point cloud data subset to be processed; For each of the target point cloud data subsets, if the visibility information of the feature point area corresponding to the target point cloud data subset indicates that the feature point area is visible, determining whether an adjacent target point cloud data subset of the target point cloud data subset belongs to the point cloud data subset to be processed, and if the adjacent target point cloud data subset belongs to the point cloud data subset to be processed, determining the target point cloud data subset as the point cloud data subset to be processed; For each of the to-be-processed point cloud data subsets, visibility information of each corresponding feature point is determined based on the point data in the to-be-processed point cloud data subset.

7. A point cloud data processing device, characterized in that: include: A data subset division module, used for dividing the point cloud data set to be processed into a plurality of point cloud data subsets, wherein each of the point cloud data subsets includes point data of a plurality of feature points; A region visibility determination module, configured to determine, for each target point cloud data subset in the plurality of point cloud data subsets, visibility information of a feature point region corresponding to the target point cloud data subset, wherein the target point cloud data subset is one or more point cloud data subsets in the plurality of point cloud data subsets; A feature point visibility determination module, for determining visibility information of each corresponding feature point based on point data in each of the point cloud data subsets to be processed when it is determined that there is at least one to-be-processed point cloud data subset based on the visibility information, wherein the to-be-processed point cloud data subset belongs to the target point cloud data subset, and the visibility information of the feature point area corresponding to the to-be-processed point cloud data subset satisfies the target visibility condition; The area visibility determination module is also used to determine, for each point cloud data subset among the multiple point cloud data subsets, whether the point data of the edge feature points in the feature point area corresponding to the point cloud data subset meets the preset target data conditions, and when the point data of the edge feature points in the feature point area corresponding to the point cloud data subset does not meet the target data conditions, determine the point cloud data subset as a target point cloud data subset; for each of the target point cloud data subsets, determine the visibility information of the feature point area corresponding to the target point cloud data subset.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the point cloud data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is run, each step included in the point cloud data processing method according to any one of claims 1 to 6 is executed.

Images