Three-dimensional data encoding method for multi-mode physical simulation system based on neural network

Abstract

The present application relates to the field of three-dimensional data encoding technology, and specifically to a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network, the method comprising: obtaining a three-dimensional point cloud at each moment in the multi-mode physical simulation system; using feature point detection and clustering algorithms, according to the distribution characteristics of the three-dimensional point cloud, combined with the clustering cluster where the three-dimensional point cloud is located and the distribution characteristics of feature points in a local range, obtaining an information distribution characteristic value; according to the change characteristics of the three-dimensional point cloud distribution over a period of time, combined with the change characteristics of the clustering cluster where the three-dimensional point cloud is located, obtaining an information change characteristic value; combining the information distribution characteristic value, dividing the three-dimensional point cloud, and encoding all three-dimensional point clouds at each moment using a compression algorithm. The present application aims to dynamically adjust the compression of the three-dimensional point cloud based on the effective information contained in the three-dimensional point cloud, and enhance the timeliness, accuracy and efficiency of the multi-mode physical simulation system.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional data encoding, and in particular to a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network. Background Art

[0002] With the development of science and technology, multi-mode physical simulation systems have developed rapidly. By using multi-mode 3D point cloud acquisition equipment to obtain a large number of 3D point clouds, encoding them and transmitting them to the simulation system, the physical simulation is completed in combination with the 3D reconstruction method. It is widely used in engineering design research in aerospace, electrical, chemical, communication, and especially military fields, and has become one of the representatives of modern high technology. In addition, with the increasing requirements for the immediacy and efficiency of multi-mode physical simulation systems, the requirements for the encoding efficiency of 3D point clouds in the system are also gradually increasing.

[0003] Generally speaking, the current 3D point cloud encoding method mainly uses the Google Draco library to encode and compress the 3D point cloud. However, in the actual acquisition process of the 3D point cloud, there will be a lot of redundant information in the dense 3D point cloud. When encoding these 3D point clouds, the same encoding compression level is usually used, which leads to more time and resources being consumed in the encoding process, and ultimately affects the immediacy and efficiency of the 3D reconstruction of the simulation system. Summary of the invention

[0004] In view of the above, it is necessary to provide a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network to solve the above problems.

[0005] An embodiment of the present application provides a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network, the method comprising:

[0006] Obtain the three-dimensional point cloud at each moment in the multi-mode physical simulation system;

[0007] Using feature point detection and clustering algorithms for all 3D point clouds at each moment, feature points and clusters at each moment are obtained; based on the quantity characteristics of the 3D point clouds at each moment, combined with the clusters where each 3D point cloud is located and the spatial distribution characteristics of the feature points in the local range, the information distribution feature values ​​of each 3D point cloud at each moment are obtained;

[0008] According to the change in the number of three-dimensional point clouds at each moment and in the previous local time, combined with the change in the number of three-dimensional point clouds in the cluster where each three-dimensional point cloud is located, the information change characteristic value of each three-dimensional point cloud at each moment is obtained; according to the information change characteristic value of all three-dimensional point clouds at each moment, combined with the information distribution characteristic value, all three-dimensional point clouds at each moment are divided;

[0009] The compression algorithm is used to encode the three-dimensional point cloud divided at each time.

[0010] The step of obtaining the information distribution characteristic value of each three-dimensional point cloud at each time comprises:

[0011] At each moment, based on the number of three-dimensional point clouds and feature points within a local range of each three-dimensional point cloud, combined with the distance characteristics between each three-dimensional point cloud and the feature points, first distribution information of each three-dimensional point cloud is obtained;

[0012] Based on the distance characteristics between each three-dimensional point cloud and other three-dimensional point clouds, and the distance characteristics between the cluster where each three-dimensional point cloud is located and other clusters, the second distribution information of each three-dimensional point cloud at each moment is obtained;

[0013] Based on the first distribution information and the second distribution information of each three-dimensional point cloud at each moment, the information distribution characteristic value of each three-dimensional point cloud at each moment is obtained, wherein the information distribution characteristic value is positively correlated with the first distribution information and negatively correlated with the second distribution information.

[0014] The specific process of obtaining the first distribution information of each three-dimensional point cloud is as follows:

[0015] Calculate the sum of the number of three-dimensional point clouds and the number of feature points in the local range of each three-dimensional point cloud, and record it as the first sum; calculate the mean of the distance measurement between each three-dimensional point cloud and all feature points, and record it as the first mean; and use the inverse proportional mapping result of the first sum and the first mean as the first distribution information of each three-dimensional point cloud at each moment.

[0016] The step of obtaining the second distribution information of each three-dimensional point cloud at each time is as follows:

[0017] For each moment, the local density value of each three-dimensional point cloud is obtained; the mean of the distance measurement between each three-dimensional point cloud and all other three-dimensional point clouds is calculated, which is recorded as the second mean; the mean of the distance measurement between the cluster where each three-dimensional point cloud is located and all other clusters is obtained, which is recorded as the third mean; the second mean, the local density value and the third mean of each three-dimensional point cloud are merged to obtain the second distribution information of each three-dimensional point cloud at each moment.

[0018] The information change characteristic value of each three-dimensional point cloud at each time is obtained as follows:

[0019] For each moment and the previous local time, according to the change in the number of feature points and clusters within the local range of each three-dimensional point cloud, the first information change value of each three-dimensional point cloud at each moment is obtained;

[0020] According to the discrete degree of the change in the number of three-dimensional point clouds within the local range of each three-dimensional point cloud and the average level of the number of three-dimensional point clouds, a second information change value of each three-dimensional point cloud at each moment is obtained;

[0021] Based on the inverse proportional mapping result of the first information change value and the second information change value of each three-dimensional point cloud at each moment, the information change characteristic value of each three-dimensional point cloud at each moment is obtained.

[0022] The process of obtaining the first information change value of each three-dimensional point cloud at each moment is specifically as follows:

[0023] For each moment and a preset number of moments before, obtain the change in the number of feature points of each three-dimensional point cloud in the local range at adjacent moments, which is recorded as the first change; obtain the change in the number of three-dimensional point clouds in the cluster where each three-dimensional point cloud is located at adjacent moments, which is recorded as the second change; merge the average level of the first change at all adjacent moments with the average level of the second change to obtain the first information change value of each three-dimensional point cloud at each moment.

[0024] The second information change value of each three-dimensional point cloud at each moment is obtained as follows:

[0025] For each moment and a preset number of moments before, the discrete degree of change in the number of three-dimensional point clouds in the local range of all adjacent moments is fused with the average level of the number of three-dimensional point clouds in the local range of all moments to obtain the second information change value of each three-dimensional point cloud at each moment.

[0026] The division of all three-dimensional point clouds at each moment is specifically as follows:

[0027] According to the information change characteristic value of each three-dimensional point cloud at each moment, the information distribution characteristic value is corrected;

[0028] Obtain the segmentation threshold of the corrected information distribution characteristic value of all three-dimensional point clouds at each moment; classify the three-dimensional point clouds whose corrected information distribution characteristic value is greater than or equal to the segmentation threshold as important information points; otherwise, classify the three-dimensional point clouds as weak information points.

[0029] The correcting of the information distribution characteristic value specifically includes: taking the product of the information change characteristic value of each three-dimensional point cloud at each moment and the information distribution characteristic value as the corrected information distribution characteristic value of each three-dimensional point cloud at each moment.

[0030] The method of encoding the three-dimensional point cloud divided at each time by using a compression algorithm is as follows:

[0031] The inverse proportional results of the mean values ​​of the corrected information distribution characteristic values ​​of all important information points and all weak information points at each moment are calculated respectively, and are merged with the default compression level to obtain the compression level of all important information points and all weak information points at each moment; using the compression level at each moment and a compression algorithm, the encoding of all important information points and all weak information points is obtained.

[0032] This application has at least the following beneficial effects:

[0033] The present application provides a data basis for the subsequent acquisition of the features of the three-dimensional point cloud by analyzing the three-dimensional point cloud of the multi-mode physical simulation system; performs feature point detection on the three-dimensional point cloud, which helps to identify and extract points with significant features, and clusters and identifies dense areas in the three-dimensional point cloud, which may contain important three-dimensional point clouds; constructs information distribution feature values, which has the beneficial effect of analyzing the amount of information contained in the three-dimensional point cloud and the effectiveness of the information contained in the three-dimensional point cloud through the distribution of the three-dimensional point cloud; since the amount of information contained in the three-dimensional point cloud will change with the movement of the acquisition device during the acquisition of the three-dimensional point cloud, constructs information change feature values, which has the beneficial effect of measuring the degree of change in the amount of information contained in the three-dimensional point cloud, and judging the importance of the three-dimensional point cloud according to the degree of change in the amount of information; further divides the three-dimensional point cloud, and completes encoding according to the divided three-dimensional point cloud, which has the beneficial effect of improving the accuracy of the decoded three-dimensional point cloud while ensuring the compression efficiency of the three-dimensional point cloud, and dynamically adjusting the compression algorithm according to the actual information of the data, thereby improving the compression efficiency and improving the response speed, data accuracy and processing efficiency of the multi-mode physical simulation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network provided in this application;

[0035] Figure 2 A flowchart for obtaining the information distribution characteristic value provided by this application;

[0036] Figure 3 A flowchart for obtaining the information change characteristic value provided by this application;

[0037] Figure 4 A flowchart for obtaining the modified information distribution characteristic value provided in this application. DETAILED DESCRIPTION

[0038] In the description of the embodiments of the present application, words such as "exemplary", "or", "for example" and the like are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "or", "for example" and the like is intended to present related concepts in a concrete manner.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art in the present application. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0040] It should also be noted that the terms "first" and "second" in this application and its drawings are used to distinguish similar objects, rather than to describe a specific order or sequence. The method disclosed in the embodiments of the present application or the method shown in the flow chart includes one or more steps for implementing the method. Without departing from the scope of protection of this application, the execution order of multiple steps can be interchanged with each other, and some steps can also be deleted.

[0041] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0042] This application first proposes a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network, which is applied to the technical field of three-dimensional data encoding. Figure 1 , the method comprises the following steps:

[0043] S1: Obtain the three-dimensional point cloud at each moment in the multi-mode physical simulation system.

[0044] In the multi-mode physical simulation system, a laser radar is used to collect a three-dimensional point cloud every time interval T, and the three-dimensional point cloud is filtered using bilateral filtering. All three-dimensional point clouds are used as input, and the multi-modal three-dimensional point cloud registration neural network model (Multiply Spatial Fusion Network, MSFNet) is used. The cross entropy is used as the loss function and the SGD algorithm is used as the optimizer to register the three-dimensional point cloud. Then, the iterative closest point (Iterative Closest Point, ICP) algorithm in the PCL point cloud library is used to combine the registered data to align the three-dimensional point cloud in the same coordinate system, and the splicing and fusion of the three-dimensional point cloud is realized, and the output is a three-dimensional point cloud that has been spliced ​​and fused. Among them, bilateral filtering, MSFNet model, and ICP algorithm are all well-known technologies, and this application will not repeat them; in addition, the implementer can use other methods to process and splice the three-dimensional point cloud according to the actual situation, and this application does not limit this.

[0045] In this embodiment, T=0.01s; the implementer can adjust it by himself, and this application does not impose any limitation on this.

[0046] S2: Use feature point detection and clustering algorithm for all 3D point clouds at each moment to obtain the feature points and clusters at each moment; according to the quantity characteristics of the 3D point clouds at each moment, combined with the clusters where each 3D point cloud is located and the spatial distribution characteristics of the feature points in the local range, obtain the information distribution characteristic value of each 3D point cloud at each moment.

[0047] Taking a single moment as an example, in a 3D point cloud, valid data mainly includes information such as the edge and plane of the scanned object. This information is crucial in the process of 3D reconstruction and affects the effect of 3D reconstruction to a large extent. These valid 3D point clouds are mainly characterized by dense spatial distribution. The positions of similar 3D point clouds have certain regularity. For example, if the scanned object is a box, the coordinate height of the 3D point cloud at the top edge of the box is relatively consistent. For a single 3D point cloud at the edge, the more 3D point clouds that are consistent with one dimension of its 3D coordinate information, the more valid information the 3D point cloud contains. There may be some invalid data in the 3D point cloud, mainly due to noise generated during the scanning process, the object is too far away from the acquisition device or is blocked by other objects, resulting in incomplete scanning results. These invalid data points are usually manifested as a small amount of scattered data, which is far away from the rest of the 3D point cloud and unevenly distributed.

[0048] At each moment, the feature points in all three-dimensional point clouds are obtained. At each moment, based on the number of three-dimensional point clouds and feature points in the local range of each three-dimensional point cloud, combined with the distance characteristics between each three-dimensional point cloud and the feature points, the first distribution information of each three-dimensional point cloud is obtained:

[0049] In this embodiment, all three-dimensional point clouds are input into the Harris 3D corner detection algorithm, and the three-dimensional point clouds as corner points are output. The Harris 3D corner detection algorithm is a well-known technology and will not be described in detail in this application. A cube window is used as the local range of each three-dimensional point cloud; the implementer can adjust the size of the cube window by himself, and this application does not impose any restrictions on this.

[0050] Calculate the sum of the number of three-dimensional point clouds and the number of feature points in the local range of each three-dimensional point cloud, and record it as the first sum; calculate the mean of the distance measurement between each three-dimensional point cloud and all feature points, and record it as the first mean; and use the inverse proportional mapping result of the first sum and the first mean as the first distribution information of each three-dimensional point cloud at each moment.

[0051] It should be noted that the distance measurement between two points in this embodiment adopts Euclidean distance; in other embodiments, Euclidean distance can be replaced by Manhattan distance. The first distribution information refers to the ratio of the first sum value to the first mean value; in other embodiments, the first sum value is recorded as B, the first mean value is recorded as C, and the formula form of the first distribution information is ,in, is an exponential function with a natural constant as its base.

[0052] It should be understood that the larger the first distribution information of a single three-dimensional point cloud is, the closer the location of the three-dimensional point cloud is to the edge or key point of the scanned object, the larger the amount of information contained is, and the higher the importance is.

[0053] Based on the distance features between each 3D point cloud and other 3D point clouds, and the distance features between the cluster where each 3D point cloud is located and other clusters, the second distribution information of each 3D point cloud at each moment is obtained:

[0054] Based on each moment, the local density value of each three-dimensional point cloud is obtained; the mean of the distance measurement between each three-dimensional point cloud and all other three-dimensional point clouds is calculated, which is recorded as the second mean; the mean of the distance measurement between the cluster where each three-dimensional point cloud is located and all other clusters is obtained, which is recorded as the third mean; the second mean, the local density value and the third mean of each three-dimensional point cloud are fused to obtain the second distribution information of each three-dimensional point cloud at each moment.

[0055] In this embodiment, all three-dimensional point clouds collected at each moment are clustered using the DPC clustering algorithm, where the cutoff distance is set to 3, and the output is a number of clusters, and the local density value of each three-dimensional point cloud at each moment is obtained; wherein the DPC clustering algorithm is a well-known technology and is not described in detail in this application. The second mean refers to the mean of the Euclidean distance between each three-dimensional point cloud and all other three-dimensional point clouds at each moment, denoted by E; the local density value is denoted by F, and the third mean refers to the mean of the Euclidean distance between the cluster where each three-dimensional point cloud is located and all other clusters at each moment, denoted by G; the formula form of the second distribution information is: .

[0056] It should be noted that the method for obtaining the Euclidean distance between clusters is: obtaining the central three-dimensional point cloud of the cluster, and taking the Euclidean distance between the central three-dimensional point clouds of two clusters as the Euclidean distance between the clusters.

[0057] It should be understood that the larger the second distribution information of the three-dimensional point cloud is, the larger the distance between the three-dimensional point cloud and the rest of the data is, and the larger the distance between the cluster where the three-dimensional point cloud is located and the rest of the clusters is, and the more scattered the distribution is.

[0058] Based on the first distribution information and the second distribution information of each three-dimensional point cloud at each moment, the information distribution characteristic value of each three-dimensional point cloud at each moment is obtained, wherein the information distribution characteristic value is positively correlated with the first distribution information and negatively correlated with the second distribution information.

[0059] In this embodiment, the information distribution characteristic value of the three-dimensional point cloud can be obtained by calculating the ratio of the first distribution information of the three-dimensional point cloud to the second distribution information; in other embodiments, the ratio can be replaced by the absolute value of the difference; in some other embodiments, the difference between the first distribution information and the second distribution information of the three-dimensional point cloud can be calculated, recorded as H, and exp(H) is used as the information distribution characteristic value of the three-dimensional point cloud, where, is an exponential function with a natural constant as its base.

[0060] Among them, the flow chart of obtaining the information distribution characteristic value is as follows: Figure 2 shown.

[0061] It should be understood that the larger the first distribution information of the three-dimensional point cloud is, the more effective information it contains and the greater its importance for subsequent three-dimensional reconstruction. When encoding it, the compression level should be adjusted to improve data accuracy. The larger the second distribution information of the three-dimensional point cloud is, the more likely it is that it is data with lower effectiveness in the three-dimensional point cloud. The larger the information distribution eigenvalue of the three-dimensional point cloud is, the closer the distribution of the three-dimensional point cloud is to the distribution law of effective data, the more effective information the three-dimensional point cloud contains and the greater its importance for subsequent three-dimensional reconstruction. When encoding it, the compression level should be adjusted to improve data accuracy.

[0062] S3: According to the change in the number of three-dimensional point clouds at each moment and in the previous local time, combined with the change in the number of three-dimensional point clouds in the cluster where each three-dimensional point cloud is located, obtain the information change characteristic value of each three-dimensional point cloud at each moment; according to the information change characteristic value of all three-dimensional point clouds at each moment, combined with the information distribution characteristic value, divide all three-dimensional point clouds at each moment.

[0063] Under normal circumstances, due to the limited scanning range of the acquisition device, in order to better reconstruct the scanned object in three dimensions, the acquisition device needs to be moved for scanning. During the movement, objects in the distance gradually approach and gradually enter the scanning range of the acquisition device completely. As a result, the three-dimensional point cloud collected by the object also gradually increases. Therefore, it is more likely that data errors will occur during the three-dimensional reconstruction, which will ultimately affect the quality of the three-dimensional reconstruction. Therefore, further judgment is needed.

[0064] Specifically, as the acquisition device moves, the 3D point cloud at the boundary of the acquisition device will change. Specifically, the 3D point cloud of objects approaching the acquisition device increases, and the texture details increase, while the 3D point cloud of objects moving away from the acquisition device decreases. However, in the process of the acquisition device moving, it is usually planned to move to a certain place. Therefore, during the movement, the objects on the side boundary of the acquisition device are usually not the focus of this scan. Therefore, even if the 3D point clouds of these objects will also change, they are difficult to use in 3D reconstruction and are still not valid 3D point clouds. Specifically, the amount of change is small and the change speed is fast.

[0065] Based on this, for each moment and the previous local time, according to the changes in the number of feature points and clusters within the local range of each three-dimensional point cloud, the first information change value of each three-dimensional point cloud at each moment is obtained:

[0066] For each moment and a preset number of moments before, obtain the change in the number of feature points of each 3D point cloud in the local range of the adjacent moments, record it as the first change, and record the average level of the first change as I; obtain the change in the number of 3D point clouds in the cluster where each 3D point cloud is located at the adjacent moments, record it as the second change, and record the average level of the second change as J; in this embodiment, the preset number is 10, and the average level is specifically the average value; the change is obtained by using the calculation method of the absolute value of the difference, and The result is taken as the first information change value of the three-dimensional point cloud at each moment; in other embodiments, the preset number is 15, which can be Replace with , get the first information change value of the three-dimensional point cloud at each moment.

[0067] It should be noted that if data at sufficient times cannot be obtained, mean filling is used to fill the data. Mean filling is a well-known technology and will not be described in detail in this application.

[0068] It should be understood that the greater the first information change value of the three-dimensional point cloud at each moment, the greater the change in the amount of information contained in it becomes as the multi-mode three-dimensional point cloud acquisition device moves, indicating that the corresponding object at the location of the three-dimensional point cloud at that moment is located in front of or behind the multi-mode three-dimensional point cloud acquisition device, and is more important in the three-dimensional reconstruction process, and the greater the effectiveness of the three-dimensional point cloud.

[0069] According to the discrete degree of the change in the number of 3D point clouds in the local range of each 3D point cloud and the average level of the number of 3D point clouds, the second information change value of each 3D point cloud at each moment is obtained:

[0070] For each moment and a preset number of moments before, the discrete degree of the change in the number of three-dimensional point clouds in the cube window of all adjacent moments is recorded as K, and the average level of the number of three-dimensional point clouds in the cube window of all moments is recorded as L; in this embodiment, the discrete degree is obtained by variance, and the average level is obtained by average value. The result is used as the second information change value of the three-dimensional point cloud at each moment; in other embodiments, the discrete degree is obtained by the extreme value, and the average level is obtained by the median. The result is used as the second information change value of the three-dimensional point cloud at each moment.

[0071] It should be understood that the larger the second information change value of the three-dimensional point cloud at each moment is, the smaller the change of the three-dimensional point cloud around it is as the multi-mode three-dimensional point cloud acquisition device moves, and the smaller the total amount of data points around the three-dimensional point cloud is, the less information the three-dimensional point cloud contains.

[0072] Based on the inverse proportional mapping results of the first information change value and the second information change value of each 3D point cloud at each moment, the information change characteristic value of each 3D point cloud at each moment is obtained:

[0073] In this embodiment, the information change characteristic value of each three-dimensional point cloud at each moment is specifically the ratio of the first information change value to the second information change value of each three-dimensional point cloud at each moment; in other embodiments, the ratio can be replaced by the absolute value of the difference.

[0074] Among them, the flow chart of obtaining the characteristic value of information change is as follows: Figure 3 shown.

[0075] It should be understood that, when the information change characteristic value of the three-dimensional point cloud is larger, it means that the amount of information contained in the three-dimensional point cloud gradually increases as the multi-mode three-dimensional point cloud acquisition device moves, and the shorter the duration of the change of the three-dimensional point cloud and the smaller the degree of change, the more likely it is that it is a three-dimensional point cloud of an object in the moving direction of the acquisition device, and the higher the importance of the three-dimensional point cloud, the more it should be defined as valid data.

[0076] According to the information change characteristic value of each three-dimensional point cloud at each time, the information distribution characteristic value is corrected; based on the corrected information distribution characteristic value, all three-dimensional point clouds at each time are divided:

[0077] In this embodiment, the product of the information change characteristic value of each three-dimensional point cloud at each moment and the information distribution characteristic value is used as the corrected information distribution characteristic value; the corrected information distribution characteristic value of all three-dimensional point clouds at each moment is used as input, and the segmentation threshold at each moment is obtained by cross-validation. When the corrected information distribution characteristic value of the three-dimensional point cloud is greater than or equal to the segmentation threshold, the corresponding three-dimensional point cloud is an important information point; otherwise, the corresponding three-dimensional point cloud is a weak information point. In other embodiments, a clustering method can be used to classify the three-dimensional point cloud based on the validity coefficient. Among them, cross-validation is an existing well-known technology and is not elaborated in this application.

[0078] Among them, the flow chart for obtaining the corrected information distribution characteristic value is as follows: Figure 4 shown.

[0079] It should be understood that the larger the corrected information distribution characteristic value of the three-dimensional point cloud is, the more effective information the three-dimensional point cloud contains. When encoding it, the compression level should be adjusted to improve data accuracy to ensure the reliability and accuracy of the data during subsequent three-dimensional reconstruction.

[0080] S4: Encode the three-dimensional point cloud divided at each time using a compression algorithm.

[0081] The inverse proportional results of the mean values ​​of the corrected information distribution characteristic values ​​of all important information points and all weak information points at each moment are calculated respectively, and are merged with the default compression level to obtain the compression level of all important information points and all weak information points at each moment; using the compression level at each moment and a compression algorithm, the encoding of all important information points and weak information points is obtained.

[0082] In this embodiment, the reciprocal of the mean of the corrected information distribution characteristic values ​​of all important information points at the current moment is calculated, and the rounded-up value of the product of the reciprocal and the default compression level is used as the compression level of all important information points; the compression level of all weak information points is obtained by the same method as the compression level of important information points, and the GoogleDraco library is used to encode all important information points and weak information points. Among them, the default compression level is 5; the GoogleDraco library is a well-known technology and will not be described in detail in this application.

[0083] It should be understood that when encoding data, the greater the compression level used, the faster the encoding compression speed and the greater the compression rate, but the accuracy of the data after decoding is lower; conversely, the smaller the compression level, the longer the encoding compression time required and the lower the compression rate, but the accuracy of the data after decoding is higher.

[0084] The codes of all important information points and weak information points at the current moment are transmitted to the multi-mode physical simulation system. After the system decodes the codes, it performs three-dimensional reconstruction using a real-time three-dimensional reconstruction method.

[0085] The embodiment of the present application provides a three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network, the method comprising: analyzing a three-dimensional point cloud of the multi-mode physical simulation system to provide a data basis for subsequent acquisition of features of the three-dimensional point cloud; performing feature point detection on the three-dimensional point cloud to help identify and extract points with significant features, and performing clustering to identify dense areas in the three-dimensional point cloud, which may contain important three-dimensional point clouds; constructing an information distribution feature value, which has the beneficial effect of analyzing the amount of information contained in the three-dimensional point cloud and the effectiveness of the information contained in the three-dimensional point cloud through the distribution of the three-dimensional point cloud; since the amount of information contained in the three-dimensional point cloud will also change with the movement of the acquisition device during the process of acquiring the three-dimensional point cloud, constructing an information change feature value, which has the beneficial effect of measuring the degree of change in the amount of information contained in the three-dimensional point cloud, and judging the importance of the three-dimensional point cloud according to the degree of change in the amount of information; further dividing the three-dimensional point cloud, and completing encoding according to the divided three-dimensional point cloud, which has the beneficial effect of improving the accuracy of the decoded three-dimensional point cloud under the premise of ensuring the compression efficiency of the three-dimensional point cloud, and dynamically adjusting the compression algorithm according to the actual information of the data, thereby improving the compression efficiency, and improving the response speed, data accuracy and processing efficiency of the multi-mode physical simulation system.

[0086] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two continuous operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

[0087] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A three-dimensional data encoding method for a multi-mode physical simulation system based on a neural network, characterized in that: The method comprises the following steps: Obtain the three-dimensional point cloud at each moment in the multi-mode physical simulation system; Using feature point detection and clustering algorithms for all 3D point clouds at each moment, feature points and clusters at each moment are obtained; based on the quantity characteristics of the 3D point clouds at each moment, combined with the clusters where each 3D point cloud is located and the spatial distribution characteristics of the feature points in the local range, the information distribution feature values ​​of each 3D point cloud at each moment are obtained; According to the change in the number of three-dimensional point clouds at each moment and in the previous local time, combined with the change in the number of three-dimensional point clouds in the cluster where each three-dimensional point cloud is located, the information change characteristic value of each three-dimensional point cloud at each moment is obtained; according to the information change characteristic value of all three-dimensional point clouds at each moment, combined with the information distribution characteristic value, all three-dimensional point clouds at each moment are divided; Using compression algorithm to encode the three-dimensional point cloud after division at each time; The step of obtaining the information distribution characteristic value of each three-dimensional point cloud at each time comprises: At each moment, based on the number of three-dimensional point clouds and feature points within a local range of each three-dimensional point cloud, combined with the distance characteristics between each three-dimensional point cloud and the feature points, first distribution information of each three-dimensional point cloud is obtained; Based on the distance characteristics between each three-dimensional point cloud and other three-dimensional point clouds, and the distance characteristics between the cluster where each three-dimensional point cloud is located and other clusters, the second distribution information of each three-dimensional point cloud at each moment is obtained; Based on the first distribution information and the second distribution information of each three-dimensional point cloud at each moment, obtaining an information distribution characteristic value of each three-dimensional point cloud at each moment, wherein the information distribution characteristic value is positively correlated with the first distribution information and negatively correlated with the second distribution information; The information change characteristic value of each three-dimensional point cloud at each time is obtained as follows: For each moment and the previous local time, according to the change in the number of feature points and clusters within the local range of each three-dimensional point cloud, the first information change value of each three-dimensional point cloud at each moment is obtained; According to the discrete degree of the change in the number of three-dimensional point clouds within the local range of each three-dimensional point cloud and the average level of the number of three-dimensional point clouds, a second information change value of each three-dimensional point cloud at each moment is obtained; Based on the inverse proportional mapping result of the first information change value and the second information change value of each three-dimensional point cloud at each moment, the information change characteristic value of each three-dimensional point cloud at each moment is obtained; The division of all three-dimensional point clouds at each moment is specifically as follows: According to the information change characteristic value of each three-dimensional point cloud at each moment, the information distribution characteristic value is corrected; Obtain the segmentation threshold of the corrected information distribution characteristic value of all three-dimensional point clouds at each moment; classify the three-dimensional point clouds whose corrected information distribution characteristic value is greater than or equal to the segmentation threshold as important information points; otherwise, classify the three-dimensional point clouds as weak information points; The compression algorithm is used to encode the three-dimensional point cloud divided at each time, specifically: The inverse proportional results of the mean values ​​of the corrected information distribution characteristic values ​​of all important information points and all weak information points at each moment are calculated respectively, and are merged with the default compression level to obtain the compression level of all important information points and all weak information points at each moment; using the compression level at each moment and a compression algorithm, the encoding of all important information points and all weak information points is obtained.

2. The three-dimensional data encoding method of a multi-mode physical simulation system based on a neural network as claimed in claim 1, characterized in that: The specific process of obtaining the first distribution information of each three-dimensional point cloud is: Calculate the sum of the number of three-dimensional point clouds and the number of feature points in the local range of each three-dimensional point cloud, which is recorded as the first sum; calculate the mean of the distance measurement between each three-dimensional point cloud and all feature points, which is recorded as the first mean; An inverse proportional mapping result of the first sum value and the first mean value is used as the first distribution information of each three-dimensional point cloud at each moment.

3. The three-dimensional data encoding method of a multi-mode physical simulation system based on a neural network as claimed in claim 1, characterized in that: The step of obtaining the second distribution information of each three-dimensional point cloud at each time is: For each moment, the local density value of each three-dimensional point cloud is obtained; the mean of the distance measurement between each three-dimensional point cloud and all other three-dimensional point clouds is calculated, which is recorded as the second mean; the mean of the distance measurement between the cluster where each three-dimensional point cloud is located and all other clusters is obtained, which is recorded as the third mean; the second mean, the local density value and the third mean of each three-dimensional point cloud are merged to obtain the second distribution information of each three-dimensional point cloud at each moment.

4. The three-dimensional data encoding method of a multi-mode physical simulation system based on a neural network as claimed in claim 1, characterized in that: The process of obtaining the first information change value of each three-dimensional point cloud at each time is specifically as follows: For each moment and a preset number of moments before, obtain the change in the number of feature points of each three-dimensional point cloud in the local range at adjacent moments, which is recorded as the first change; obtain the change in the number of three-dimensional point clouds in the cluster where each three-dimensional point cloud is located at adjacent moments, which is recorded as the second change; merge the average level of the first change at all adjacent moments with the average level of the second change to obtain the first information change value of each three-dimensional point cloud at each moment.

5. The three-dimensional data encoding method of a multi-mode physical simulation system based on a neural network as claimed in claim 1, characterized in that: The second information change value of each three-dimensional point cloud at each time is obtained as follows: For each moment and a preset number of moments before, the discrete degree of change in the number of three-dimensional point clouds in the local range of all adjacent moments is fused with the average level of the number of three-dimensional point clouds in the local range of all moments to obtain the second information change value of each three-dimensional point cloud at each moment.

6. The three-dimensional data encoding method of a multi-mode physical simulation system based on a neural network as claimed in claim 1, characterized in that: The correcting of the information distribution characteristic value specifically includes: taking the product of the information change characteristic value of each three-dimensional point cloud at each moment and the information distribution characteristic value as the corrected information distribution characteristic value of each three-dimensional point cloud at each moment.

Images