Time sequence multi-dimensional point cloud deformation recognition method and system, electronic device and storage medium

Abstract

The application discloses a kind of time sequence multidimensional point cloud deformation identification method, system, electronic equipment and storage medium, wherein time sequence multidimensional point cloud deformation identification method includes: based on the surface of measured object of the multiple frames of point cloud of deformation occurrence, construct time sequence multidimensional point cloud;Grid time sequence multidimensional point cloud;After grid time sequence multidimensional point cloud, point deformation tracking is carried out gradually;Based on deformation tracking result, establish the point pair matching relationship of time sequence multidimensional point cloud;Based on point pair matching relationship, construct local deformation descriptor and global deformation descriptor, to identify the surface deformation process of measured object.This application is applied to small scene fine full-field deformation identification, by constructing local and global deformation descriptor in time sequence multidimensional point cloud, provide a comprehensive and complete identification method for the deformation of object surface, accurately depict the whole process of the surface deformation of measured object from local and global angle, better reflect deformation law, reveal deformation mechanism and characteristics.

Description

Technical Field

[0001] This field relates to the field of point cloud deformation recognition technology, and in particular to a temporal multidimensional point cloud deformation recognition method, system, electronic device and storage medium. Background Technology

[0002] With the rapid development of sensor technology and observation platforms, point clouds have become a common way to characterize the three-dimensional information of object surfaces. At present, the deformation recognition technology of object surfaces has gradually evolved from recognizing the deformation of object surface markers to full-field deformation recognition based on object surface point clouds.

[0003] Full-field deformation recognition methods based on object surface point clouds can generally be divided into two categories: point cloud model-based and surface model-based. The former mainly includes C2C (Cloud-to-Cloud) and M3C2 (Multiscale Model-to-Model Cloud) methods, while the latter mainly includes C2M (Cloud-to-Mesh) and M2M (Mesh-to-Mesh) methods. The difference is that the former identifies deformation by directly comparing point clouds, while the latter is based on surface modeling of the point clouds. However, currently, both point cloud model-based and surface model-based methods identify deformation between point clouds based on the geometric information of the point clouds, and are generally used for deformation recognition in large scenes. Furthermore, because both methods identify object surface deformation based on the principle of nearest neighbor or adjacent points, the deformation identified by these two types of methods is usually smaller than the actual deformation of the object surface, resulting in low recognition accuracy and unsuitability for fine-grained deformation recognition in small scenes. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology in the low accuracy of fine full-field deformation recognition in small scenes, and to provide a temporal multi-dimensional point cloud deformation recognition method, system, electronic device and storage medium.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution:

[0006] This invention provides a method for identifying temporal multidimensional point cloud deformations, the method comprising:

[0007] A temporal multidimensional point cloud is constructed based on the multi-frame point cloud of the deformed surface of the measured object.

[0008] The temporal multidimensional point cloud described by gridding;

[0009] Point-by-point deformation tracking is performed on the gridded temporal multidimensional point cloud.

[0010] The point pair matching relationship of the temporal multidimensional point cloud is established based on the deformation tracking results;

[0011] Wherein, the point pair matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud;

[0012] Based on the point-pair matching relationship, local deformation descriptors and global deformation descriptors are constructed to identify the deformation process of the surface of the measured object.

[0013] Preferably, the step of constructing the temporal multidimensional point cloud includes the following:

[0014] Multiple frames of point cloud data of the deformed surface of the object under test are collected at preset time intervals.

[0015] The steps for constructing a temporal multidimensional point cloud include:

[0016] Based on the collected point clouds from multiple frames, a temporal multidimensional point cloud is constructed according to the acquisition time sequence.

[0017] or,

[0018] The steps for constructing a temporal multidimensional point cloud include:

[0019] Real-time acquisition of multi-frame point clouds of the surface of the object under test that has undergone deformation;

[0020] The steps for constructing a temporal multidimensional point cloud include:

[0021] Based on the real-time acquisition of multi-frame point clouds, a temporal multidimensional point cloud is constructed in real time.

[0022] Preferably, the step of gridding the temporal multidimensional point cloud includes the following:

[0023] The temporal multidimensional point cloud is preprocessed, and the preprocessing includes at least one of background data removal, noise data removal, and normalization processing.

[0024] Preferably, the step of performing point-by-point deformation tracking on the gridded temporal multidimensional point cloud includes:

[0025] Extract key points and multidimensional features of the key points from the gridded temporal multidimensional point cloud, wherein the multidimensional features include rotation- and scaling-invariant features and / or texture features.

[0026] Based on the multidimensional features, initial point pair matching is performed on the key points;

[0027] The initial point pair matching results are optimized using a correlation model and an iterative optimization method to establish a grid point pair matching relationship; the iterative optimization method includes the Gauss-Newton iterative method; the correlation model is used to determine whether the point pair matching is accurate.

[0028] Preferably, the step of establishing the point-pair matching relationship of the temporal multidimensional point cloud based on the deformation tracking results includes:

[0029] Based on the grid point pair matching relationship, establish the point pair matching relationship of the temporal multidimensional point cloud.

[0030] Preferably, the step of constructing a local deformation descriptor based on the point pair matching relationship includes:

[0031] Based on the point pair matching relationship, discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector are obtained.

[0032] Based on the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector, a local deformation descriptor is constructed using the following formula:

[0033] LDD = [xΔev] T

[0034] Wherein, LDD represents the local deformation descriptor, x represents the discrete spatiotemporal coordinate information, Δ represents the discrete displacement vector, e represents the discrete strain vector, v represents the discrete deformation rate vector, and T represents the transpose sign.

[0035] Preferably, the step of constructing a global deformation descriptor based on the point pair matching relationship includes:

[0036] Interpolating the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector yields a continuous displacement field, a continuous strain field, and a continuous velocity field at a preset time point and a preset spatial location.

[0037] Wherein, the displacement field is a function of the displacement state of the surface of the object under test, the strain field is a function of the strain state of the surface of the object under test, and the velocity field is a function of the deformation rate of the surface of the object under test.

[0038] Based on the continuous displacement field, continuous strain field, and continuous velocity field, a global deformation descriptor is constructed using the following formula:

[0039] GDD=[D(x,y,z,t)E(x,y,z,t)V(x,y,z,t)] T

[0040] Wherein, GDD represents the global deformation descriptor, D(x,y,z,t) is the displacement field, E(x,y,z,t) is the strain field, V(x,y,z,t) is the velocity field, x, y, z represent the three-dimensional coordinate information of a point in a frame of multidimensional point cloud along the x-axis, y-axis, and z-axis, t represents the time information of a point in a frame of multidimensional point cloud, and T represents the transpose symbol.

[0041] The present invention also provides a temporal multidimensional point cloud deformation recognition system, the temporal multidimensional point cloud deformation recognition system comprising:

[0042] The acquisition module is used to construct a temporal multidimensional point cloud based on the multi-frame point cloud of the deformed surface of the measured object.

[0043] The grid module is used to grid the temporal multidimensional point cloud;

[0044] The deformation tracking module is used to perform point-by-point deformation tracking on the gridded temporal multidimensional point cloud.

[0045] The matching module is used to establish point-to-point matching relationships for the temporal multidimensional point cloud based on the deformation tracking results;

[0046] Wherein, the point pair matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud;

[0047] A construction module is used to construct local deformation descriptors and global deformation descriptors based on the point pair matching relationship, so as to identify the deformation process of the surface of the object under test.

[0048] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described temporal multidimensional point cloud deformation recognition method.

[0049] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described temporal multidimensional point cloud deformation recognition method.

[0050] The positive and progressive effects of this invention are as follows:

[0051] This invention is applied to the fine-grained full-field deformation recognition in small scenes. By constructing local deformation descriptors and global deformation descriptors in temporal multidimensional point clouds, it provides a comprehensive and complete method for recognizing the deformation of object surfaces. This method accurately describes the entire process of deformation of the measured object surface from both local and global perspectives, better reflecting the deformation law and revealing the deformation mechanism and characteristics. Attached Figure Description

[0052] Figure 1This is the first flowchart of the temporal multidimensional point cloud deformation recognition method of Embodiment 1 of the present invention;

[0053] Figure 2 This is the second flowchart of the temporal multidimensional point cloud deformation recognition method of Embodiment 1 of the present invention;

[0054] Figure 3 This is the third flowchart of the temporal multidimensional point cloud deformation recognition method in Embodiment 1 of the present invention;

[0055] Figure 4 This is the fourth flowchart of the temporal multidimensional point cloud deformation recognition method according to Embodiment 1 of the present invention;

[0056] Figure 5 This is a first structural diagram of the temporal multidimensional point cloud deformation recognition system according to Embodiment 2 of the present invention;

[0057] Figure 6 This is a second structural diagram of the temporal multidimensional point cloud deformation recognition system according to Embodiment 2 of the present invention;

[0058] Figure 7 This is a schematic diagram of the electronic device according to Embodiment 3 of the present invention. Detailed Implementation

[0059] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0060] Example 1

[0061] This embodiment provides a method for temporal multidimensional point cloud deformation recognition. See [link to relevant documentation]. Figure 1 Temporal multidimensional point cloud deformation recognition methods include:

[0062] S1. Construct a temporal multidimensional point cloud based on the multi-frame point cloud of the deformed surface of the measured object.

[0063] In one alternative implementation, see Figure 2 Before step S1, the following are included:

[0064] S11. Collect multiple frames of point cloud data of the deformed surface of the object under test according to a preset time interval.

[0065] The preset time interval can be set according to actual needs.

[0066] Step S1 includes:

[0067] S12. Construct a temporal multidimensional point cloud based on the collected multi-frame point clouds in the order of acquisition time.

[0068] For example, a structured light system consisting of a camera, a structured light projector, and a computer system can be used to continuously acquire multiple frames of point clouds on the surface of the object being measured that has undergone deformation at certain time intervals, forming streaming data. Each frame of point cloud has three-dimensional coordinate information, grayscale information, and time information, and is sorted according to the order of acquisition time to construct a temporal multidimensional point cloud.

[0069] A point P in a frame of a temporal multidimensional point cloud can be represented as:

[0070] P = [xyzgt] T

[0071] Wherein, P represents a point in a certain frame of multidimensional point cloud, x, y, z represent the three-dimensional coordinate information of P on the x-axis, y-axis and z-axis, g represents the grayscale information of P, and t represents the time information of P.

[0072] In this embodiment, a temporal multidimensional point cloud is constructed by collecting multi-frame point clouds of the deformed surface of the object under test at preset time intervals, thereby obtaining the deformation process of the surface of the object under test in real time and ensuring the timeliness of recognition.

[0073] In one alternative implementation, see Figure 3 Before step S1, the following are included:

[0074] S13. Real-time acquisition of multi-frame point clouds of the deformed surface of the measured object.

[0075] Step S1 includes:

[0076] S14. Based on real-time acquisition of multi-frame point clouds, construct temporal multi-dimensional point clouds in real time.

[0077] In this embodiment, a temporal multidimensional point cloud is constructed based on real-time acquired multi-frame point clouds, and the deformation information of the surface of the measured object is updated in real time to improve the accuracy of recognition.

[0078] S2, gridded temporal multidimensional point cloud.

[0079] In this embodiment, the temporal multidimensional point cloud is divided into grids to achieve ordered data organization. The grid row number m, column number n, and layer number c of P can be calculated using the following formula:

[0080]

[0081] Where, x min y min z minThese represent the minimum values ​​of the x, y, and d coordinates of all points in a frame of multidimensional point cloud, respectively. Grid represents the grid resolution, which is generally set according to the size of the measured area on the object's surface. Here, the specific value of the grid resolution is not limited.

[0082] Points in a multidimensional point cloud that share the same row number m, column number n, layer number c, and time information t are grouped into the same grid and described by grid point Q, which can be expressed as the following formula:

[0083]

[0084] Where Q represents a grid point, c represents the grid layer number, n represents the grid column number, and m represents the grid row number. The value of grayscale information of all points in the grid is represented by t, and the value of t represents the temporal information of the grid points.

[0085] In an optional implementation, step S2 is preceded by:

[0086] S21. Preprocessing temporal multidimensional point cloud.

[0087] The preprocessing includes at least one of background data removal, noise removal, and normalization.

[0088] Because the constructed temporal multidimensional point cloud may contain background data and noise data, and since background data does not belong to the region of interest for deformation recognition, while noise data can affect the accuracy of deformation recognition, it is necessary to remove both background and noise data. Specifically, point cloud filtering and point cloud cropping are used to remove background data. Point cloud filtering can use methods such as pass-through filtering, voxel filtering, and statistical filtering. Noise data can be removed using RANSAC (Random Sample Consensus) or other outlier removal methods. In addition, the temporal multidimensional point cloud can be homogenized to obtain a temporal multidimensional point cloud with uniform point spacing.

[0089] In this embodiment, by preprocessing the temporal multidimensional point cloud, data that is more conducive to accurate deformation identification is obtained, thereby providing a good data foundation for the subsequent deformation identification process.

[0090] S3. Perform point-by-point deformation tracking on the gridded temporal multidimensional point cloud.

[0091] In one alternative implementation, see Figure 4 Step S3 includes:

[0092] S31. Extract key points and multidimensional features of key points from the gridded temporal multidimensional point cloud.

[0093] Among them, multidimensional features include rotation- and scaling-invariant features, texture features, or a combination of rotation- and scaling-invariant features and texture features.

[0094] The key point can be represented by the following formula:

[0095]

[0096] Among them, KQ represents key points, F Rsi F is the rotation- and scaling-invariant eigenvector of keypoint KQ. Tex The texture feature vector of key point KQ.

[0097] Specifically, keypoints KQ and their F values ​​are extracted using algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), or ORB (Oriented Fast and Rotated BRIEF). Rsi The F-values ​​are extracted using methods such as GLCM (Grey-Level Co-occurrence Matrix), LBP (Linear Back Projection), and Gabor filtering (wavelet feature extraction algorithm). Tex .

[0098] S32. Based on multidimensional features, perform initial point-to-point matching on key points.

[0099] Multidimensional features include rotation- and scaling-invariant features and texture features. Specifically, standardized Euclidean distance is used as the distance metric, and the matching index Dist is the distance-weighted sum of the rotation- and scaling-invariant features and the texture features. The formula for calculating Dist is as follows:

[0100]

[0101] Where w1 and w2 represent the weight coefficients of rotation- and scaling-invariant features and texture features, respectively. These are the rotation-scaling invariant feature vectors and texture feature vectors of keypoints in the previous frame's multidimensional point cloud after gridding, and keypoints in the next frame's multidimensional point cloud, respectively. Rsi s Tex These are the standard deviations of the rotation- and scaling-invariant eigenvectors and the texture eigenvectors, respectively.

[0102] The smaller Dist is, the higher the degree of matching. Therefore, by finding the minimum value min1(Dist) of Dist, point-to-point matching of key points in the previous frame of multi-dimensional point cloud and key points in the next frame of multi-dimensional point cloud can be performed, that is, the initial matching of key point pairs can be established.

[0103] In addition, the ratio τ of the minimum value min1(Dist) to the second minimum value min2(Dist) of Dist must be calculated using the following formula. If τ is less than the threshold τ0, the initial match of the keypoint pair is considered a correct match; otherwise, it is considered an incorrect match and discarded. The threshold τ0 can be set according to actual needs.

[0104]

[0105] S33. Optimize the initial point pair matching results through correlation model and iterative optimization method to establish grid point pair matching relationship.

[0106] Iterative optimization methods include iterative methods such as the Gauss-Newton iteration method. Correlation models are used to determine the accuracy of point pair matching.

[0107] The following describes the steps of an iterative method for optimizing the matching results of initial point pairs:

[0108] Construct a reference subregion of size (2γ+1)×(2γ+1) (γ∈N) centered on each reference point (the reference point is a key point in the previous frame's multidimensional point cloud after gridding), and denote the coordinates of the center point as m. 0 Reference the coordinates m of other grid points in the sub-region i This can be expressed as the following formula:

[0109] m i =Δm i m 0 +m 0 =W(Δm) i m 0 ,P rr )+m 0

[0110] Where, Δm i m 0 To determine the offset of a reference sub-grid point, the shape function W(Δm) can be used. i m 0 ,P rr ) indicates that P rr It updates the shape function parameter vector of the reference subregion.

[0111] A deformation sub-region of size (2γ+1)×(2γ+1) (γ∈N) is constructed with the deformation point (the key point matched in the previous frame of the multi-dimensional point cloud after gridding, with the deformation point as the reference point) as the center point. The grid point coordinates m in the deformation sub-region are... i′ This can be expressed as the following formula:

[0112] m i′ =Δm i′ m 0 +m 0 =W(Δm) i m 0 ,P rd )+m 0

[0113] Where, Δm i′ m 0 The offset of the deformed sub-grid points can be expressed using the shape function W(Δm). i m 0 ,P rd ) indicates that P rd To update the shape function parameter vector of the deformed subregion, at least three known reference points and their matching points in the reference subregion are used, and then based on the formula... P can be calculated rd initial value

[0114] A correlation model between the reference subregion and the deformed subregion was constructed using the zero-mean normalized sum of squared differences (ZNSSD). The formula for calculating the correlation model is as follows:

[0115]

[0116] Where G(·) represents the grayscale information of the grid points, and I represents the number of grid points in the reference sub-region. These represent the average gray values ​​of the grid points in the reference sub-region and the deformed sub-region, respectively.

[0117] The above formula is iteratively optimized using the Gauss-Newton iteration method to obtain the result at P. rr Optimal solution when = 0 Alternatively, other iterative optimization methods can be selected to match grid points in the reference subregion and the deformed subregion, and establish grid point pair matching relationships.

[0118] S4. Establish point-to-point matching relationships for temporal multidimensional point clouds based on deformation tracking results.

[0119] Among them, the point-to-point matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud.

[0120] That is, the matching relationship between the first frame multidimensional point cloud and the second frame multidimensional point cloud, the matching relationship between the second frame multidimensional point cloud and the third frame multidimensional point cloud, the matching relationship between the third frame multidimensional point cloud and the fourth frame multidimensional point cloud, ... the matching relationship between the Nth frame multidimensional point cloud and the N+1th frame multidimensional point cloud.

[0121] In one alternative implementation, see Figure 4 Step S4 includes:

[0122] S41. Based on the grid point pair matching relationship, establish the point pair matching relationship of the temporal multidimensional point cloud.

[0123] In this embodiment, the deformation tracking result is to establish a grid point pair matching relationship. The average value of the three-dimensional coordinate information of all points contained in the grid where grid point Q is located is taken as the point in the temporal multidimensional point cloud corresponding to the grid point. Based on this, the point pair matching relationship in the temporal multidimensional point cloud can be established through the grid point pair matching relationship.

[0124] In this embodiment, by employing multi-dimensional feature fusion and iterative optimization, point-by-point deformation tracking is performed in the temporal multi-dimensional point cloud, establishing a high-precision full-field point pair matching relationship, improving the full-field deformation recognition accuracy based on object surface point cloud, and making it suitable for fine-grained deformation recognition in small scenes.

[0125] S5. Construct local deformation descriptors and global deformation descriptors based on point-pair matching relationships to identify the deformation process of the surface of the object being measured.

[0126] In an optional implementation, step S5 includes:

[0127] S51. Based on point-to-point matching relationships, obtain discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector.

[0128] S52. Based on discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector, a local deformation descriptor is constructed using the following formula:

[0129] LDD = [a Δ ev] T

[0130] In this context, LDD (Local Deformation Descriptor) represents the local deformation descriptor, a represents the discrete spatiotemporal coordinate information, Δ represents the discrete displacement vector, e represents the discrete strain vector, v represents the discrete deformation rate vector, and T represents the transpose sign.

[0131] Where 'a' represents spatiotemporal coordinate information, which can be represented as [xyzt]. T It includes three-dimensional coordinate information and time information.

[0132] Where Δ is the displacement vector, it can be represented as [δ x δ y δ z ] T The calculation formula is as follows:

[0133] δ i =i d -i r (i = x, y, z)

[0134] Where e is the strain vector, which can be represented as [ε x ε y ε z γ xy γ yz γ xz ] T It can be calculated based on geometric equations, and the calculation formula is as follows:

[0135]

[0136]

[0137] Where v is the deformation rate vector, it can be represented as [v x v y v z ] T The calculation formula is as follows:

[0138]

[0139] In an optional implementation, step S5 further includes:

[0140] S53. Interpolate the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector to obtain a continuous displacement field, a continuous strain field, and a continuous velocity field at a preset time point and a preset spatial position.

[0141] Among them, the displacement field is a function of the displacement state of the surface of the object being measured, the strain field is a function of the strain state of the surface of the object being measured, and the velocity field is a function of the deformation rate of the surface of the object being measured.

[0142] S54. Based on a continuous displacement field, a continuous strain field, and a continuous velocity field, a global deformation descriptor is constructed using the following formula:

[0143] GDD=[D(x,y,z,t)E(x,y,z,t)V(x,y,z,t)] T

[0144] In this context, GDD (Global Deformation Descriptor) represents the global deformation descriptor, D(x,y,z,t) is the displacement field, E(x,y,z,t) is the strain field, V(x,y,z,t) is the velocity field, x, y, and z represent the three-dimensional coordinate information of a point in a frame of multidimensional point cloud along the x, y, and z axes, t represents the temporal information of a point in a frame of multidimensional point cloud, and T represents the transpose sign.

[0145] This embodiment provides a comprehensive and complete method for identifying the deformation of an object's surface by constructing local and global deformation descriptors in a temporal multidimensional point cloud. This method accurately depicts the entire process of deformation of the measured object's surface from both local and global perspectives, better reflecting the deformation laws and revealing the deformation mechanism and characteristics.

[0146] Example 2

[0147] This embodiment provides a temporal multidimensional point cloud deformation recognition system. (See also...) Figure 5 The temporal multidimensional point cloud deformation recognition system includes:

[0148] Module 1 is used to construct a temporal multidimensional point cloud based on the multi-frame point cloud of the deformed surface of the measured object.

[0149] Grid module 2 is used for gridding temporal multidimensional point clouds.

[0150] Deformation tracking module 3 is used to perform point-by-point deformation tracking on the gridded temporal multidimensional point cloud.

[0151] Matching module 4 is used to establish point-to-point matching relationships for temporal multidimensional point clouds based on deformation tracking results.

[0152] Among them, the point-to-point matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud.

[0153] Module 5 is used to construct local deformation descriptors and global deformation descriptors based on point-pair matching relationships in order to identify the surface deformation process of the measured object.

[0154] In one alternative implementation, see Figure 6 The temporal multidimensional point cloud deformation recognition system also includes:

[0155] Acquisition module 6 is used to acquire multi-frame point clouds of the deformed surface of the object under test at preset time intervals.

[0156] The acquisition module 1 is also used to construct a temporal multidimensional point cloud based on the acquired multi-frame point cloud in the order of acquisition time.

[0157] In an optional implementation, the acquisition module 6 is also used to acquire multi-frame point clouds of the deformed surface of the measured object in real time.

[0158] Module 1 is also used to construct a time-series multidimensional point cloud in real time based on multi-frame point clouds acquired in real time.

[0159] In one alternative implementation, see Figure 6 The temporal multidimensional point cloud deformation recognition system also includes:

[0160] The preprocessing module 7 is used to preprocess the temporal multidimensional point cloud. The preprocessing includes at least one of background data removal, noise data removal, and normalization processing.

[0161] In an optional implementation, the deformation tracking module 3 is further configured to extract key points and multidimensional features of key points from the gridded temporal multidimensional point cloud, wherein the multidimensional features include rotation- and scaling-invariant features and / or texture features; and to perform initial point pair matching on key points based on the multidimensional features.

[0162] In an optional implementation, the matching module 4 is further configured to optimize the initial point pair matching results through a correlation model and an iterative optimization method to establish a grid point pair matching relationship; the iterative optimization method includes the Gauss-Newton iterative method; the correlation model is used to determine whether the point pair matching is accurate; and it is also configured to establish a point pair matching relationship of the temporal multidimensional point cloud based on the grid point pair matching relationship.

[0163] In an optional implementation, the construction module 5 is further configured to obtain discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector based on point-pair matching relationships; and to construct a local deformation descriptor based on the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector using the following formula:

[0164] LDD = [aΔev] T

[0165] Wherein, LDD represents the local deformation descriptor, a represents the discrete spatiotemporal coordinate information, Δ represents the discrete displacement vector, e represents the discrete strain vector, v represents the discrete deformation rate vector, and T represents the transpose sign.

[0166] In an optional implementation, the construction module 5 is further used to interpolate discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector and discrete deformation rate vector to obtain a continuous displacement field, a continuous strain field and a continuous velocity field at a preset time point and a preset spatial position.

[0167] Among them, the displacement field is a function of the displacement state of the surface of the object being measured, the strain field is a function of the strain state of the surface of the object being measured, and the velocity field is a function of the deformation rate of the surface of the object being measured.

[0168] Based on a continuous displacement field, a continuous strain field, and a continuous velocity field, a global deformation descriptor is constructed using the following formula:

[0169] GDD=[D(x,y,z,t)E(x,y,z,t)V(x,y,z,t)] T

[0170] Wherein, GDD represents the global deformation descriptor, D(x,y,z,t) is the displacement field, E(x,y,z,t) is the strain field, V(x,y,z,t) is the velocity field, x, y, z represent the three-dimensional coordinate information of a point in a frame of multidimensional point cloud along the x-axis, y-axis, and z-axis, t represents the temporal information of a point in a frame of multidimensional point cloud, and T represents the transpose symbol.

[0171] It should be noted that the implementation principles and technical effects of each module of the temporal multidimensional point cloud deformation recognition system in this embodiment can be referred to the corresponding parts of Embodiment 1, and will not be repeated here.

[0172] Example 3

[0173] This embodiment provides an electronic device. Figure 7 This is a schematic diagram of the electronic device. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the temporal multidimensional point cloud deformation recognition method of Embodiment 1. Figure 7 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0174] like Figure 7 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0175] Bus 33 includes a data bus, an address bus, and a control bus.

[0176] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0177] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0178] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the temporal multidimensional point cloud deformation recognition method of Embodiment 1 of the present invention.

[0179] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, the model-generating device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 36. Figure 7 As shown, network adapter 36 communicates with other modules of the model-generated device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0180] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0181] Example 4

[0182] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the temporal multidimensional point cloud deformation recognition method of Embodiment 1.

[0183] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0184] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, causes the terminal device to execute the temporal multidimensional point cloud deformation recognition method of Embodiment 1.

[0185] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0186] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A method for recognizing temporal multidimensional point cloud deformations, characterized in that, The temporal multidimensional point cloud deformation recognition method includes: A temporal multidimensional point cloud is constructed based on the multi-frame point cloud of the deformed surface of the measured object. The temporal multidimensional point cloud described by gridding; Point-by-point deformation tracking is performed on the gridded temporal multidimensional point cloud. The point pair matching relationship of the temporal multidimensional point cloud is established based on the deformation tracking results; Wherein, the point pair matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud; Based on the point-pair matching relationship, local deformation descriptors and global deformation descriptors are constructed to identify the deformation process of the surface of the object under test; The step of constructing a local deformation descriptor based on the point pair matching relationship includes: Based on the point pair matching relationship, discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector are obtained. Based on the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector, a local deformation descriptor is constructed using the following formula: ； Wherein, LDD represents the local deformation descriptor, a represents the discrete spatiotemporal coordinate information, Δ represents the discrete displacement vector, e represents the discrete strain vector, v represents the discrete deformation rate vector, and T represents the transpose sign. The step of constructing a global deformation descriptor based on the point pair matching relationship includes: Interpolating the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector yields a continuous displacement field, a continuous strain field, and a continuous velocity field at a preset time point and a preset spatial location. Wherein, the displacement field is a function of the displacement state of the surface of the object under test, the strain field is a function of the strain state of the surface of the object under test, and the velocity field is a function of the deformation rate of the surface of the object under test. Based on the continuous displacement field, continuous strain field, and continuous velocity field, a global deformation descriptor is constructed using the following formula: ； Among them, the Characterizing global deformation descriptors, For displacement field, For strain field, For the velocity field, x, y, and z represent the three-dimensional coordinate information of a point in a frame of multidimensional point cloud along the x, y, and z axes, t represents the temporal information of a point in a frame of multidimensional point cloud, and T represents the transpose sign.

2. The temporal multidimensional point cloud deformation recognition method as described in claim 1, characterized in that, The steps for constructing a temporal multidimensional point cloud include: Multiple frames of point cloud data of the deformed surface of the object under test are collected at preset time intervals. The steps for constructing a temporal multidimensional point cloud include: Based on the collected point clouds from multiple frames, a temporal multidimensional point cloud is constructed according to the acquisition time sequence. or, The steps for constructing a temporal multidimensional point cloud include: Real-time acquisition of multi-frame point clouds of the surface of the object under test that has undergone deformation; The steps for constructing a temporal multidimensional point cloud include: Based on the real-time acquisition of multi-frame point clouds, a temporal multidimensional point cloud is constructed in real time.

3. The temporal multidimensional point cloud deformation recognition method as described in claim 1, characterized in that, The step of gridding the temporal multidimensional point cloud includes: The temporal multidimensional point cloud is preprocessed, and the preprocessing includes at least one of background data removal, noise data removal, and normalization processing.

4. The temporal multidimensional point cloud deformation recognition method as described in claim 1, characterized in that, The step of performing point-by-point deformation tracking on the gridded temporal multidimensional point cloud includes: Extract key points and multidimensional features of the key points from the gridded temporal multidimensional point cloud, wherein the multidimensional features include rotation- and scaling-invariant features and / or texture features. Based on the multidimensional features, initial point pair matching is performed on the key points; The initial point pair matching results are optimized using a correlation model and an iterative optimization method to establish a grid point pair matching relationship; the iterative optimization method includes the Gauss-Newton iterative method; the correlation model is used to determine whether the point pair matching is accurate.

5. The temporal multidimensional point cloud deformation recognition method as described in claim 4, characterized in that, The step of establishing the point-pair matching relationship of the temporal multidimensional point cloud based on the deformation tracking results includes: Based on the grid point pair matching relationship, establish the point pair matching relationship of the temporal multidimensional point cloud.

6. A temporal multidimensional point cloud deformation recognition system, characterized in that, The temporal multidimensional point cloud deformation recognition system includes: The acquisition module is used to construct a temporal multidimensional point cloud based on the multi-frame point cloud of the deformed surface of the measured object. The grid module is used to grid the temporal multidimensional point cloud; The deformation tracking module is used to perform point-by-point deformation tracking on the gridded temporal multidimensional point cloud. The matching module is used to establish point-to-point matching relationships for the temporal multidimensional point cloud based on the deformation tracking results; Wherein, the point pair matching relationship is the matching relationship between the previous frame of the multidimensional point cloud and the next frame of the temporal multidimensional point cloud; A construction module is used to construct local deformation descriptors and global deformation descriptors based on the point pair matching relationship, so as to identify the deformation process of the surface of the object under test; The construction module is further configured to obtain discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector based on the point pair matching relationship; and to construct a local deformation descriptor based on the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector, and discrete deformation rate vector using the following formula: ； Wherein, LDD represents the local deformation descriptor, a represents the discrete spatiotemporal coordinate information, Δ represents the discrete displacement vector, e represents the discrete strain vector, v represents the discrete deformation rate vector, and T represents the transpose sign. The construction module is also used to interpolate the discrete spatiotemporal coordinate information, discrete displacement vector, discrete strain vector and discrete deformation rate vector to obtain a continuous displacement field, a continuous strain field and a continuous velocity field at a preset time point and a preset spatial position. Wherein, the displacement field is a function of the displacement state of the surface of the object under test, the strain field is a function of the strain state of the surface of the object under test, and the velocity field is a function of the deformation rate of the surface of the object under test. Based on the continuous displacement field, continuous strain field, and continuous velocity field, a global deformation descriptor is constructed using the following formula: ； Among them, the Characterizing global deformation descriptors, For displacement field, For strain field, For the velocity field, x, y, and z represent the three-dimensional coordinate information of a point in a frame of multidimensional point cloud along the x, y, and z axes, t represents the temporal information of a point in a frame of multidimensional point cloud, and T represents the transpose sign.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the temporal multidimensional point cloud deformation recognition method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the temporal multidimensional point cloud deformation recognition method according to any one of claims 1-5.

Images