A method, device, electronic device and storage medium for estimating point cloud motion

Abstract

The present disclosure provides a method, apparatus, electronic device, and storage medium for estimating point cloud motion. The method includes: obtaining a point cloud frame to be estimated and a reference point cloud frame; sampling motion center points in the reference point cloud frame to construct a point cloud motion deformation map; determining a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; constructing a point cloud motion energy equation according to the first matching relationship and the point cloud motion deformation map; solving the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; performing interpolation on the sparse non-rigid motion field to determine a fine non-rigid motion field; compensating the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame; determining a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame; constructing an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame based on the second matching relationship; and iteratively estimating the inter-frame rate distortion function to determine the point cloud motion. The estimation accuracy of the point cloud motion can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of point cloud compression, and in particular, to a method, apparatus, electronic device, and storage medium for estimating point cloud motion. Background Art

[0002] Currently, with the rapid development of three-dimensional scanning devices, it has become possible to quickly digitize three-dimensional information in the real world, and point clouds are gradually becoming an effective way to represent three-dimensional scenes and the three-dimensional surfaces of objects. A point cloud is obtained by sampling the surface of an object by a three-dimensional scanning device. The number of points in a frame of point cloud is numerous, and each point contains geometric information and attribute information such as color and texture, with a large amount of information. A dynamic point cloud is a collection of point cloud frames continuously acquired for a moving object or a moving scene. Therefore, the data volume of the point cloud sequence is even larger. Considering the large data volume of the point cloud and the limited bandwidth of network transmission, point cloud compression is an inevitable task. How to make full use of the inter-frame correlation to remove the redundancy of time-domain information is a key issue in dynamic point cloud compression, and point cloud motion estimation is an active and promising research field.

[0003] In existing point cloud motion estimation schemes, an octree is usually used to perform spatial decomposition on the point cloud to obtain macroblocks, and the texture variance within each macroblock is calculated. For macroblocks with a texture variance less than a threshold, a reference block with the same spatial position is selected in the reference frame, and a matching relationship between the reference block and the current block is constructed to estimate the point cloud motion. However, the estimation accuracy of this method is poor. Summary of the Invention

[0004] Embodiments of the present disclosure at least provide a method, apparatus, electronic device, and storage medium for estimating point cloud motion, which can improve the estimation accuracy of point cloud motion.

[0005] Embodiments of the present disclosure provide a method for estimating point cloud motion, the method includes:

[0006] Obtain a point cloud frame to be estimated, and determine a reference point cloud frame corresponding to the point cloud frame to be estimated;

[0007] Sample a plurality of motion center points in the reference point cloud frame, and based on the motion center points, construct a point cloud motion deformation map reflecting the motion correlation relationship between the motion center points;

[0008] Determine a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; according to the first matching relationship and the point cloud motion deformation map, construct a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated;

[0009] Iteratively solve the point cloud motion energy equation to determine the sparse non-rigid motion field corresponding to the motion center point; perform interpolation on the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame;

[0010] Compensate the reference point cloud frame based on the fine non-rigid motion field to obtain the reference motion compensation frame corresponding to the reference point cloud frame;

[0011] Determine the second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; based on the second matching relationship, construct an inter-frame rate distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

[0012] In an optional implementation manner, the sampling multiple motion center points in the reference point cloud frame and constructing a point cloud motion deformation map reflecting the motion association relationship between the motion center points specifically includes:

[0013] In the reference point cloud frame, select the target coordinate axis with the largest geometric distribution variance, where the target coordinate axis is formed by connecting the points in the reference point cloud frame;

[0014] Sample multiple motion center points on the target coordinate axis according to a preset sampling step size;

[0015] Divide the reference point cloud frame into multiple point cloud subsets with the motion center point as the center according to a preset sampling radius;

[0016] Traverse all the motion center points to determine whether there are intersection points between the point cloud subsets corresponding to every two motion center points; if so, connect the motion center points to form the point cloud motion deformation map.

[0017] In an optional implementation manner, the determining the first matching relationship between the reference point cloud frame and the to-be-estimated point cloud frame; and constructing a point cloud motion energy equation between the reference point cloud frame and the to-be-estimated point cloud frame according to the first matching relationship and the point cloud motion deformation map specifically includes:

[0018] For each point in the reference point cloud frame, perform motion compensation on the point according to the corresponding motion center point to determine the motion compensation point corresponding to the point;

[0019] Determine the point-to-point matching relationship between the motion compensation point and the corresponding point in the to-be-estimated point cloud frame, and form the first matching relationship by all the point-to-point matching relationships;

[0020] Construct a geometric distortion term reflecting the geometric distortion between the reference point cloud frame and the point cloud frame to be estimated according to the first matching relationship;

[0021] Determine the motion association relationship between the motion center points according to the point cloud motion deformation map, and construct a motion difference term reflecting the motion difference between the motion center points according to the motion association relationship;

[0022] For each of the motion center points, configure a corresponding motion rigidity constraint term for the motion center point;

[0023] Construct the motion energy equation based on the geometric distortion term, the motion difference term, and the motion rigidity constraint term.

[0024] In an optional implementation manner, the point-to-point matching relationship includes a forward matching relationship and a backward matching relationship, where the forward matching relationship represents the point-to-point matching relationship from the reference point cloud frame to the point cloud frame to be estimated; the backward matching relationship represents the point-to-point matching relationship from the point cloud frame to be estimated to the reference point cloud frame.

[0025] In an optional implementation manner, iteratively solve the point cloud motion energy equation to determine the sparse non-rigid motion field corresponding to the motion center points; compensate for the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame, specifically including:

[0026] Iteratively solve the point cloud motion energy equation to determine the target non-rigid motion field that minimizes the point cloud motion energy equation, and use the target non-rigid motion field as the sparse non-rigid motion field corresponding to the motion center points;

[0027] Process the sparse non-rigid motion field by interpolation to estimate the motion vectors corresponding to the points other than the motion center points in the reference point cloud frame;

[0028] Combine the motion vectors with the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame.

[0029] In an optional implementation manner, determine the second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame; based on the second matching relationship, construct an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame; iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame, specifically including:

[0030] Divide the point cloud frame to be estimated and the reference motion compensation frame into multiple motion prediction blocks respectively by the same division method;

[0031] Determine the inter-block matching relationship between each corresponding motion prediction block in the to-be-estimated point cloud frame and the reference motion compensation frame, where the second matching relationship is constituted by all the inter-block matching relationships;

[0032] Based on the inter-block matching relationship, construct an inter-block rate-distortion function between each corresponding motion prediction block, where the inter-frame rate-distortion function is constituted by all the inter-block rate-distortion functions;

[0033] Calculate the motion estimation vector that minimizes the rate-distortion cost corresponding to the inter-block rate-distortion function, and use the motion estimation vector as the inter-block point cloud motion of the motion prediction block between the to-be-estimated point cloud frame and the reference point cloud frame;

[0034] Combine all the inter-block point cloud motions to constitute the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

[0035] An embodiment of the present disclosure further provides an estimation device for point cloud motion, and the device includes:

[0036] An acquisition module, configured to acquire a to-be-estimated point cloud frame and determine a reference point cloud frame corresponding to the to-be-estimated point cloud frame;

[0037] A motion deformation map construction module, configured to sample a plurality of motion center points in the reference point cloud frame and construct a point cloud motion deformation map reflecting the motion association relationship between the motion center points based on the motion center points;

[0038] An energy equation construction module, configured to determine a first matching relationship between the reference point cloud frame and the to-be-estimated point cloud frame; construct a point cloud motion energy equation between the reference point cloud frame and the to-be-estimated point cloud frame according to the first matching relationship and the point cloud motion deformation map;

[0039] A non-rigid motion field determination module, configured to iteratively solve the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; compensate for the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame;

[0040] A reference frame motion compensation module, configured to compensate the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame;

[0041] A local motion estimation module is used to determine a second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; based on the second matching relationship, construct an inter-frame rate distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

[0042] In an optional implementation manner, the motion deformation map construction module is specifically configured to:

[0043] In the reference point cloud frame, select a target coordinate axis with the largest geometric distribution variance, where the target coordinate axis is formed by connecting points in the reference point cloud frame;

[0044] According to a preset sampling step, sample a plurality of the motion center points on the target coordinate axis;

[0045] According to a preset sampling radius, divide the reference point cloud frame into a plurality of point cloud subsets with the motion center points as the centers;

[0046] Traverse all the motion center points to determine whether there are intersection points between the point cloud subsets corresponding to every two of the motion center points; if so, connect the motion center points to form the point cloud motion deformation map.

[0047] In an optional implementation manner, the energy equation construction module is specifically configured to:

[0048] For each point in the reference point cloud frame, perform motion compensation on the point according to the corresponding motion center point to determine the motion compensation point corresponding to the point;

[0049] Determine the point-to-point matching relationship between the motion compensation point and the corresponding point in the to-be-estimated point cloud frame, and form the first matching relationship from all the point-to-point matching relationships;

[0050] According to the first matching relationship, construct a geometric distortion term reflecting the geometric distortion between the reference point cloud frame and the to-be-estimated point cloud frame;

[0051] According to the point cloud motion deformation map, determine the motion association relationship between the motion center points, and according to the motion association relationship, construct a motion difference term reflecting the motion difference between the motion center points;

[0052] For each of the motion center points, configure a corresponding motion rigidity constraint term for the motion center point;

[0053] Based on the geometric distortion term, the motion difference term, and the motion rigidity constraint term, construct the motion energy equation.

[0054] In an alternative embodiment, the non-rigid motion field determination module is specifically configured to:

[0055] Iteratively solve the point cloud motion energy equation to determine a target non-rigid motion field that minimizes the point cloud motion energy equation, and use the target non-rigid motion field as the sparse non-rigid motion field corresponding to the motion center point;

[0056] Process the sparse non-rigid motion field by interpolation to estimate the motion vectors corresponding to other points in the reference point cloud frame except the motion center point;

[0057] Combine the motion vectors with the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame.

[0058] In an alternative embodiment, the motion estimation module is specifically configured to:

[0059] Divide the point cloud frame to be estimated and the reference motion compensation frame into multiple motion prediction blocks respectively by the same partitioning method;

[0060] Determine the inter-block matching relationship between each corresponding motion prediction block in the point cloud frame to be estimated and the reference motion compensation frame, where all the inter-block matching relationships constitute the second matching relationship;

[0061] Based on the inter-block matching relationship, construct an inter-block rate distortion function between each corresponding motion prediction block, where all the inter-block rate distortion functions constitute the inter-frame rate distortion function;

[0062] Calculate a motion estimation vector that minimizes the rate distortion cost corresponding to the inter-block rate distortion function, and use the motion estimation vector as the inter-block point cloud motion of the motion prediction block between the point cloud frame to be estimated and the reference point cloud frame;

[0063] Combine all the inter-block point cloud motions to constitute the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame.

[0064] The embodiments of the present disclosure further provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the above method for estimating point cloud motion, or the steps in any possible implementation manner of the above method for estimating point cloud motion are executed.

[0065] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the above-mentioned method for estimating point cloud motion, or the steps in any possible implementation manner of the above-mentioned method for estimating point cloud motion.

[0066] A method, device, electronic device, and storage medium for estimating point cloud motion provided by an embodiment of the present disclosure. By obtaining a point cloud frame to be estimated, a reference point cloud frame corresponding to the point cloud frame to be estimated is determined; a plurality of motion center points are sampled in the reference point cloud frame, and based on the motion center points, a point cloud motion deformation map reflecting the motion association relationship between the motion center points is constructed; a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated is determined; according to the first matching relationship and the point cloud motion deformation map, a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated is constructed; the point cloud motion energy equation is iteratively solved to determine a sparse non-rigid motion field corresponding to the motion center points; interpolation is performed on the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame; the reference point cloud frame is compensated based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame is determined; based on the second matching relationship, an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame is constructed; the inter-frame rate distortion function is iteratively estimated to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame. The estimation accuracy of point cloud motion can be improved.

[0067] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specific preferred embodiments are given in conjunction with the accompanying drawings and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for the embodiments. The drawings herein are incorporated into the specification and form a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0069] Figure 1 Shows a flowchart of a method for estimating point cloud motion provided by an embodiment of the present disclosure;

[0070] Figure 2 Shows a flowchart of constructing a point cloud motion energy equation provided by an embodiment of the present disclosure;

[0071] Figure 3Shows a schematic diagram of an apparatus for estimating point cloud motion provided by an embodiment of the present disclosure;

[0072] Figure 4 Shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0073] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only some of the embodiments of the present disclosure, rather than all of the embodiments. Components of the embodiments of the present disclosure described and illustrated in the accompanying drawings herein may be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.

[0074] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0075] The term "and / or" in this article merely describes an associated relationship and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, both A and B exist simultaneously, and B exists alone. In addition, the term "at least one" in this article means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set composed of A, B, and C.

[0076] It has been found through research that in existing point cloud motion estimation schemes, an octree is usually used to perform spatial decomposition on the point cloud to obtain macroblocks, calculate the texture variance within each macroblock, and for macroblocks with a texture variance less than a threshold, a reference block with the same spatial position is selected in the reference frame, and a matching relationship between the reference block and the current block is constructed to estimate the point cloud motion. However, the estimation accuracy of this method is relatively poor.

[0077] Based on the above research, the present disclosure provides a method, an apparatus, an electronic device, and a storage medium for estimating point cloud motion. By obtaining a point cloud frame to be estimated, a reference point cloud frame corresponding to the point cloud frame to be estimated is determined; a plurality of motion center points are sampled in the reference point cloud frame, and based on the motion center points, a point cloud motion deformation map reflecting the motion association relationship between the motion center points is constructed; a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated is determined; according to the first matching relationship and the point cloud motion deformation map, a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated is constructed; the point cloud motion energy equation is iteratively solved to determine a sparse non-rigid motion field corresponding to the motion center points; interpolation is performed on the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame; the reference point cloud frame is compensated based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame is determined; based on the second matching relationship, an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame is constructed; the inter-frame rate distortion function is iteratively estimated to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame. The estimation accuracy of the point cloud motion can be improved.

[0078] To facilitate the understanding of this embodiment, first, a method for estimating point cloud motion disclosed in the embodiments of the present disclosure will be introduced in detail. The execution subject of the method for estimating point cloud motion provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities. Such a computer device includes, for example: a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for estimating point cloud motion may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0079] See Figure 1 As shown, it is a flowchart of a method for estimating point cloud motion provided in the embodiments of the present disclosure. The method includes steps S101 to S106, where:

[0080] S101. Obtain a point cloud frame to be estimated, and determine a reference point cloud frame corresponding to the point cloud frame to be estimated.

[0081] In a specific implementation, the method for estimating point cloud motion is applied to a time-sequence dynamic point cloud frame sequence. The time-sequence dynamic point cloud frame sequence is composed of multiple point cloud frames, and each point cloud frame includes a point cloud composed of multiple points. Motion estimation is performed on the point cloud frame to be estimated starting from the second frame in the time-sequence dynamic point cloud frame sequence.

[0082] Preferably, the previous frame of the point cloud frame to be estimated in the time-series dynamic point cloud frame sequence is used as the reference point cloud frame.

[0083] S102. Sample a plurality of motion center points in the reference point cloud frame, and based on the motion center points, construct a point cloud motion deformation map reflecting the motion correlation relationship between the motion center points.

[0084] In a specific implementation, a plurality of motion center points reflecting the non-rigid motion centers in the reference point cloud frame are sampled in the reference point cloud frame, and then based on all the motion center points, the motion center points are connected as nodes to construct a point cloud motion deformation map reflecting the motion correlation relationship between the motion center points.

[0085] As a possible implementation manner, the method for constructing the point cloud motion deformation map may specifically include:

[0086] In the reference point cloud frame, select the target coordinate axis with the largest geometric distribution variance, where the target coordinate axis is formed by connecting the points in the reference point cloud frame; sample a plurality of the motion center points on the target coordinate axis according to a preset sampling step; divide the reference point cloud frame into a plurality of point cloud subsets with the motion center points as the centers according to a preset sampling radius; traverse all the motion center points to determine whether there are intersection points between the point cloud subsets corresponding to every two of the motion center points; if so, connect the motion center points to form the point cloud motion deformation map.

[0087] Specifically, the target coordinate axis with the largest geometric distribution variance can be determined based on the following method:

[0088] First, traverse the three-dimensional coordinates of all the points in the reference point cloud frame to determine the minimum coordinate x min and the maximum coordinate x max of the x-axis, the minimum coordinate y min and the maximum coordinate y max of the y-axis, and the minimum coordinate z min and the maximum coordinate z max . Secondly, according to the minimum coordinate x min and the maximum coordinate x max of the x-axis, the minimum coordinate y min and the maximum coordinate y max of the y-axis, and the minimum coordinate z min and the maximum coordinate z max , construct a point cloud bounding box in the reference point cloud frame, and determine the longest side in the point cloud bounding box as the target coordinate axis with the largest geometric distribution variance in the reference point cloud frame.

[0089] Optionally, the formula for constructing the point cloud bounding box can be expressed as:

[0090] B = (x max - x min ) × (y max - y min ) × (z max - z min )

[0091] Wherein, B represents the point cloud bounding box.

[0092] Furthermore, in the process of dividing the reference point cloud frame into multiple point cloud subsets, the target coordinate axis with the largest geometric distribution variance in the reference point cloud frame can be used as the rearrangement axis first, and all points on the axis are renumbered along this rearrangement axis. For example: (n1,..., n i ); After that, the first point n1 after renumbering is used as the first motion center point c1. Starting from the motion center point c1, along the rearrangement axis, a preset sampling step size is used to sample multiple motion center points (c1,..., c k ) equidistantly on the axis. Then, all the remaining points in the reference point cloud frame except the motion center points are traversed, and the Euclidean distance between each remaining point and each motion center point is calculated. If the Euclidean distance from this point to a certain motion center point is less than the preset sampling radius, then this point belongs to the point cloud subset to which this sampling center belongs; if the distance from this point to all motion centers is greater than the preset sampling radius, then this point is used as a new sampling center; Finally, after the traversal is completed, a series of motion center points and their corresponding point cloud subsets can be generated. Among them, there may be intersections of points between the point cloud subsets, and these intersection points fall within the radiation ranges of multiple point cloud subsets.

[0093] It should be noted that the preset sampling step size and the preset sampling radius can be selected according to actual needs, and no specific limitations are made here.

[0094] Furthermore, in the process of constructing the point cloud motion deformation graph, a graph G can be constructed for the sampled motion center points. All the motion center points C = {c1, c2,..., c k} are the nodes on the graph G. When there is an intersecting point set between the point cloud subset P i corresponding to the motion center point c i , and the point cloud subset P j of the motion center c j , then an edge ε ij is used to connect the two motion centers. After all the connections are completed, the point cloud motion deformation graph can be formed.

[0095] S103. Determine the first matching relationship between the reference point cloud frame and the to-be-estimated point cloud frame; According to the first matching relationship and the point cloud motion deformation graph, construct the point cloud motion energy equation between the reference point cloud frame and the to-be-estimated point cloud frame.

[0096] In a specific implementation, in order to find the optimal non-rigid transformation of each motion center point so that the points in the reference point cloud frame are as close as possible to the points in the point cloud frame to be estimated that match this point, this application introduces a point cloud motion energy equation to help find the optimal non-rigid transformation of the optimal motion center point. Here, the first matching relationship reflects the matching relationship of the corresponding points between the reference point cloud frame and the point cloud frame to be estimated. Furthermore, from the matching relationships between all corresponding points, the matching relationship between the reference point cloud frame and the point cloud frame to be estimated can be reflected.

[0097] Among them, the non-rigid transformation of the motion center point can be represented by a rotation matrix and a translation vector.

[0098] As a possible implementation manner, referring to Figure 2 shown in the figure, it is a flowchart of a method for constructing a point cloud motion energy equation provided by an embodiment of the present disclosure. The method includes steps S1031 to S1036, where:

[0099] S1031. For each point in the reference point cloud frame, perform motion compensation on this point according to the corresponding motion center point to determine the corresponding motion compensation point of this point.

[0100] Here, the points in the reference point cloud frame are affected by the corresponding motion center and can generate corresponding motion compensation points.

[0101] Specifically, determining the corresponding motion compensation point in the reference point cloud frame can be expressed based on the following formula:

[0102]

[0103] Among them, represents the motion compensation point; v i represents the point in the reference point cloud frame; u j represents the point in the point cloud frame to be estimated; c j represents the motion center point; R j represents the rotation matrix corresponding to the motion center point; t j represents the translation vector corresponding to the motion center point; T represents the non-rigid transformation of the motion center point; ω ij represents the interpolation preset weight coefficient of the motion center point c j for the point v in the reference point cloud frame i ; I(v i ) = {c j | dist(v i - c j ) ≤ r} is the set of motion centers to which the point v in the reference point cloud frame i belongs, and r represents the preset sampling radius.

[0104] It should be noted that the value of ω ij can be selected according to actual needs and is not specifically limited herein.

[0105] S1032. Determine the point - to - point matching relationship between the motion compensation points and the corresponding points in the point cloud frame to be estimated, and form the first matching relationship from all the point - to - point matching relationships.

[0106] Specifically, the point - to - point matching relationship includes a forward matching relationship and a backward matching relationship. Among them, the forward matching relationship represents the point - to - point matching relationship from the reference point cloud frame to the point cloud frame to be estimated; the backward matching relationship represents the point - to - point matching relationship from the point cloud frame to be estimated to the reference point cloud frame.

[0107] Here, the forward matching relationship can be the matching relationship between forward matching pairs, where the forward matching pair is a point in the reference point cloud frame and the corresponding matching point u map (i) in the point cloud frame to be estimated; the backward matching relationship can be the matching relationship between backward matching pairs, where the backward matching pair is a point u j and u j the corresponding matching point in the reference point cloud frame

[0108] S1033. According to the first matching relationship, construct a geometric distortion term reflecting the geometric distortion between the reference point cloud frame and the point cloud frame to be estimated.

[0109] Specifically, the geometric distortion term can be constructed based on the following formula:

[0110]

[0111] Among them, E distortion (T) represents the geometric distortion term; map(·) represents the index mapping function of the point - to - point matching relationship; α for and α back respectively represent the weight coefficients of the forward geometric distortion and the backward geometric distortion in E distortion (T), where constitutes the forward geometric distortion, and n represents the number of points in the reference point cloud frame, and m represents the number of points in the point cloud frame to be estimated;

[0112] S1034. According to the point cloud motion deformation map, determine the motion correlation relationship between the motion center points, and construct a motion difference term reflecting the motion difference between the motion center points according to the motion correlation relationship.

[0113] Here, the motion correlation relationship between the motion center points can be represented by the translation vector and rotation matrix corresponding to each motion center point, and the motion correlation relationship between the motion center points can be further extended to the motion correlation relationship between the sub-point clouds in the reference point cloud frame.

[0114] Specifically, the motion difference term can be constructed based on the following formula:

[0115]

[0116] where E motionD (T) represents the motion difference term; c j represents the motion center point; R j represents the rotation matrix corresponding to the motion center point; t j represents the translation vector corresponding to the motion center point; k represents the number of motion center points in the reference point cloud frame; N(c i ) = {c j |ε i，j = 1} represents the set of adjacent motion center points connected to the motion center point c i on the point cloud motion deformation diagram;

[0117] S1035. For each of the motion center points, configure a corresponding motion rigidity constraint term for the motion center point.

[0118] Here, a motion rigidity constraint term is introduced for the non-rigid motion of each motion center, where the motion rigidity constraint term can be determined according to the rotation matrix corresponding to the motion center point.

[0119] Specifically, the motion rigidity constraint term can be constructed based on the following formula:

[0120]

[0121] where E motionC (T) represents the motion rigidity constraint term; R j represents the rotation matrix corresponding to the motion center point; SVD(R i ) is the eigenmatrix obtained by performing singular value decomposition on the rotation matrix R i ; k represents the number of motion center points in the reference point cloud frame.

[0122] S1036. Based on the geometric distortion term, the motion difference term, and the motion rigidity constraint term, construct the motion energy equation.

[0123] In a specific implementation, corresponding weight coefficients are respectively configured for the motion difference term and the motion rigidity constraint term, and the motion difference term and the motion rigidity constraint term after the weight coefficients are configured are summed with the geometric distortion term to construct a motion energy equation.

[0124] Specifically, a motion energy equation can be constructed based on the following formula:

[0125] J = E distortion (T) + λ d E motionD (T) + λ c E motionC (T)

[0126] Among them, J represents the point cloud motion energy equation; E distortion (T) represents the geometric distortion term; E motionD (T) represents the motion difference term; E motionC (T) represents the motion rigidity constraint term; λ d represents the weight coefficient corresponding to the motion difference term; λ c represents the weight coefficient corresponding to the motion rigidity constraint term.

[0127] It should be noted that the weight coefficient λ d corresponding to the motion difference term and the weight coefficient λ c corresponding to the motion rigidity constraint term can be selected according to actual needs and are not specifically limited here.

[0128] S104. Iteratively solve the point cloud motion energy equation to determine the sparse non-rigid motion field corresponding to the motion center point; interpolate the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame.

[0129] In a specific implementation, after the point cloud motion energy equation for point cloud non-rigid motion estimation is constructed, the point cloud motion energy equation is iteratively solved with the goal of optimizing the point cloud motion energy equation, and a sparse non-rigid motion field that minimizes the point cloud motion energy equation is calculated. However, since the sparse non-rigid motion field is only composed of the non-rigid motions of all motion center points in the reference point cloud frame and cannot fully reflect the non-rigid motion states of all points in the reference point cloud frame, it is necessary to further perform interpolation upsampling based on this sparse non-rigid motion field to obtain a fine non-rigid motion field that reflects the non-rigid motion states of all points in the reference point cloud frame.

[0130] It should be noted that the point cloud motion reflected by the sparse non-rigid motion field and the fine non-rigid motion field is the motion based on the reference point cloud frame.

[0131] Specifically, a distance-related joint interpolation method can be adopted to estimate the motions of other points in the reference point cloud frame except the motion center point, and compensate the motions of other points in the reference point cloud frame except the motion center point into the sparse non-rigid motion field, so as to obtain a fine non-rigid motion field. Among them, the method for estimating the motions of other points in the reference point cloud frame except the motion center point can refer to the content in step S1031.

[0132] As a possible implementation manner, the method for determining the fine non-rigid motion field corresponding to the reference point cloud frame may specifically include the following steps S1041 - step S1043:

[0133] S1041. Iteratively solve the point cloud motion energy equation to determine the target non-rigid motion field that minimizes the point cloud motion energy equation, and use the target non-rigid motion field as the sparse non-rigid motion field corresponding to the motion center point.

[0134] Specifically, the method for iteratively solving the point cloud motion energy equation may include: before the first iteration, initialize the non-rigid motions of each motion center point to obtain the initial values of the geometric distortion term, the motion difference term, and the motion rigidity constraint term.

[0135] Furthermore, in each iteration, with the goal of optimizing the energy equation, under the point-to-point matching relationship between the motion compensation points and the corresponding points in the point cloud frame to be estimated, obtain the non-rigid motion field that minimizes the point cloud motion energy equation. Based on the max-min optimization framework, design surrogate functions for the motion compensation geometric distortion term and the non-rigid motion difference term, obtain the surrogate point cloud motion energy equation and its gradient equation and the initial value of the Hessian matrix, and design an iterative optimization scheme based on the L-BFGS algorithm to obtain the sparse non-rigid motion field that minimizes the surrogate energy equation.

[0136] Finally, after each iteration, update the geometric position of the reference point cloud frame based on the obtained sparse non-rigid motion field, and re-establish the point-to-point matching relationship between the updated reference point cloud frame and the point cloud frame to be estimated as the input for the next iteration; at the same time, the result of the previous iteration will be used as the input of the surrogate function for the current iteration optimization. The iteration terminates until the preset number of iterations is reached or the geometric distortion before and after the iteration is less than the set threshold. The set threshold and the preset number of iterations can be selected according to actual needs and are not specifically limited here.

[0137] S1042. Process the sparse non-rigid motion field by interpolation to estimate the motion vectors corresponding to other points in the reference point cloud frame except the motion center point.

[0138] S1043. Combine the motion vectors with the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame.

[0139] In a specific implementation, the motion vectors corresponding to the points other than the motion center point in the reference point cloud frame can be supplemented into the sparse non-rigid motion field to obtain a fine non-rigid motion field that comprehensively reflects the non-rigid motion states of all the points in the reference point cloud frame.

[0140] S105. Compensate the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame.

[0141] In a specific implementation, after obtaining the fine non-rigid motion field that reflects the non-rigid motion states of all the points in the reference point cloud frame, motion compensation is performed on the reference point cloud frame according to the fine non-rigid motion field. In the obtained reference motion compensation frame, it not only includes the point cloud itself but also the non-rigid motion of each point. The points included in the reference motion compensation frame are motion compensation points.

[0142] S106. Determine a second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; based on the second matching relationship, construct an inter-frame rate-distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimate the inter-frame rate-distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

[0143] In a specific implementation, the second matching relationship is similar to the first matching relationship, but it reflects the matching relationship between the points in the to-be-estimated point cloud frame and the motion compensation points corresponding to these points in the reference motion compensation frame. Furthermore, the matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame can be reflected by the matching relationships between all corresponding points.

[0144] Here, the inter-frame rate-distortion function is used to describe the joint cost between the geometric distortion term and the bitrate overhead term between the to-be-estimated point cloud frame and the reference motion compensation frame under the second matching relationship.

[0145] It should be noted that the point cloud motion estimated based on the inter-frame rate-distortion function here is the motion with the to-be-estimated point cloud frame as the reference, and its presentation form can be a non-rigid motion field.

[0146] As a possible implementation manner, the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame can be determined based on the following steps S1061 - S1065:

[0147] S1061. Divide the to-be-estimated point cloud frame and the reference motion compensation frame into multiple motion prediction blocks respectively through the same division method.

[0148] As a possible implementation, the octree partitioning method can be adopted. According to the preset prediction block size, the point cloud frame to be estimated and the reference motion compensation frame are respectively partitioned to a preset depth, and a plurality of motion prediction blocks are obtained in the point cloud frame to be estimated and the reference motion compensation frame respectively.

[0149] It should be noted that the preset prediction block size and the preset depth of the octree partitioning method can be selected according to actual needs, and no specific limitation is made here.

[0150] S1062. Determine the inter-block matching relationship between each corresponding motion prediction block in the point cloud frame to be estimated and the reference motion compensation frame, where all the inter-block matching relationships constitute the second matching relationship.

[0151] It should be noted that the motion prediction blocks processed in this step are non-empty blocks, that is, motion prediction blocks with points.

[0152] S1063. Based on the inter-block matching relationship, construct an inter-block rate-distortion function between each corresponding motion prediction block, where all the inter-block rate-distortion functions constitute the inter-frame rate-distortion function.

[0153] S1064. Calculate the motion estimation vector that minimizes the rate-distortion cost corresponding to the inter-block rate-distortion function, and use the motion estimation vector as the inter-block point cloud motion between the motion prediction block in the point cloud frame to be estimated and the reference point cloud frame.

[0154] Here, the inter-block point cloud motion can reflect the local point cloud motion between the point cloud frame to be estimated and the reference point cloud frame, and this motion is based on the point cloud frame to be estimated.

[0155] In a specific implementation, the method for calculating the motion estimation vector that minimizes the rate-distortion cost corresponding to the inter-block rate-distortion function may include:

[0156] Step 1. Set the search range and the starting search step size. Take the lower left corner geometric position of the current motion prediction block as the starting search point, traverse 19 search blocks including 1 block with the same position and 18 coplanar and collinear blocks within the search range of the reference point cloud frame, calculate the rate-distortion cost of these 19 search blocks respectively, and select the point with the minimum cost as the starting position for the next iteration. If the result of the search optimization is still the starting search point, go to Step 3, otherwise go to Step 2.

[0157] Step 2. Update the search starting point to the two points with the minimum cost in the search result of Step 1, keep the search step size unchanged, continue to traverse 19 search blocks within the search range of the reference point cloud frame respectively and calculate the corresponding rate-distortion cost, and select the point with the minimum cost as the starting position for the next iteration. If the result of the search optimization is still the starting search point, go to Step 3, otherwise go to Step 2.

[0158] Step 3: Update the search starting point to the two points with the smallest costs in the search results of Step 1 (including the original starting search point). Before reaching the motion accuracy, halve the search step size, and continue to traverse 19 search blocks respectively within the search range of the reference point cloud frame and calculate the corresponding rate-distortion cost. If the search step size at this time has not reached the motion accuracy, continue to loop and determine whether the result of the search optimization is still the starting search point. If so, enter Step 3; otherwise, enter Step 2. If the search step size reaches the motion accuracy at this time, the loop stops and the motion search corresponding to the motion prediction block ends.

[0159] It should be noted that the above-mentioned number of search blocks, search range, and search step size can be selected according to actual needs. The embodiments of the present application only provide exemplary references and do not make specific limitations here.

[0160] S1065: Combine all the inter-block point cloud motions to form the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

[0161] An estimation method of point cloud motion provided by an embodiment of the present disclosure includes: obtaining a to-be-estimated point cloud frame, and determining a reference point cloud frame corresponding to the to-be-estimated point cloud frame; sampling a plurality of motion center points in the reference point cloud frame, and based on the motion center points, constructing a point cloud motion deformation map reflecting the motion association relationship between the motion center points; determining a first matching relationship between the reference point cloud frame and the to-be-estimated point cloud frame; according to the first matching relationship and the point cloud motion deformation map, constructing a point cloud motion energy equation between the reference point cloud frame and the to-be-estimated point cloud frame; iteratively solving the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; compensating for the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame; compensating the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; determining a second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; based on the second matching relationship, constructing an inter-frame rate-distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimating the inter-frame rate-distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame. The estimation accuracy of the point cloud motion can be improved.

[0162] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order that constitutes any limitation to the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0163] Based on the same inventive concept, an apparatus for estimating point cloud motion corresponding to the method for estimating point cloud motion is further provided in the embodiments of the present disclosure. Since the principle of solving problems by the apparatus in the embodiments of the present disclosure is similar to that of the above-mentioned method for estimating point cloud motion in the embodiments of the present disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be described in detail.

[0164] Please refer to Figure 3 , Figure 3 which is a schematic diagram of an apparatus for estimating point cloud motion provided by an embodiment of the present disclosure. As Figure 3 shown therein, the apparatus 300 for estimating point cloud motion provided by an embodiment of the present disclosure includes:

[0165] An acquisition module 310, configured to acquire a point cloud frame to be estimated and determine a reference point cloud frame corresponding to the point cloud frame to be estimated.

[0166] A motion deformation map construction module 320, configured to sample a plurality of motion center points in the reference point cloud frame and construct a point cloud motion deformation map reflecting the motion association relationship between the motion center points based on the motion center points.

[0167] An energy equation construction module 330, configured to determine a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; and construct a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated according to the first matching relationship and the point cloud motion deformation map.

[0168] A non-rigid motion field determination module 340, configured to iteratively solve the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; and compensate for the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame.

[0169] A reference frame motion compensation module 350, configured to compensate the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame.

[0170] A local motion estimation module 360, configured to determine a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame; construct an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame based on the second matching relationship; and iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame.

[0171] For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0172] An apparatus for estimating point cloud motion provided by an embodiment of the present disclosure obtains a point cloud frame to be estimated and determines a reference point cloud frame corresponding to the point cloud frame to be estimated; samples a plurality of motion center points in the reference point cloud frame, and constructs a point cloud motion deformation map reflecting the motion association relationship between the motion center points based on the motion center points; determines a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; constructs a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated according to the first matching relationship and the point cloud motion deformation map; iteratively solves the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; performs interpolation on the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame; compensates the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; determines a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame; constructs an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame based on the second matching relationship; iteratively estimates the inter-frame rate distortion function to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame. The estimation accuracy of the point cloud motion can be improved.

[0173] Corresponding to Figure 1 the method for estimating point cloud motion in Figure 4 As shown in

[0174] a schematic structural diagram of an electronic device 400 provided by an embodiment of the present disclosure, which includes: a processor 41, a memory 42, and a bus 43; the memory 42 is used to store execution instructions, including an internal memory 421 and an external memory 422; the internal memory 421 here is also called the main memory, which is used to temporarily store the operation data in the processor 41 and the data exchanged with the external memory 422 such as a hard disk. The processor 41 exchanges data with the external memory 422 through the internal memory 421. When the electronic device 400 runs, the processor 41 communicates with the memory 42 through the bus 43, so that the processor 41 executes Figure 1 the steps of the method for estimating point cloud motion in

[0175] An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the method for estimating point cloud motion described in the above method embodiment. Among them, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0176] An embodiment of the present disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can execute the steps of the method for estimating point cloud motion described in the above method embodiment. For details, please refer to the above method embodiment and will not be elaborated here.

[0177] Among them, the above computer program product can be specifically implemented by means of hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is specifically embodied as a computer storage medium. In another alternative embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0178] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described device can refer to the corresponding process in the foregoing method embodiment, which will not be elaborated herein. In several embodiments provided by the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical, or other form.

[0179] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, in each embodiment of the present disclosure, the various functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0181] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0182] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for estimating the motion of a point cloud, characterized in that, The method includes: Obtaining a point cloud frame to be estimated, and determining a reference point cloud frame corresponding to the point cloud frame to be estimated; Sampling a plurality of motion center points in the reference point cloud frame, and based on the motion center points, constructing a point cloud motion deformation map reflecting the motion association relationship between the motion center points; Determining a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; according to the first matching relationship and the point cloud motion deformation map, constructing a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated; Iteratively solving the point cloud motion energy equation to determine a sparse non-rigid motion field corresponding to the motion center points; performing interpolation on the sparse non-rigid motion field to determine a fine non-rigid motion field corresponding to the reference point cloud frame; Compensating the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; Determining a second matching relationship between the point cloud frame to be estimated and the reference motion compensation frame; based on the second matching relationship, constructing an inter-frame rate distortion function between the point cloud frame to be estimated and the reference motion compensation frame; iteratively estimating the inter-frame rate distortion function to determine the point cloud motion between the point cloud frame to be estimated and the reference point cloud frame.

2. The method according to claim 1, characterized in that, The step of sampling a plurality of motion center points in the reference point cloud frame and constructing a point cloud motion deformation map reflecting the motion association relationship between the motion center points based on the motion center points specifically includes: In the reference point cloud frame, selecting a target coordinate axis with the largest geometric distribution variance, where the target coordinate axis is formed by connecting points in the reference point cloud frame; Sampling a plurality of the motion center points on the target coordinate axis according to a preset sampling step; Dividing the reference point cloud frame into a plurality of point cloud subsets with the motion center points as the centers according to a preset sampling radius; Traversing all the motion center points, and determining whether there are intersection points between the point cloud subsets corresponding to every two motion center points; if so, connecting the motion center points to form the point cloud motion deformation map.

3. The method according to claim 1, characterized in that, The step of determining a first matching relationship between the reference point cloud frame and the point cloud frame to be estimated; and constructing a point cloud motion energy equation between the reference point cloud frame and the point cloud frame to be estimated according to the first matching relationship and the point cloud motion deformation map specifically includes: For each point in the reference point cloud frame, performing motion compensation on the point according to the corresponding motion center point to determine a motion compensation point corresponding to the point; Determining an inter-point matching relationship between the motion compensation point and the corresponding point in the point cloud frame to be estimated, and forming the first matching relationship by all the inter-point matching relationships; According to the first matching relationship, constructing a geometric distortion term reflecting the geometric distortion between the reference point cloud frame and the point cloud frame to be estimated; According to the point cloud motion deformation map, determining the motion association relationship between the motion center points, and according to the motion association relationship, constructing a motion difference term reflecting the motion difference between the motion center points; For each motion center point, configuring a corresponding motion rigidity constraint term for the motion center point; Construct the motion energy equation based on the geometric distortion term, the motion difference term, and the motion rigidity constraint term.

4. The method according to claim 3, characterized in that, The point-to-point matching relationship includes a forward matching relationship and a backward matching relationship. Among them, the forward matching relationship represents the point-to-point matching relationship from the reference point cloud frame to the to-be-estimated point cloud frame; the backward matching relationship represents the point-to-point matching relationship from the to-be-estimated point cloud frame to the reference point cloud frame.

5. The method according to claim 1, characterized in that, Iteratively solve the point cloud motion energy equation to determine the sparse non-rigid motion field corresponding to the motion center point; Interpolate the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame, which specifically includes: Iteratively solve the point cloud motion energy equation to determine the target non-rigid motion field that minimizes the point cloud motion energy equation, and use the target non-rigid motion field as the sparse non-rigid motion field corresponding to the motion center point; Process the sparse non-rigid motion field by interpolation method to estimate the motion vectors corresponding to other points in the reference point cloud frame except the motion center point; Combine the motion vectors with the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame.

6. The method according to claim 1, characterized in that, Determine the second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; based on the second matching relationship, construct the inter-frame rate distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame, which specifically includes: Divide the to-be-estimated point cloud frame and the reference motion compensation frame into multiple motion prediction blocks respectively by the same division method; Determine the block-to-block matching relationship between the corresponding motion prediction blocks in the to-be-estimated point cloud frame and the reference motion compensation frame, where all the block-to-block matching relationships constitute the second matching relationship; Based on the block-to-block matching relationship, construct the block-to-block rate distortion function between the corresponding motion prediction blocks, where all the block-to-block rate distortion functions constitute the inter-frame rate distortion function; Calculate the motion estimation vector that minimizes the rate distortion cost corresponding to the block-to-block rate distortion function, and use the motion estimation vector as the block-to-block point cloud motion between the motion prediction block in the to-be-estimated point cloud frame and the reference point cloud frame; Combine all the block-to-block point cloud motions to constitute the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

7. An apparatus for estimating the motion of a point cloud, characterized in that, Include: An acquisition module, configured to acquire the to-be-estimated point cloud frame and determine the reference point cloud frame corresponding to the to-be-estimated point cloud frame; A motion deformation map construction module, configured to sample multiple motion center points in the reference point cloud frame and construct a point cloud motion deformation map reflecting the motion association relationship between the motion center points based on the motion center points; An energy equation construction module, configured to determine the first matching relationship between the reference point cloud frame and the to-be-estimated point cloud frame; Construct the point cloud motion energy equation between the reference point cloud frame and the to-be-estimated point cloud frame according to the first matching relationship and the point cloud motion deformation map; A non-rigid motion field determination module, which is used to iteratively solve the point cloud motion energy equation to determine the sparse non-rigid motion field corresponding to the motion center point; Interpolate the sparse non-rigid motion field to determine the fine non-rigid motion field corresponding to the reference point cloud frame; A reference frame motion compensation module, which is used to compensate the reference point cloud frame based on the fine non-rigid motion field to obtain a reference motion compensation frame corresponding to the reference point cloud frame; A local motion estimation module, which is used to determine the second matching relationship between the to-be-estimated point cloud frame and the reference motion compensation frame; Based on the second matching relationship, construct an inter-frame rate distortion function between the to-be-estimated point cloud frame and the reference motion compensation frame; iteratively estimate the inter-frame rate distortion function to determine the point cloud motion between the to-be-estimated point cloud frame and the reference point cloud frame.

8. The device according to claim 7, wherein, The energy equation construction module is specifically used for: For each point in the reference point cloud frame, perform motion compensation on the point according to the corresponding motion center point to determine the motion compensation point corresponding to the point; Determine the point-to-point matching relationship between the motion compensation point and the corresponding point in the to-be-estimated point cloud frame, and form the first matching relationship from all the point-to-point matching relationships; According to the first matching relationship, construct a geometric distortion term reflecting the geometric distortion between the reference point cloud frame and the to-be-estimated point cloud frame; According to the point cloud motion deformation map, determine the motion correlation relationship between the motion center points, and according to the motion correlation relationship, construct a motion difference term reflecting the motion difference between the motion center points; For each motion center point, configure a corresponding motion rigidity constraint term for the motion center point; Based on the geometric distortion term, the motion difference term, and the motion rigidity constraint term, construct the motion energy equation.

9. An electronic device, wherein, It includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the method for estimating point cloud motion according to any one of claims 1 to 6 are executed.

10. A computer-readable storage medium, wherein, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, the steps of the method for estimating point cloud motion according to any one of claims 1 to 6 are executed.

Images