Head movement data determination method and apparatus

Abstract

The application provides a head motion data determination method and device, the method comprising: obtaining reference frame point cloud data and current frame point cloud data of head motion; registering the current frame point cloud data based on the reference frame point cloud data to obtain current frame registration point cloud data; determining an expression estimation value according to the reference frame point cloud data and the current frame registration point cloud data; adjusting the current frame point cloud data according to the expression estimation value to determine current frame target point cloud data; and registering the reference frame point cloud data and the current frame target point cloud data to determine motion data. The application eliminates the influence of expression on target motion information and improves the reliability of motion data.

Description

Technical Field

[0001] This invention relates to the field of head motion detection technology, and specifically to a method and apparatus for determining head motion data. Background Technology

[0002] Accurate determination of head motion data is of great significance for systems or devices that require motion data, such as robots, magnetic resonance imaging (MRI) equipment, positron emission tomography (PET) and other medical imaging equipment, virtual equipment, and augmented reality devices.

[0003] The existing method for determining head motion data involves acquiring a 3D image of the face to determine the head motion data, specifically by using point cloud data from an initial reference frame and the current frame. However, this existing technology suffers from the following problem: due to the presence of facial expressions, facial expressions may be mistakenly identified as head motion data, leading to low accuracy in determining head motion data from 3D facial images.

[0004] Therefore, there is an urgent need to provide a method and apparatus for determining head motion data in order to improve the accuracy of the determined head motion data. Summary of the Invention

[0005] In view of this, it is necessary to provide a method and apparatus for determining head motion data to solve the technical problem of low accuracy of head motion data determined by facial expressions in the prior art.

[0006] On one hand, the present invention provides a method for determining head motion data, including:

[0007] Acquire point cloud data of the reference frame and the current frame for head motion;

[0008] Based on the reference frame point cloud data, the current frame point cloud data is registered to obtain the current frame registered point cloud data.

[0009] The expression estimation value is determined based on the reference frame point cloud data and the current frame registration point cloud data;

[0010] Based on the estimated facial expression value, the current frame point cloud data is adjusted to determine the target point cloud data for the current frame;

[0011] Based on the reference frame point cloud data, the target point cloud data of the current frame is registered to determine the motion data.

[0012] In some possible implementations, determining the expression estimate based on the reference frame point cloud data and the current frame registration point cloud data includes:

[0013] A two-dimensional projection grid plane is constructed based on the reference frame point cloud data, and the two-dimensional projection grid plane includes multiple grids;

[0014] Determine the reference projection coordinates of each reference point in the reference frame point cloud data in the two-dimensional projection grid plane, and determine the initial value of each grid in the plurality of grids based on the reference projection coordinates;

[0015] The projection distance of each current point in the current frame registration point cloud data is determined based on the initial value;

[0016] The expression points and non-expression points in the current point are determined based on the projection distance, and the expression estimate is determined based on the expression points.

[0017] In some possible implementations, determining the projection distance of each current point in the current frame registration point cloud data based on the initial value includes:

[0018] Determine the current projection coordinates of each current point in the two-dimensional projection grid plane, and determine the target grid where the current point is located based on the current projection coordinates;

[0019] When the initial value of the target raster is a preset initial value, a target reference point corresponding to the current point in the reference frame point cloud data is determined, and the projection distance of the current point is determined based on the target reference point, the current point, and the normal vector at the target reference point.

[0020] In some possible implementations, determining the estimated expression value based on the expression point includes:

[0021] Determine the maximum connected region of the expression point;

[0022] The number of facial expression points in the largest connected region is used as the estimated value of the facial expression.

[0023] In some possible implementations, adjusting the current frame point cloud data based on the expression estimation value to determine the current frame target point cloud data includes:

[0024] Determine whether the estimated expression value is greater than the estimated threshold;

[0025] If the estimated expression value is less than the estimated threshold, then the current frame point cloud data is the current frame target point cloud data;

[0026] If the estimated expression value is greater than or equal to the estimated threshold, then the expression points in the current frame point cloud data are removed to obtain the target point cloud data of the current frame.

[0027] In some possible implementations, the registration of the reference frame point cloud data and the current frame target point cloud data to determine motion data includes:

[0028] Based on the reference frame point cloud data, the target point cloud data of the current frame is registered to obtain the target registered point cloud data of the current frame.

[0029] Based on the reference frame point cloud data and the current frame target registration point cloud data, determine the target motion information and the motion information reliability value;

[0030] Motion data is determined based on the target motion information and the reliability value of the motion information.

[0031] In some possible implementations, determining the motion information reliability value based on the reference frame point cloud data and the current frame target registration point cloud data includes:

[0032] Multiple pairs of mapping points are obtained from the reference frame point cloud data and the current frame target registration point cloud data. Each pair of mapping points includes a first mapping point and a second mapping point. The first mapping point is a reference mapping point in the reference frame point cloud data, and the second mapping point is a target mapping point in the current frame target registration point cloud data that corresponds to the reference mapping point.

[0033] Determine the target projection distance between the reference mapping point and the target mapping point, and determine the number of target mapping points whose target projection distance is less than a preset distance;

[0034] The ratio of the number of target mapping points whose target projection distance is less than a preset distance to the total number of points in the target frame point cloud data of the current frame is determined, and the ratio is used as the reliability value of the motion information.

[0035] In some possible implementations, the target motion information includes translational and rotational motion; the step of determining motion data based on the target motion information and the reliability value of the motion information includes:

[0036] Determine whether the motion translation amount is greater than a translation amount threshold, whether the motion rotation amount is greater than a rotation threshold, and whether the motion reliability value is greater than a reliability threshold;

[0037] When the motion translation amount is greater than the translation amount threshold or the motion rotation amount is greater than the rotation amount threshold, and the motion reliability value is greater than the reliability threshold, the current frame point cloud data is motion data.

[0038] In some possible implementations, the head motion data determination method further includes:

[0039] When the current frame point cloud data is motion data, the reference frame point cloud data is updated based on the current frame point cloud data.

[0040] On the other hand, the present invention also provides a head motion data determination device, comprising:

[0041] The point cloud data acquisition unit is used to acquire the reference frame point cloud data and the current frame point cloud data of the head motion.

[0042] The current frame point cloud data registration unit is used to register the current frame point cloud data based on the reference frame point cloud data to obtain the current frame registered point cloud data.

[0043] The facial expression estimation unit is used to determine the facial expression estimation value based on the reference frame point cloud data and the current frame registration point cloud data.

[0044] The current frame target point cloud determination unit is used to adjust the current frame point cloud data according to the expression estimation value to determine the current frame target point cloud data;

[0045] The motion data determination unit is used to register the target point cloud data of the current frame with the point cloud data of the reference frame to determine the motion data.

[0046] The beneficial effects of the above embodiments are as follows: The head motion data determination method provided by the present invention determines the expression estimation value based on the reference frame point cloud data and the current frame registration point cloud data, and determines the target point cloud data of the current frame based on the expression estimation value and the current frame point cloud data. This can eliminate the influence of facial expression on the target point cloud data of the current frame, thereby eliminating the influence of expression on the target motion information and the reliability of motion information, improving the accuracy of the obtained target motion information and the reliability of motion information, and thus achieving the technical effect of improving the accuracy of the determined motion data. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A schematic flowchart of an embodiment of the head motion data determination method provided by the present invention;

[0049] Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S103;

[0050] Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S203;

[0051] Figure 4 For the present invention Figure 2 A schematic diagram of an embodiment of determining the expression estimate in S204;

[0052] Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S104;

[0053] Figure 6 For the present invention Figure 1 A schematic diagram of an embodiment of S105;

[0054] Figure 7 For the present invention Figure 6 A schematic diagram of an embodiment of determining the reliability value of motion information in S602;

[0055] Figure 8 For the present invention Figure 6 A schematic diagram of an embodiment of S603;

[0056] Figure 9 A schematic diagram of an embodiment of the head motion data determination device provided by the present invention;

[0057] Figure 10 This is a schematic diagram of an embodiment of the head motion tracking device provided by the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," etc., used in the embodiments of the present invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0060] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] This invention provides a method and apparatus for determining head motion data, which will be described below.

[0062] Figure 1 A schematic flowchart of an embodiment of the head motion data determination method provided by the present invention is shown below. Figure 1 As shown, the methods for determining head motion data include:

[0063] S101. Obtain the reference frame point cloud data and the current frame point cloud data for head motion;

[0064] S102. Register the current frame point cloud data based on the reference frame point cloud data to obtain the current frame registered point cloud data;

[0065] S103. Determine the expression estimate based on the reference frame point cloud data and the current frame registration point cloud data;

[0066] S104. Adjust the current frame point cloud data based on the expression estimation value to determine the target point cloud data of the current frame;

[0067] S105. Register the target point cloud data of the current frame with the point cloud data of the reference frame to determine the motion data.

[0068] Compared with the prior art, the head motion data determination method provided in this embodiment of the invention determines the expression estimate based on the reference frame point cloud data and the current frame registration point cloud data, and determines the target point cloud data of the current frame based on the expression estimate and the current frame point cloud data. This can eliminate the influence of facial expression on the target point cloud data of the current frame, thereby eliminating the influence of expression on the target motion information and the reliability of motion information, improving the accuracy of the obtained target motion information and the reliability of motion information, and thus achieving the technical effect of improving the accuracy of the determined motion data.

[0069] In step S101, the head refers to a human head.

[0070] It should be understood that the reference frame point cloud data and the current frame point cloud data can be obtained through point cloud acquisition devices, such as binocular cameras, structured light cameras, and lidar.

[0071] In some embodiments of the present invention, the reference frame point cloud data is point cloud data formed by fusing point cloud data based on a preset number of previous data frames acquired by the point cloud acquisition device.

[0072] In a specific embodiment of the present invention, the reference frame point cloud data is point cloud data formed by fusing the first 16 frames of point cloud data obtained from the point cloud acquisition device.

[0073] In some embodiments of the present invention, the registration algorithm in steps S102 and S105 can be any one of the registration algorithms such as ICP (Iterative Closest Point) algorithm, NICP (Normal Iterative Closest Point) algorithm, and PL-ICP (Point to Line-Iterative Closest Point) algorithm. In a specific embodiment of the present invention, the registration algorithm is the NICP algorithm.

[0074] In a specific embodiment of the present invention, the implementation process of steps S101-S105 is as follows: Reference frame point cloud data and current frame point cloud data of human head movement are acquired using point cloud acquisition devices such as binocular cameras, structured light cameras, and lidar. The current frame point cloud data is registered to the coordinate system of the reference frame point cloud data based on the NICP registration algorithm to obtain the current frame registered point cloud data. An expression estimation value is determined based on the current frame registered point cloud data and the reference frame point cloud data. The target point cloud data of the current frame is determined based on the expression estimation value and the current frame point cloud data. Then, the target point cloud data of the current frame is registered to the coordinate system of the reference frame point cloud data based on the NICP registration algorithm to determine the motion data.

[0075] In some embodiments of the present invention, such as Figure 2 As shown, step S103 includes:

[0076] S201. Construct a two-dimensional projection grid plane based on the reference frame point cloud data. The two-dimensional projection grid plane includes multiple grids.

[0077] S202. Determine the reference projection coordinates of each reference point in the reference frame point cloud data in the two-dimensional projection grid plane, and determine the initial value of each grid in multiple grids based on the reference projection coordinates.

[0078] S203. Determine the projection distance of each current point in the current frame registration point cloud data based on the initial value;

[0079] S204. Determine the expression points and non-expression points in the current point based on the projection distance, and determine the expression estimate based on the expression points.

[0080] This invention improves the convenience and reliability of determining expression points and non-expression points by setting a method to determine expression points and non-expression points at the current point based on the projection distance, thereby improving the speed and reliability of expression estimation.

[0081] Specifically, step S201 involves: constructing a two-dimensional projection plane based on the reference frame point cloud data, and rasterizing the two-dimensional projection plane based on a preset partitioning step size to obtain a two-dimensional projection raster plane.

[0082] In a specific embodiment of the present invention, step S201 is: to obtain the maximum and minimum values ​​of the reference frame point cloud data in the X and Y directions, namely: minX, minY, maxX, maxY, and construct a two-dimensional projection plane based on minX, minY, maxX, maxY. The preset division step size is 3, so the two-dimensional projection plane is divided into a 3×3 grid.

[0083] Specifically, step S202 involves: initially setting the initial value of each grid to 0, determining the grid where each reference point is located based on the reference projection coordinates, and setting the initial value of the grid to 1 when the projection of the reference point is within the grid range.

[0084] In some embodiments of the present invention, the step S204 of determining the expression point and non-expression point in the current point based on the projection distance specifically involves:

[0085] Determine if the projection distance is less than the distance threshold; if the projection distance is less than the distance threshold, the current point is a non-expression point; if the projection distance is greater than or equal to the distance threshold, the current point is an expression point.

[0086] It should be understood that the distance threshold can be set or adjusted according to the actual application scenario or experience value, and no specific limitation is made here.

[0087] In a specific embodiment of the present invention, the specific implementation process of steps S201-S204 is as follows: obtain the maximum and minimum values ​​of the reference frame point cloud data in the X and Y directions, construct a two-dimensional projection plane based on the maximum and minimum values, and perform rasterization processing on the two-dimensional projection plane based on a preset division step size to obtain a two-dimensional projection raster plane; then determine the reference projection coordinates of each reference point in the reference frame point cloud data in the two-dimensional projection raster plane, and determine the initial value of each grid in multiple grids according to the reference projection coordinates; then determine the projection distance of each current point in the current frame registration point cloud data according to the initial value, and then determine whether the projection distance is less than the distance threshold; if the projection distance is less than the distance threshold, the current point is a non-expression point; if the projection distance is greater than or equal to the distance threshold, the current point is an expression point, and finally determine the expression estimation value based on each expression point.

[0088] In some embodiments of the present invention, such as Figure 3 As shown, step S203 includes:

[0089] S301. Determine the current projection coordinates of each current point in the current frame registration point cloud data in the two-dimensional projection grid plane, and determine the target grid where the current point is located based on the current projection coordinates;

[0090] S302. When the initial value of the target raster is a preset initial value, determine the target reference point in the reference frame point cloud data corresponding to the current point, and determine the projection distance of the current point based on the target reference point, the current point, and the normal vector at the target reference point.

[0091] This invention determines the target grid where the current point is located and judges whether the initial value of the target grid is a preset initial value. Only when the initial value of the target grid is a preset initial value is the projection distance between the current point and the corresponding target reference point calculated. This can eliminate false point pairs in the point cloud data of the reference frame and the registered point cloud data of the current frame, thereby improving the accuracy of the determined expression points and non-expression points, and further improving the accuracy of the determined expression estimation value.

[0092] Specifically, step S301 involves determining the target grid where the current projection coordinates are located based on the grid range of each grid.

[0093] The preset initial value in step S302 is 1, that is: when the initial value of the target grid is 1, the projection distance of the current point is determined according to the target reference point, the current point and the normal vector at the target reference point.

[0094] In a specific embodiment of the present invention, the projection distance Dis of the current point in step S302 is:

[0095] Dis = FixNormal f *(Point j -Point f )

[0096] Among them, FixNormal f Point is the normal vector at the target reference point. f The three-dimensional coordinates of the target reference point, Point j These are the three-dimensional coordinates of the current point.

[0097] Both the reference frame point cloud data and the current frame point cloud data are three-dimensional data, which include components in the X, Y, and Z directions.

[0098] In a specific embodiment of the present invention, the specific implementation process of steps S301-S302 is as follows: determine the current projection coordinates of each current point in the current frame registration point cloud data in the two-dimensional projection grid plane, and determine the target grid where the current projection coordinates are located based on the current projection coordinates and the grid range of each grid; when the initial value of the target grid is 1, determine the target reference point corresponding to the current point in the reference frame point cloud data, and calculate and determine the projection distance of each current point in the current frame registration point cloud data based on the projection distance calculation formula.

[0099] Furthermore, since the head may have slightly rotated or moved in the current frame, there may be areas where the point cloud data of the current frame and the point cloud data of the reference frame do not overlap. These non-overlapping areas are generally located at the edges of the point cloud data. Therefore, in order to eliminate the non-overlapping areas between the current frame and the reference frame point cloud data as much as possible, in some embodiments of the present invention, edge erosion processing is performed on the reference binary image. Edge erosion processing can eliminate the non-overlapping areas between the current frame and the reference frame point cloud data, avoiding mismatching of points in non-overlapping areas, thereby further improving the accuracy of the determined expression estimation value.

[0100] Because facial expressions generate expression points in multiple areas of the face—for example, when blinking, expression points often appear not only in the eye area but also on the cheeks and around the corners of the mouth—in practical applications, to avoid treating points from multiple areas as expression points due to overall facial muscle movement, which could lead to inaccurate expression estimation, some embodiments of this invention, such as... Figure 4 As shown, step S204, determining the estimated expression value based on the expression points, includes:

[0101] S401. Determine the maximum connected region of the expression points;

[0102] S402. Use the number of expression points in the largest connected region as the expression estimate.

[0103] By using the number of facial expression points in the largest connected region as the facial expression estimate, this invention can eliminate incorrect estimation of facial expression points caused by the overall movement of facial muscles, thereby further improving the accuracy of the facial expression estimate.

[0104] Specifically, step S401 involves: first determining multiple connected regions of the expression point, then determining the area of ​​each connected region, and selecting the region with the largest area as the largest connected region.

[0105] In a specific embodiment of the present invention, steps S401-S402 are implemented as follows: first, multiple connected regions of the expression points are determined; then, the area of ​​each connected region is determined; the region with the largest area is selected as the largest connected region; finally, the number of expression points in the largest connected region is counted, and the total number of expression points is used as the expression estimate.

[0106] In some embodiments of the present invention, such as Figure 5 As shown, step S104 includes:

[0107] S501. Determine whether the estimated value of the facial expression is greater than the estimated threshold.

[0108] S502. If the estimated expression value is less than the estimated threshold, then the current frame point cloud data is the current frame target point cloud data.

[0109] S503. If the estimated expression value is greater than or equal to the estimated threshold, then remove the expression points from the current frame point cloud data to obtain the target point cloud data of the current frame.

[0110] This invention, by setting the current frame point cloud data as the target point cloud data when the estimated expression value is less than the estimation threshold, can retain as many points as possible in the target point cloud data when the influence of the expression estimate is minimal, thereby improving the reliability of the motion information reliability value calculation. By setting the expression points to be removed when the estimated expression value is greater than or equal to the estimation threshold, the influence of expression points on the motion data can be avoided, further improving the reliability of the determined motion data.

[0111] It should be understood that the estimated threshold can be set or adjusted according to the actual application scenario or experience value, and no specific limitation is made here.

[0112] In some embodiments of the present invention, such as Figure 6 As shown, step S105 includes:

[0113] S601. Register the target point cloud data of the current frame based on the reference frame point cloud data to obtain the target registered point cloud data of the current frame.

[0114] S602. Determine the target motion information and motion information reliability value based on the reference frame point cloud data and the target registration point cloud data of the current frame;

[0115] S603. Determine motion data based on target motion information and motion information reliability values.

[0116] The embodiments of the present invention determine motion data by means of the reliability of motion information and target motion information, rather than by directly obtaining motion data based on the registration results, which can further improve the reliability of the determined motion data.

[0117] In some embodiments of this invention, the target motion information is six-degree-of-freedom (6DOF) information, which includes, but is not limited to, head displacement and rotation information. Typically defined in three-dimensional space, the horizontal direction is the X-axis, the vertical downward direction is the Y-axis, and the Z-axis is obtained according to the right-hand rule and the X and Y axes. The displacement information includes the displacement along the X, Y, and Z directions. The rotation information includes the head pitch angle, yaw angle, and roll angle. Typically defined in three-dimensional space, the horizontal direction is the X-axis, and the angle of head rotation around the X-axis is the pitch angle; the vertical downward direction is the Y-axis, and the angle of head rotation around the Y-axis is the yaw angle; the Z-axis is obtained according to the right-hand rule, and the angle of head rotation around the Z-axis is the roll angle.

[0118] In a specific embodiment of the present invention, the specific method for obtaining the motion translation and motion rotation is as follows: the reference pose information of the head is determined based on the reference frame point cloud data, that is, the reference six degrees of freedom information; the current pose information of the head is determined based on the target registration point cloud data of the current frame, that is, the current six degrees of freedom information; and the difference between the current six degrees of freedom information and the reference six degrees of freedom information is the target motion information.

[0119] In a specific embodiment of the present invention, the specific implementation process of steps S601-S603 is as follows: the target point cloud data of the current frame is registered to the coordinate system of the target point cloud data of the reference frame based on the NICP algorithm to obtain the target registered point cloud data of the current frame; the difference between the current six degrees of freedom information and the reference six degrees of freedom information is determined according to the target motion information and the motion information reliability value; the motion data is determined according to the target motion information and the motion information reliability value.

[0120] In some embodiments of the present invention, such as Figure 7 As shown, in step S602, determining the motion information reliability value based on the reference frame point cloud data and the current frame target registration point cloud data includes:

[0121] S701. Obtain multiple pairs of mapping points from the reference frame point cloud data and the current frame target registration point cloud data. Each pair of mapping points includes a first mapping point and a second mapping point. The first mapping point is the reference mapping point in the reference frame point cloud data, and the second mapping point is the target mapping point in the current frame target registration point cloud data that corresponds to the reference mapping point.

[0122] S702. Determine the target projection distance between the reference mapping point and the target mapping point, and determine the number of target mapping points whose target projection distance is less than a preset distance;

[0123] S703. Determine the ratio of the number of target mapping points whose target projection distance is less than a preset distance to the total number of points in the target frame point cloud data of the current frame, and use the ratio as the reliability value of motion information.

[0124] It should be understood that the motion information reliability value represents the evaluation standard for the registration quality of the reference frame point cloud data and the current frame target registration point cloud data. By determining the motion information reliability value, motion data can be determined under the premise of high registration accuracy, thereby further improving the accuracy and reliability of the determined motion data.

[0125] It should be noted that the target projection distance between the reference mapping point and the target mapping point refers to the distance from the target mapping point to the normal vector plane at the reference mapping point.

[0126] Because a low motion reliability value may lead to incorrect matching between the reference mapping point and the target mapping point, resulting in inaccurate motion information reliability values ​​and consequently inaccurate motion data, in some embodiments of the present invention, such as... Figure 8 As shown, step S603 includes:

[0127] S801. Determine whether the translational motion is greater than the translational motion threshold, whether the rotational motion is greater than the rotational motion threshold, and whether the motion reliability value is greater than the reliability threshold.

[0128] S802. When the motion translation amount is greater than the translation amount threshold or the motion rotation amount is greater than the rotation amount threshold, and the motion reliability value is greater than the reliability threshold, the current frame point cloud data is motion data.

[0129] In this embodiment of the invention, when the motion translation amount is greater than a translation amount threshold or the motion rotation amount is greater than a rotation amount threshold, it is simultaneously determined whether the motion reliability value is greater than a reliability threshold. Only when the motion reliability value is greater than the reliability threshold is the current frame point cloud data considered motion data. This allows the current frame point cloud data with a motion translation amount greater than the translation threshold or a motion rotation amount greater than the rotation threshold to be considered motion data, provided that the overlap rate between the reference frame point cloud data and the current frame target registration point cloud data is high. This avoids the technical problem of misjudging motion data when the overlap rate between the reference frame point cloud data and the current frame target registration point cloud data is low due to head movement, and further improves the accuracy of the determined motion data.

[0130] It should be understood that the translation threshold, rotation threshold, and reliability threshold can be set or adjusted according to the actual application scenario or empirical values. In a specific embodiment of the present invention, the reliability threshold is 0.8.

[0131] Since the current frame point cloud data is determined to be motion data, the next frame point cloud data also needs to be evaluated. To improve the registration reliability between the next frame point cloud data and the reference point cloud data, in some embodiments of the present invention, the head motion data determination method further includes:

[0132] When the current frame point cloud data is motion data, update the reference frame point cloud data based on the current frame point cloud data.

[0133] The specific process of updating the reference frame point cloud data is as follows: superimpose the current frame point cloud data and the reference frame point cloud data to obtain superimposed point cloud data, and downsample the superimposed point cloud data.

[0134] The purpose of downsampling the overlay point cloud data is to make the density of the updated reference frame point cloud data uniform.

[0135] It should be noted that the overlap between the updated reference frame point cloud data and the current frame point cloud data should exceed 95% to ensure a high success rate in registering the next frame point cloud data with the reference point cloud data.

[0136] To better implement the head motion data determination method in the embodiments of the present invention, based on the head motion data determination method, correspondingly, as follows: Figure 9 As shown, this embodiment of the invention also provides a head motion data determination device, the head motion data determination device 900 including:

[0137] The point cloud data acquisition unit 901 is used to acquire the reference frame point cloud data and the current frame point cloud data of the head motion.

[0138] The current frame point cloud data registration unit 902 is used to register the current frame point cloud data based on the reference frame point cloud data to obtain the current frame registered point cloud data;

[0139] The expression estimation value determination unit 903 is used to determine the expression estimation value based on the reference frame point cloud data and the current frame registration point cloud data.

[0140] The current frame target point cloud determination unit 904 is used to adjust the current frame point cloud data according to the expression estimation value and determine the current frame target point cloud data;

[0141] The motion data determination unit 905 is used to register the target point cloud data of the current frame with the point cloud data of the reference frame to determine the motion data.

[0142] The head motion data determination device 900 provided in this embodiment of the invention determines the expression estimate based on the reference frame point cloud data and the current frame registration point cloud data, and determines the target point cloud data of the current frame based on the expression estimate and the current frame point cloud data. This can eliminate the influence of facial expression on the target point cloud data of the current frame, thereby eliminating the influence of expression on the target motion information and the reliability of motion information, improving the accuracy of the obtained target motion information and the reliability of motion information, and thus achieving the technical effect of improving the accuracy of the determined motion data.

[0143] In some embodiments of the present invention, the motion data determination unit 904 is specifically used to: register the target point cloud data of the current frame based on the reference frame point cloud data to obtain the target registered point cloud data of the current frame; determine the target motion information and the motion information reliability value according to the reference frame point cloud data and the target registered point cloud data of the current frame; and determine the motion data according to the target motion information and the motion information reliability value.

[0144] The embodiments of the present invention determine motion data by means of the reliability of motion information and target motion information, rather than by directly obtaining motion data based on the registration results, which can further improve the reliability of the determined motion data.

[0145] The head motion data determination device 900 provided in the above embodiments can realize the technical solutions described in the above head motion data determination method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above head motion data determination method embodiments, and will not be repeated here.

[0146] like Figure 10 As shown, the present invention also provides a head motion tracking device 1000. The head motion tracking device 1000 includes, but is not limited to, medical imaging equipment such as robots, magnetic resonance imaging (MRI) equipment, positron emission tomography (PET) equipment, virtual equipment, and augmented display equipment.

[0147] The head motion tracking device 1000 includes a processor 1001, a memory 1002, and a display 1003. Figure 10 Only some components of the head motion tracking device 1000 are shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented instead.

[0148] In some embodiments, processor 1001 may be a central processing unit (CPU), microprocessor or other data processing chip, used to run program code stored in memory 1002 or process data, such as the head motion data determination method of the present invention.

[0149] In some embodiments, processor 1001 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 1001 may be local or remote. In some embodiments, processor 1001 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0150] In some embodiments, memory 1002 may be an internal storage unit of the head motion tracking device 1000, such as a hard disk or memory of the head motion tracking device 1000. In other embodiments, memory 1002 may also be an external storage device of the head motion tracking device 1000, such as a pluggable hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the head motion tracking device 1000.

[0151] Furthermore, the memory 1002 may include both internal storage units of the head motion tracking device 1000 and external storage devices. The memory 1002 is used to store application software and various types of data installed on the head motion tracking device 1000.

[0152] In some embodiments, display 1003 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1003 is used to display information from the head motion tracking device 1000 and to display a visual user interface. Components 1001-1003 of the head motion tracking device 1000 communicate with each other via a system bus.

[0153] In some embodiments of the present invention, when the processor 1001 executes the head motion data determination program in the memory 1002, the following steps can be implemented:

[0154] Acquire point cloud data of the reference frame and the current frame for head motion;

[0155] Register the current frame point cloud data with the reference frame point cloud data to obtain the current frame registered point cloud data.

[0156] The expression estimate is determined based on the reference frame point cloud data and the current frame registration point cloud data;

[0157] Based on the expression estimation value, adjust the point cloud data of the current frame to determine the target point cloud data of the current frame;

[0158] The current frame target point cloud data is registered based on the reference frame point cloud data to determine the motion data.

[0159] It should be understood that when the processor 1001 executes the head motion data determination program in the memory 1002, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0160] Furthermore, this embodiment of the invention does not specifically limit the type of the head motion tracking device 1000 mentioned. The head motion tracking device 1000 can be a portable head motion tracking device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable head motion tracking devices include, but are not limited to, portable head motion tracking devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable head motion tracking device can also be other portable head motion tracking devices, such as laptop computers with touch-sensitive surfaces (e.g., touch panels).

[0161] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0162] The head motion data determination method and device provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for determining head motion data, characterized in that, include: Acquire point cloud data of the reference frame and the current frame for head motion; Based on the reference frame point cloud data, the current frame point cloud data is registered to obtain the current frame registered point cloud data. The expression estimation value is determined based on the reference frame point cloud data and the current frame registration point cloud data; Based on the estimated facial expression value, the current frame point cloud data is adjusted to determine the target point cloud data for the current frame; Based on the reference frame point cloud data, the target point cloud data of the current frame is registered to determine the motion data.

2. The method for determining head motion data according to claim 1, characterized in that, The step of determining the expression estimate based on the reference frame point cloud data and the current frame registration point cloud data includes: A two-dimensional projection grid plane is constructed based on the reference frame point cloud data, and the two-dimensional projection grid plane includes multiple grids; Determine the reference projection coordinates of each reference point in the reference frame point cloud data in the two-dimensional projection grid plane, and determine the initial value of each grid in the plurality of grids based on the reference projection coordinates; The projection distance of each current point in the current frame registration point cloud data is determined based on the initial value; The expression points and non-expression points in the current point are determined based on the projection distance, and the expression estimate is determined based on the expression points.

3. The method for determining head motion data according to claim 2, characterized in that, The step of determining the projection distance of each current point in the current frame registration point cloud data based on the initial value includes: Determine the current projection coordinates of each current point in the two-dimensional projection grid plane, and determine the target grid where the current point is located based on the current projection coordinates; When the initial value of the target raster is a preset initial value, a target reference point corresponding to the current point in the reference frame point cloud data is determined, and the projection distance of the current point is determined based on the target reference point, the current point, and the normal vector at the target reference point.

4. The method for determining head motion data according to claim 2, characterized in that, Determining the estimated expression value based on the expression points includes: Determine the maximum connected region of the expression point; The number of facial expression points in the largest connected region is used as the estimated value of the facial expression.

5. The method for determining head motion data according to claim 2, characterized in that, The step of adjusting the current frame point cloud data based on the expression estimation value to determine the target point cloud data of the current frame includes: Determine whether the estimated expression value is greater than the estimated threshold; If the estimated expression value is less than the estimated threshold, then the current frame point cloud data is the current frame target point cloud data; If the estimated expression value is greater than or equal to the estimated threshold, then the expression points in the current frame point cloud data are removed to obtain the target point cloud data of the current frame.

6. The method for determining head motion data according to claim 1, characterized in that, The process of registering the reference frame point cloud data and the current frame target point cloud data to determine motion data includes: Based on the reference frame point cloud data, the target point cloud data of the current frame is registered to obtain the target registered point cloud data of the current frame. Based on the reference frame point cloud data and the current frame target registration point cloud data, determine the target motion information and the motion information reliability value; Motion data is determined based on the target motion information and the reliability value of the motion information.

7. The method for determining head motion data according to claim 6, characterized in that, The step of determining the motion information reliability value based on the reference frame point cloud data and the current frame target registration point cloud data includes: Multiple pairs of mapping points are obtained from the reference frame point cloud data and the current frame target registration point cloud data. Each pair of mapping points includes a first mapping point and a second mapping point. The first mapping point is a reference mapping point in the reference frame point cloud data, and the second mapping point is a target mapping point in the current frame target registration point cloud data that corresponds to the reference mapping point. Determine the target projection distance between the reference mapping point and the target mapping point, and determine the number of target mapping points whose target projection distance is less than a preset distance; The ratio of the number of target mapping points whose target projection distance is less than a preset distance to the total number of points in the target frame point cloud data of the current frame is determined, and the ratio is used as the reliability value of the motion information.

8. The method for determining head motion data according to claim 6, characterized in that, The target motion information includes translational and rotational motion; the step of determining motion data based on the target motion information and the reliability value of the motion information includes: Determine whether the motion translation amount is greater than a translation amount threshold, whether the motion rotation amount is greater than a rotation threshold, and whether the motion reliability value is greater than a reliability threshold; When the motion translation amount is greater than the translation amount threshold or the motion rotation amount is greater than the rotation amount threshold, and the motion reliability value is greater than the reliability threshold, the current frame point cloud data is motion data.

9. The method for determining head motion data according to claim 1, characterized in that, The method for determining head motion data further includes: When the current frame point cloud data is motion data, the reference frame point cloud data is updated based on the current frame point cloud data.

10. A head motion data determination device, characterized in that, include: The point cloud data acquisition unit is used to acquire the reference frame point cloud data and the current frame point cloud data of the head motion. The current frame point cloud data registration unit is used to register the current frame point cloud data based on the reference frame point cloud data to obtain the current frame registered point cloud data. The facial expression estimation unit is used to determine the facial expression estimation value based on the reference frame point cloud data and the current frame registration point cloud data. The current frame target point cloud determination unit is used to adjust the current frame point cloud data according to the expression estimation value to determine the current frame target point cloud data; The motion data determination unit is used to register the target point cloud data of the current frame with the point cloud data of the reference frame to determine the motion data.

Images