6Dof handle filtering method and head-mounted display device

Abstract

The invention relates to the technical field of computer vision, and provides a filtering method of a 6Dof handle and head-mounted display equipment, which are used for improving the display quality of handle rays. Considering that the motion state of the 6Dof handle changes continuously in the interaction process, the cut-off frequency of the filter is determined according to the corresponding current speed of the 6Dof handle in the current frame image, and since the cut-off frequency is in direct proportion to the current speed of the 6Dof handle, the filter coefficient determined based on the cut-off frequency can be adaptive to the motion state of the 6Dof handle, so that the filter efficiency is improved. When a filter with a self-adaptive filter coefficient is used for performing smooth filtering on an initial pose of a 6Dof handle in a current frame image and a target pose of the 6Dof handle in a previous frame image, jitter of a handle ray in the image when the 6Dof handle is static can be eliminated, and the handle ray in the image can be ensured to follow the actual action of the handle when the 6Dof handle moves greatly. And the jamming phenomenon is reduced.

Description

Background Art

[0002] The position output by the 6-DOF (Degrees of Freedom, Dof) handle includes displacement and rotation, a total of 6 degrees of freedom. As one of the most common interaction methods of head-mounted display (HMD), the positioning accuracy directly affects the user's interactive experience.

[0003] During the interaction, the motion state of the handle will constantly change, and the point where the rays emitted by the handle land on the UI panel of the head-mounted display device is more easily affected by the change in the angle of the handle, resulting in unstable positioning results. For example, when the handle is placed on a table and is stationary, the ideal angle should remain unchanged, but in reality there will be a change of about 0.05 degrees, which is subjectively manifested as the shaking of the handle rays and a lack of a sense of stillness. Summary of the invention

[0004] The embodiments of the present application provide a 6Dof handle filtering method and a head-mounted display device, which are used to improve the display quality of handle rays during the interaction process.

[0005] In a first aspect, an embodiment of the present application provides a filtering method for a 6Dof controller, comprising:

[0006] Obtaining the initial pose corresponding to the 6Dof controller in the current frame image, and the first target pose corresponding to the 6Dof controller in the previous frame image;

[0007] Acquire the current speed corresponding to the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed; wherein the cutoff frequency is proportional to the current speed;

[0008] Determining, according to the cutoff frequency, a filter coefficient of a filter corresponding to the current frame image, wherein the filter coefficient increases as the cutoff frequency increases;

[0009] The filter is used to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain a second target posture of the 6Dof handle in the current frame image.

[0010] The beneficial effects of the above technical solution are: for the interaction scenario between the 6Dof handle and the head-mounted display device, taking into account the constantly changing motion state of the 6Dof handle, the cutoff frequency of the filter is determined according to the current speed of the 6Dof handle in the current frame image. Since the cutoff frequency is proportional to the current speed of the 6Dof handle, the filter coefficient determined based on the cutoff frequency can adapt to the motion state of the 6Dof handle. The smaller the motion speed of the 6Dof handle, the smaller the filter coefficient, and the greater the motion speed of the 6Dof handle, the larger the filter coefficient. In this way, when a filter with an adaptive filter coefficient is used to smooth the initial posture of the 6Dof handle in the current frame image and the target posture in the previous frame image, not only can the jitter of the handle ray in the image displayed by the head-mounted display device when the 6Dof handle is stationary be eliminated, but also when the 6Dof handle moves significantly, the handle ray in the image displayed by the head-mounted display device can keep up with the actual action, thereby reducing the jamming phenomenon.

[0011] Optionally, after obtaining the target posture of the handle in the current frame image, the method further includes:

[0012] Determine a smooth difference according to an initial pose and a target pose of the 6DOf handle in a current frame image;

[0013] If the smoothing difference is greater than or equal to a first smoothing threshold and less than a second smoothing threshold, an initial filter coefficient is determined according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and a maximum of the initial filter coefficient and a filter coefficient determined according to the cutoff frequency is determined as a final filter coefficient; wherein the first smoothing threshold is less than the second smoothing threshold;

[0014] If the smoothing difference is greater than or equal to the second smoothing threshold, the filter coefficient of the filter is set to 1.

[0015] Optionally, the filter coefficient is determined by:

[0016]

[0017] Wherein, α represents the filtering coefficient, f cut represents the cut-off frequency, |y n -x n | represents the smooth difference, γ0 represents the first smooth threshold, and γ1 represents the second smooth threshold.

[0018] The beneficial effect of the above technical solution is: in order to cope with the situation where the input posture changes slowly and the output posture after filtering and smoothing is increasingly different from the input posture, the filtering process is made smoother by increasing the filtering coefficient, ensuring that the handle ray is displayed continuously and stably in the displayed image.

[0019] Optionally, the posture includes displacement and rotation, the current speed is the moving speed of the handle, and the using of the filter to perform first-order linear filtering on the initial posture and target posture of the handle to obtain the target posture of the handle in the current frame image includes:

[0020] Using a first filter to perform first-order linear filtering on the initial displacement of the 6Dof handle in the current frame image and the target displacement in the previous frame image, to obtain the target displacement of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the first filter is determined according to the cutoff frequency determined by the moving speed;

[0021] A second filter is used to perform first-order linear filtering on the initial rotation of the 6Dof handle in the current frame image and the target rotation in the previous frame image to obtain the target rotation of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the second filter is determined according to the cutoff frequency determined by the angular velocity.

[0022] The beneficial effect of the above technical solution is that different first-order linear filters are used for the displacement and rotation of the handle, thereby improving the smoothing effect.

[0023] Alternatively, the first-order linear filter of the displacement is formulated as:

[0024]

[0025] Wherein, n represents the current frame image, n-1 represents the previous frame image, represents the target displacement of the 6Dof handle in the previous frame image, Indicates the initial displacement of the 6Dof handle in the current frame image, y n t represents the target displacement of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the first filter.

[0026] The beneficial effect of the above technical solution is: the initial displacement of the handle in the current frame image and the target displacement in the previous frame image are filtered by the filter coefficient to obtain the target displacement of the handle in the current frame image, so that the displacement of the moving handle during the interaction process is smoother.

[0027] The formula for the optional, rotated first-order linear filter is:

[0028]

[0029] Wherein, the slerp() is a circular interpolation function, n represents the current frame image, n-1 represents the previous frame image, Indicates the target rotation of the 6Dof handle in the previous frame image, Indicates the initial rotation of the 6Dof handle in the current frame image, represents the target rotation of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the second filter.

[0030] The beneficial effect of the above technical solution is: the initial rotation of the handle in the current frame image and the target rotation in the previous frame image are filtered through the filter coefficient to obtain the target rotation of the handle in the current frame image, so that the rotation of the moving handle during the interaction process is smoother.

[0031] Optionally, the cut-off frequency is determined by:

[0032] f cut =f min +K α |v|

[0033] Among them, K α represents the scale factor, v represents the moving speed or angular velocity of the 6Dof handle corresponding to the current frame image, and f min Represents the minimum frequency of the filter.

[0034] The beneficial effect of the above technical solution is: when the handle is stationary, the speed is zero, at this time the cutoff frequency is equal to the minimum frequency, and the filter coefficient is the smallest. As the handle movement speed increases, the cutoff frequency increases and the filter coefficient also increases, thereby achieving the filter coefficient adaptive to the movement of the handle.

[0035] In a second aspect, an embodiment of the present application provides a head-mounted display device, including a processor, a memory, a display screen, and a communication interface, wherein the communication interface, the display screen, the memory, and the processor are connected via a bus;

[0036] The communication interface is used to communicate with the 6Dof handle;

[0037] The display screen is used to display images;

[0038] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0039] Obtaining the initial pose corresponding to the 6Dof controller in the current frame image, and the first target pose corresponding to the 6Dof controller in the previous frame image;

[0040] Acquire the current speed corresponding to the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed; wherein the cutoff frequency is proportional to the current speed;

[0041] Determining, according to the cutoff frequency, a filter coefficient of a filter corresponding to the current frame image, wherein the filter coefficient increases as the cutoff frequency increases;

[0042] The filter is used to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain a second target posture of the 6Dof handle in the current frame image.

[0043] Optionally, after obtaining the target posture of the handle in the current frame image, the processor further executes:

[0044] Determine a smooth difference according to an initial pose and a target pose of the 6DOf handle in a current frame image;

[0045] If the smoothing difference is greater than or equal to a first smoothing threshold and less than a second smoothing threshold, an initial filter coefficient is determined according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and a maximum of the initial filter coefficient and a filter coefficient determined according to the cutoff frequency is determined as a final filter coefficient; wherein the first smoothing threshold is less than the second smoothing threshold;

[0046] If the smoothing difference is greater than or equal to the second smoothing threshold, the filter coefficient of the filter is set to 1.

[0047] Optionally, the filter coefficient is determined by:

[0048]

[0049] Wherein, α represents the filtering coefficient, f cut represents the cut-off frequency, |y n -x n | represents the smooth difference, γ0 represents the first smooth threshold, and γ1 represents the second smooth threshold.

[0050] Optionally, the posture includes displacement and rotation, the current speed is the moving speed of the handle, and the processor uses the filter to perform first-order linear filtering on the initial posture and target posture of the handle to obtain the target posture of the handle in the current frame image. The specific operation is:

[0051] Using a first filter to perform first-order linear filtering on the initial displacement of the 6Dof handle in the current frame image and the target displacement in the previous frame image, to obtain the target displacement of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the first filter is determined according to the cutoff frequency determined by the moving speed;

[0052] A second filter is used to perform first-order linear filtering on the initial rotation of the 6Dof handle in the current frame image and the target rotation in the previous frame image to obtain the target rotation of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the second filter is determined according to the cutoff frequency determined by the angular velocity.

[0053] Alternatively, the first-order linear filter of the displacement is formulated as:

[0054]

[0055] Wherein, n represents the current frame image, n-1 represents the previous frame image, represents the target displacement of the 6Dof handle in the previous frame image, represents the initial displacement of the 6Dof handle in the current frame image, represents the target displacement of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the first filter.

[0056] The formula for the optional, rotated first-order linear filter is:

[0057]

[0058] Wherein, the slerp() is a circular interpolation function, n represents the current frame image, n-1 represents the previous frame image, Indicates the target rotation of the 6Dof handle in the previous frame image, Indicates the initial rotation of the 6Dof handle in the current frame image, represents the target rotation of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the second filter.

[0059] Optionally, the cut-off frequency is determined by:

[0060] f cut =f min +K α |v|

[0061] Among them, K αrepresents the scale factor, v represents the moving speed or angular velocity of the 6Dof handle corresponding to the current frame image, and f min Represents the minimum frequency of the filter.

[0062] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the steps of any 6Dof handle filtering method provided in the first aspect can be implemented.

[0063] The technical effects brought about by any one of the implementation methods in the second aspect to the third aspect can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0065] Figure 1A and Figure 1B This is a curve diagram of the angle change of the static output of the 6Dof handle provided in an embodiment of the present application;

[0066] Figure 1C This is a diagram showing the jitter effect of the handle ray provided in an embodiment of the present application;

[0067] Figure 2 A flow chart of a filtering method for a 6Dof handle provided in an embodiment of the present application;

[0068] Figure 3 A display effect diagram of a handle ray after filtering is provided for adopting the embodiment of the present application;

[0069] Figure 4 A flow chart of a method for adjusting a filter coefficient is provided for using an embodiment of the present application;

[0070] Figure 5A and Figure 5B A curve diagram of handle angle changes after filtering provided in an embodiment of the present application;

[0071] Figure 6 A structural diagram of a filter device for a 6Dof handle provided in an embodiment of the present application;

[0072] Figure 7 A structural diagram of a head-mounted display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0074] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0075] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0076] The terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0077] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0078] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0079] The following is an overview of the design concept of the embodiments of the present application in conjunction with application scenarios.

[0080] When the controller is placed on a table and remains stationary, the rays emitted by the controller are not completely stationary in the user interface displayed on the head-mounted display device, and there will be slight jitter. Figure 1AThis is a curve diagram of the angle change of the handle output when the handle is stationary. The horizontal axis is time (unit: second) and the vertical axis is angle (unit: degree). There is a change of about 0.05 degrees. Figure 1B is the angle difference between adjacent moments. The angle change difference is quite obvious, and the user can feel the jitter of the handle ray, such as Figure 1C As shown by the dotted line in .

[0081] At present, related technologies usually use traditional linear filters to smooth the positioning results of the handle. The smaller the filter coefficient, the better the smoothing effect. Therefore, in order to better suppress the jitter of the ray when the handle is stationary, the traditional linear filter usually sets a fixed and small filter coefficient. However, since the movement state of the handle is constantly changing during the interaction process, when the filter coefficient is small, the movement amplitude of the handle will be severely suppressed. In this way, the ray displayed in the user interface of the head-mounted display device will not keep up with the actual movement of the handle, forming a ghosting, resulting in a stuttering visual experience.

[0082] In view of this, an embodiment of the present application provides a filtering method for a 6Dof handle, which is used to improve the positioning accuracy of the handle rays under different motion states. The method determines the cutoff frequency of the filter according to the current speed of the handle in the current frame image, and uses the cutoff frequency to determine the filter coefficient of the filter. Since the cutoff frequency is proportional to the current speed of the handle, the filter coefficient can adapt to the motion state of the handle, that is, the smaller the movement speed of the handle, the smaller the filter coefficient, and the greater the movement speed of the handle, the larger the filter coefficient. In this way, when a filter with an adaptive filter coefficient is used to smooth the initial posture of the handle in the current frame image and the target posture in the previous frame image, not only can the jitter of the handle rays in the image displayed by the head-mounted display device when the handle is stationary be eliminated, but also it can ensure that the handle rays in the image displayed by the head-mounted display device when the handle moves significantly can keep up with the actual movement, thereby reducing the jamming phenomenon.

[0083] See also Figure 2 , is a flow chart of the filtering method of the 6Dof handle provided in an embodiment of the present application, which is executed by a head mounted display device and mainly includes the following steps:

[0084] S201: Obtain an initial posture corresponding to the 6Dof handle in the current frame image, and a first target posture corresponding to the 6Dof handle in the previous frame image.

[0085] Among them, the initial posture is the handle posture before smoothing corresponding to the current frame image, including initial displacement and initial rotation, and the first target posture is the handle posture after smoothing corresponding to the previous frame image, including first target displacement and first target rotation.

[0086] Generally, the rotation parameters of the handle are represented by quaternions.

[0087] S202: Obtain the current speed of the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed.

[0088] Among them, the cut-off frequency is proportional to the current speed, and the formula is expressed as:

[0089] f cut =f min +K α |v| Formula 1

[0090] Where K α represents the scale factor, v represents the current speed of the 6Dof controller corresponding to the current frame image, and f min Indicates the minimum frequency of the filter.

[0091] Usually, a 6Dof handle has an inertial measurement unit (IMU) built in, which can measure the acceleration and angular velocity of the 6Dof handle in real time. The acceleration can be used to obtain the movement speed of the 6Dof handle. Therefore, the current speed in formula 1 includes the movement speed and angular velocity of the 6Dof handle corresponding to the current frame image. The movement speed is used to determine the handle displacement, and the angular velocity is used to determine the handle rotation.

[0092] It should be noted that K α and f min It can be set according to actual needs, and the present application embodiment does not impose any restrictive requirements. For example, setting f min Equal to 0.1Hz.

[0093] S203: Determine the filter coefficient of the filter corresponding to the current frame image according to the cutoff frequency.

[0094] Specifically, the formula of the filter coefficient is expressed as:

[0095]

[0096] Among them, 0≤α≤1, the smaller α is, the stronger the smoothing effect is. When the handle is stationary, when |v| is 0, f cut =f min , at this time α reaches the minimum value, as the handle movement speed increases, f cut Gradually increases, α also gradually increases. When |v|→∞, α→1, so that the filter coefficient can be adaptively changed according to the movement of the handle. That is, the smaller the movement speed of the handle, the smaller the cutoff frequency and the smaller the filter coefficient. The larger the movement speed of the handle, the larger the cutoff frequency and the larger the filter coefficient.

[0097] The embodiment of the present application is aimed at the interaction scenario between the 6Dof handle and the head-mounted display device. Considering that the 6Dof handle is in a constant state of motion, the cutoff frequency of the filter is determined according to the current speed of the 6Dof handle in the current frame image. Since the cutoff frequency is proportional to the current speed of the 6Dof handle, the filter coefficient determined based on the cutoff frequency can adapt to the motion state of the 6Dof handle. The smaller the motion speed of the 6Dof handle, the smaller the filter coefficient, and the greater the motion speed of the 6Dof handle, the larger the filter coefficient. In this way, when the filter with an adaptive filter coefficient is used to smooth the initial posture of the 6Dof handle in the current frame image and the target posture in the previous frame image, it can not only eliminate the jitter of the handle ray in the image displayed by the head-mounted display device when the 6Dof handle is stationary, but also ensure that the handle ray in the image displayed by the head-mounted display device when the 6Dof handle moves significantly can keep up with the actual action, thereby reducing the jamming phenomenon.

[0098] S204: Use a filter to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain a second target posture of the 6Dof handle in the current frame image.

[0099] Since the movement speed of the handle includes the moving speed and the angular velocity, the moving speed is used to determine the handle displacement and the angular velocity is used to determine the handle rotation. Therefore, the displacement and rotation of the handle can be smoothed using the filter coefficient corresponding to the moving speed and the filter coefficient corresponding to the angular velocity respectively.

[0100] The first-order linear filtering process of the displacement is as follows: the cutoff frequency of the first filter is determined according to the moving speed of the 6Dof handle, and the filter coefficient of the first filter is determined by the cutoff frequency. The first filter is then used to perform first-order linear filtering on the initial displacement and target displacement of the 6Dof handle to obtain the target displacement of the 6Dof handle in the current frame image. The formula is expressed as follows:

[0101]

[0102] Where n represents the current frame image, n-1 represents the previous frame image, Indicates the target displacement of the handle in the previous frame image Indicates the initial displacement of the 6Dof handle in the current frame image represents the target displacement of the handle in the current frame image, and α is the filter coefficient determined by the cutoff frequency corresponding to the moving speed.

[0103] The first-order linear filtering process of the rotation is as follows: the cutoff frequency of the second filter is determined according to the angular velocity of the 6Dof handle, and the filter coefficient of the second filter is determined by the cutoff frequency. Then, the first-order linear filtering is performed on the initial rotation and target rotation of the 6Dof handle by the second filter to obtain the target rotation of the 6Dof handle in the current frame image. The formula is expressed as follows:

[0104]

[0105] Where slerp() is the arc interpolation function, n represents the current frame image, n-1 represents the previous frame image, Indicates the target rotation of the 6Dof controller in the previous frame. Indicates the initial rotation of the 6Dof handle in the current frame image. Indicates the target rotation of the 6Dof handle in the current frame image, and α is the filter coefficient determined by the cutoff frequency corresponding to the angular velocity.

[0106] In an embodiment of the present application, the filter coefficient of the first filter is determined according to the cutoff frequency corresponding to the moving speed of the 6Dof handle, and the filter coefficient of the second filter is determined according to the cutoff frequency corresponding to the angular velocity of the 6Dof handle, so that the displacement and rotation of the 6Dof handle are respectively smoothed using the first filter and the second filter with different filter coefficients, thereby improving the smoothing effect of the displacement and rotation.

[0107] See also Figure 3 , which is a display effect diagram of the filtering method provided in the embodiment of the present application, the rays of the handle can adapt to the movement state of the handle. When the handle is stationary, the user cannot perceive the jitter of the rays. When the handle moves, the rays can keep up with the actual movement of the handle.

[0108] In one example, to deal with the filter input x n Slowly changing, and smoothed output y n With input x n When the difference between the two is getting bigger and bigger, the filter coefficient needs to be increased. Figure 4 , mainly includes the following steps:

[0109] S205: Determine a smooth difference according to the initial position and target position of the handle in the current frame image.

[0110] The smooth difference includes a first smooth difference and a second smooth difference, the first smooth difference is the difference between the initial displacement and the target displacement, and the second smooth difference is the difference between the initial rotation and the target rotation.

[0111] S206: Determine whether the smooth difference is greater than or equal to the first smooth threshold and less than the second smooth threshold. If so, execute S206. If the smooth difference is greater than or equal to the second smooth threshold, execute S207. If the smooth difference is less than the first smooth threshold, execute S208.

[0112] The first smoothing threshold is smaller than the second smoothing threshold.

[0113] Optionally, the smoothing thresholds corresponding to the displacement and the rotation may be different.

[0114] S207: determining an initial filter coefficient according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and determining the largest one of the initial filter coefficient and the filter coefficient determined according to the cutoff frequency as the final filter coefficient.

[0115] Specifically, the first initial filter coefficient of the displacement is determined based on the first smoothing difference, the first smoothing threshold and the second smoothing threshold corresponding to the displacement, the first initial filter coefficient is compared with the filter coefficient determined according to the cutoff frequency corresponding to the moving speed, and the largest one is determined as the final filter coefficient of the displacement; and the second initial filter coefficient of the rotation is determined based on the second smoothing difference, the first smoothing threshold and the second smoothing threshold corresponding to the rotation, the second initial filter coefficient is compared with the filter coefficient determined according to the cutoff frequency corresponding to the angular velocity, and the largest one is determined as the final filter coefficient of the rotation.

[0116] S208: Set the filter coefficient of the filter to 1.

[0117] Specifically, when the first smooth difference is greater than or equal to the first smooth threshold corresponding to the displacement and less than the second smooth threshold corresponding to the displacement, the filter coefficient of the first filter is set to 1; when the second smooth difference is greater than or equal to the first smooth threshold corresponding to the rotation and less than the second smooth threshold corresponding to the rotation, the filter coefficient of the second filter is set to 1.

[0118] S209: Keep the filter coefficient unchanged.

[0119] Specifically, when the first smooth difference is less than the first smooth threshold corresponding to the displacement, the filter coefficient of the first filter is kept unchanged, that is, the filter coefficient of the first filter is determined according to the cutoff frequency corresponding to the moving speed of the 6DOf handle; when the second smooth difference is less than the first smooth threshold corresponding to the rotation, the filter coefficient of the second filter is kept unchanged, that is, the filter coefficient of the second filter is determined according to the cutoff frequency corresponding to the angular velocity of the 6DOf handle.

[0120] According to the description of S205 to S209, the final calculation formula of the filter coefficient of the filter in the embodiment of the present application is:

[0121]

[0122] Among them, α represents the filter coefficient, f cut represents the cutoff frequency, |y n -x n | represents the smoothed difference, γ0 represents the first smoothing threshold, and γ1 represents the second smoothing threshold.

[0123] In an embodiment of the present application, in order to cope with the situation where the input posture changes slowly and the output posture after filtering and smoothing is increasingly different from the input posture, the filtering process is made smoother by increasing the filtering coefficient to ensure that the handle ray is displayed continuously and stably in the displayed image.

[0124] The effect of smoothing the positioning of the 6Dof handle using the filtering method of the embodiment of the present application is shown in the figure below: Figure 5A and Figure 5B As shown, Figure 5A This is the angle change effect diagram after the handle rotation is smoothed, relative to Figure 1A , after smoothing, the angle change of the handle becomes more gradual. Figure 5B The effect diagram of the angle difference change at adjacent moments after the rotation of the handle is smoothed, relative to Figure 1B After smoothing, the angle change of the handle is more gradual, and the user cannot feel the jitter of the ray when the handle is still.

[0125] Based on the same technical concept, an embodiment of the present application provides a filtering device for a 6Dof handle, which can implement the steps of the above-mentioned filtering method for a 6Dof handle and achieve the same technical effect.

[0126] See also Figure 6 The filtering device includes an acquisition module 601, a filtering parameter determination module 602, and a filtering processing module 603, wherein:

[0127] An acquisition module 601 is used to acquire an initial posture corresponding to the 6Dof handle in the current frame image, and a first target posture corresponding to the 6Dof handle in the previous frame image;

[0128] A filter parameter determination module 602 is used to obtain the current speed corresponding to the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed; and, according to the cutoff frequency, determine the filter coefficient of the filter corresponding to the current frame image; wherein the cutoff frequency is proportional to the current speed, and the filter coefficient increases as the cutoff frequency increases;

[0129] The filtering processing module 603 is used to use the filter to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain the second target posture of the 6Dof handle in the current frame image.

[0130] Optionally, the filtering parameter determination module 602 is further used to:

[0131] Determine a smooth difference according to an initial pose and a target pose of the 6DOf handle in a current frame image;

[0132] If the smoothing difference is greater than or equal to a first smoothing threshold and less than a second smoothing threshold, an initial filter coefficient is determined according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and a maximum of the initial filter coefficient and a filter coefficient determined according to the cutoff frequency is determined as a final filter coefficient; wherein the first smoothing threshold is less than the second smoothing threshold;

[0133] If the smoothing difference is greater than or equal to the second smoothing threshold, the filter coefficient of the filter is set to 1.

[0134] Optionally, the filter coefficient is determined by:

[0135]

[0136] Wherein, α represents the filtering coefficient, f cut represents the cut-off frequency, |y n -x n | represents the smooth difference, γ0 represents the first smooth threshold, and γ1 represents the second smooth threshold.

[0137] Optionally, the posture includes displacement and rotation, the current speed is the moving speed of the handle, and the filtering processing module 603 is specifically used for:

[0138] Using a first filter to perform first-order linear filtering on the initial displacement of the 6Dof handle in the current frame image and the target displacement in the previous frame image, to obtain the target displacement of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the first filter is determined according to the cutoff frequency determined by the moving speed;

[0139] A second filter is used to perform first-order linear filtering on the initial rotation of the 6Dof handle in the current frame image and the target rotation in the previous frame image to obtain the target rotation of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the second filter is determined according to the cutoff frequency determined by the angular velocity.

[0140] Alternatively, the first-order linear filter of the displacement is formulated as:

[0141]

[0142] Wherein, n represents the current frame image, n-1 represents the previous frame image, represents the target displacement of the 6Dof handle in the previous frame image, represents the initial displacement of the 6Dof handle in the current frame image, represents the target displacement of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the first filter.

[0143] The formula for the optional, rotated first-order linear filter is:

[0144]

[0145] Wherein, the slerp() is a circular interpolation function, n represents the current frame image, n-1 represents the previous frame image, Indicates the target rotation of the 6Dof handle in the previous frame image, Indicates the initial rotation of the 6Dof handle in the current frame image, represents the target rotation of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the second filter.

[0146] Optionally, the cut-off frequency is determined by:

[0147] f cut =f min +K α |v|

[0148] Among them, K α represents the scale factor, v represents the moving speed or angular velocity of the 6Dof handle corresponding to the current frame image, and f min Represents the minimum frequency of the filter.

[0149] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0150] After introducing the filtering method and apparatus of the 6Dof handle according to an exemplary embodiment of the present application, next, a head-mounted display device according to another exemplary embodiment of the present application is introduced.

[0151] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0152] Based on the same inventive concept as the above method embodiment, the head mounted display device provided in the embodiment of the present application. In this embodiment, the structure of the head mounted display device can be as follows: Figure 7 As shown, it includes a processor 701, a memory 702, a display screen 703 and a communication interface 704;

[0153] The communication interface 704 is used to communicate with the 6Dof handle;

[0154] The display screen 703 is used to display images;

[0155] The memory 702 stores a computer program, and the processor 701 executes the steps of any one of the 6Dof handle filtering methods in the above embodiments according to the computer program.

[0156] In the embodiment of the present application, the memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. The memory 702 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 702 may be any other medium that can be used to carry or store a desired computer program in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 may be a combination of the above memories.

[0157] The processor 701 may include one or more central processing units (CPU), GPU or a digital processing unit, etc.

[0158] In the embodiment of the present application, the specific connection medium between the communication interface 704, the display screen 703, the memory 702 and the processor 701 is not limited. In the embodiment of the present application, the bus 705 between the communication interface 704, the display screen 703, the memory 702 and the processor 701 is Figure 7 The connections between the other components are only for illustration and are not intended to be limiting. The bus 705 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 7 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0159] It should be noted that Figure 7 It is only the equipment necessary for the head-mounted display device to implement the 6Dof handle filtering in the embodiment of the present application. Not shown, the head-mounted display device may also include the hardware of conventional head-mounted display devices such as IMU, camera, speaker, microphone, power supply, buttons, etc.

[0160] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions. When these instructions are executed, the steps of any one of the 6Dof handle filtering methods in the aforementioned embodiments can be completed.

[0161] An embodiment of the present application also provides a computer program product for storing a computer program, which is used to execute the steps of any one of the 6Dof handle filtering methods in the aforementioned embodiments.

[0162] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0163] The present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram, and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart or multiple flows and / or one box or multiple boxes in the block diagram.

[0164] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0166] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A filtering method for a 6Dof handle, characterized in that: Applied to head-mounted display devices, including: Obtaining the initial pose corresponding to the 6Dof controller in the current frame image, and the first target pose corresponding to the 6Dof controller in the previous frame image; Acquire the current speed corresponding to the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed; wherein the cutoff frequency is proportional to the current speed; Determining, according to the cutoff frequency, a filter coefficient of a filter corresponding to the current frame image, wherein the filter coefficient increases as the cutoff frequency increases; The filter is used to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain a second target posture of the 6Dof handle in the current frame image.

2. The method according to claim 1, characterized in that After obtaining the target posture of the handle in the current frame image, the method further includes: Determine a smooth difference according to an initial pose and a target pose of the 6DOf handle in a current frame image; If the smoothing difference is greater than or equal to a first smoothing threshold and less than a second smoothing threshold, an initial filter coefficient is determined according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and a maximum of the initial filter coefficient and a filter coefficient determined according to the cutoff frequency is determined as a final filter coefficient; wherein the first smoothing threshold is less than the second smoothing threshold; If the smoothing difference is greater than or equal to the second smoothing threshold, the filter coefficient of the filter is set to 1.

3. The method according to claim 2, characterized in that The formula for determining the filtering coefficient is: Wherein, α represents the filtering coefficient, f cut represents the cut-off frequency, |y n -x n | represents the smooth difference, γ0 represents the first smooth threshold, and γ1 represents the second smooth threshold.

4. The method according to claim 1, characterized in that The posture includes displacement and rotation, the current speed is the moving speed of the handle, and the filter is used to perform first-order linear filtering on the initial posture and target posture of the handle to obtain the target posture of the handle in the current frame image, including: Using a first filter to perform first-order linear filtering on the initial displacement of the 6Dof handle in the current frame image and the target displacement in the previous frame image, to obtain the target displacement of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the first filter is determined according to the cutoff frequency determined by the moving speed; A second filter is used to perform first-order linear filtering on the initial rotation of the 6Dof handle in the current frame image and the target rotation in the previous frame image to obtain the target rotation of the 6Dof handle in the current frame image; wherein the filter coefficient corresponding to the second filter is determined according to the cutoff frequency determined by the angular velocity.

5. The method according to claim 4, characterized in that The formula for the first-order linear filter of displacement is: Wherein, n represents the current frame image, n-1 represents the previous frame image, represents the target displacement of the 6Dof handle in the previous frame image, represents the initial displacement of the 6Dof handle in the current frame image, represents the target displacement of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the first filter.

6. The method according to claim 4, characterized in that The formula for the rotated first-order linear filter is: Wherein, the slerp() is a circular interpolation function, n represents the current frame image, n-1 represents the previous frame image, Indicates the target rotation of the 6Dof handle in the previous frame image, Indicates the initial rotation of the 6Dof handle in the current frame image, represents the target rotation of the 6Dof handle in the current frame image, and α represents the filtering coefficient of the second filter.

7. The method according to any one of claims 1 to 6, characterized in that The cut-off frequency is determined by: f cut =f min +K α |v| Among them, K α represents the scale factor, v represents the moving speed or angular velocity of the 6Dof handle corresponding to the current frame image, and f min Represents the minimum frequency of the filter.

8. A head mounted display device, characterized in that: It includes a processor, a memory, a display screen and a communication interface, wherein the communication interface, the display screen, the memory and the processor are connected via a bus; The communication interface is used to communicate with the 6Dof handle; The display screen is used to display images; The memory stores a computer program, and the processor performs the following operations according to the computer program: Obtaining the initial pose corresponding to the 6Dof controller in the current frame image, and the first target pose corresponding to the 6Dof controller in the previous frame image; Acquire the current speed corresponding to the 6Dof handle in the current frame image, and determine the cutoff frequency of the filter according to the current speed; wherein the cutoff frequency is proportional to the current speed; Determining, according to the cutoff frequency, a filter coefficient of a filter corresponding to the current frame image, wherein the filter coefficient increases as the cutoff frequency increases; The filter is used to perform first-order linear filtering on the initial posture and target posture of the 6Dof handle to obtain a second target posture of the 6Dof handle in the current frame image.

9. The head mounted display device according to claim 8, wherein: After obtaining the target position of the handle in the current frame image, the processor further executes: Determine a smooth difference according to an initial pose and a target pose of the 6DOf handle in a current frame image; If the smoothing difference is greater than or equal to a first smoothing threshold and less than a second smoothing threshold, an initial filter coefficient is determined according to the smoothing difference, the first smoothing threshold and the second smoothing threshold, and a maximum of the initial filter coefficient and a filter coefficient determined according to the cutoff frequency is determined as a final filter coefficient; wherein the first smoothing threshold is less than the second smoothing threshold; If the smoothing difference is greater than or equal to the second smoothing threshold, the filter coefficient of the filter is set to 1.

10. The head mounted display device according to claim 9, wherein: The formula for determining the filtering coefficient is: Wherein, α represents the filtering coefficient, f cut represents the cut-off frequency, |y n -x n | represents the smooth difference, γ0 represents the first smooth threshold, and γ1 represents the second smooth threshold.

Images