Remote control interaction method and device, terminal and storage medium

Abstract

The invention provides a remote control interaction method and device, a terminal and a storage medium, and the method comprises the steps: carrying out the motion judgment according to the three-dimensional motion data collected by a built-in motion sensor on remote control equipment when the control mode of the remote control equipment is detected to be a motion control mode; determining a target action executed when a user holds the remote control equipment by hand; and determining whether to trigger a corresponding action instruction based on the target action, so that the remote control equipment interacts with the to-be-interacted equipment based on a control command corresponding to the action instruction. The action control mode is compatible on the basis that the remote control device has the numerical control mode, more interactive operations can be expanded, and the action instruction is triggered through the target action executed when the user holds the remote control device by hand. And the action control-based interaction mode and interaction fun realized by executing the target action when the user holds the remote control equipment by hand are increased.

Description

Technical Field

[0001] The present invention relates to the field of remote control interaction technology, and in particular to a remote control interaction method, device, terminal and storage medium. Background Art

[0002] At present, the existing instructions generated by sensor data are mostly numerical control, that is, directly mapping the specific data obtained by the sensor, resulting in all instructions being basic displacements or rotations around the axis within the three-dimensional rectangular coordinate system. The instructions are single and lack interest, and this type of numerical control is generally high-frequency and continuous, which is related to the frequency of motion data returned by the sensor.

[0003] For example, an air mouse remote control maps the rotation angle changes of the remote control's z-axis and x-axis to the focus movement of the TV's two-dimensional plane. When the air mouse remote control is shaken, a specific focus movement instruction can be generated, wherein the direction, speed and distance of the focus movement can be regarded as the data mapping generated by the sensor. However, this interaction method based on numerical control is similar to the interaction between the existing mouse and computer equipment, and cannot realize the action control interaction method, that is, it cannot realize the interaction method in which the user directly triggers the action instructions with corresponding control commands by performing corresponding actions when holding the air mouse remote control. Only focus movement instructions can be generated, and there are the disadvantages of single instructions and lack of fun. For another example, in the existing smart TV system, generally only by pressing the setting button on the air mouse remote control on the TV system home page, or moving the focus to the setting button on the TV system home page and pressing the confirm button on the air mouse remote control, can the system settings be evoked, and the system settings cannot be directly evoked in other applications.

[0004] In summary, the existing remote control interaction method based on numerical control has the disadvantages of single instructions and lack of fun. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a remote control interaction method, device, terminal and storage medium are provided, which can achieve a richer interactive experience and increase the action-controlled interaction mode and interaction fun achieved by the execution of target actions when the user holds the remote control device.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] A remote control interaction method, wherein the method comprises:

[0008] When it is detected that the control mode of the remote control device is the motion control mode, motion determination is performed based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device to determine the target motion performed by the user when holding the remote control device;

[0009] Based on the target action, it is determined whether to trigger a corresponding action instruction, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction.

[0010] In one implementation, the performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes:

[0011] Determine whether the action performed by the user holding the remote control device meets a preset hook action determination condition according to the three-dimensional motion data collected by the built-in motion sensor on the remote control device;

[0012] When the action performed by the user holding the remote control device meets the preset hook action determination condition, it is determined that the target action performed by the user holding the remote control device is a hook action;

[0013] Among them, the preset hook action determination condition is:

[0014] The starting action state and the ending action state corresponding to the action performed by the user holding the remote control device satisfy the first preset starting state and the first preset ending state, and the time interval of the action performed by the user holding the remote control device during the transition from the starting action state to the ending action state is less than the first preset time interval, and in the process of the transition from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle between the gravity direction and the x-axis direction changes less than the first preset angle value;

[0015] Furthermore, the first preset starting state is:

[0016] The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the first preset starting state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the second preset starting state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the third preset starting state;

[0017] The first preset end state is:

[0018] The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the first preset end state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the second preset end state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the third preset end state;

[0019] Wherein, a three-dimensional rectangular coordinate system consisting of an x-axis, a y-axis and a z-axis is established with the center of the remote control device as the origin;

[0020] The x-axis is a coordinate axis that points from the left side to the right side when the remote control device is placed horizontally with the front side facing upward and the back side facing downward. The y-axis is a coordinate axis that points from the rear end to the front end when the remote control device is placed horizontally with the front side facing upward and the back side facing downward. The z-axis is a coordinate axis that is perpendicular to the x-axis and the y-axis and points from the back end to the front end.

[0021] Furthermore, the sum of the gravity on the x-axis, y-axis and z-axis is 1, and the gravity direction of the remote control device is vertically downward.

[0022] In one implementation, determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes:

[0023] When the target action is the hook action, the remote control device is controlled to trigger a hook action instruction corresponding to the hook action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the hook action instruction.

[0024] In one implementation, the performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes:

[0025] Determining whether the action performed by the user holding the remote control device meets a preset swing action determination condition based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device;

[0026] When the action performed by the user while holding the remote control device meets the preset swing action determination condition, it is determined that the target action performed by the user while holding the remote control device is a swing action;

[0027] Wherein, the preset swing action determination condition is:

[0028] The starting action state and the ending action state corresponding to the action performed by the user holding the remote control device satisfy the second preset starting state and the second preset ending state, and the time interval between the transition from the starting action state to the ending action state of the action performed by the user holding the remote control device is less than the second preset time interval, and in the process of the transition from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle between the gravity direction and the x-axis direction changes less than the second preset angle value;

[0029] Furthermore, the second preset starting state is:

[0030] The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the fourth preset starting state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the fifth preset starting state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the sixth preset starting state;

[0031] The second preset end state is:

[0032] The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the fourth preset end state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the fifth preset end state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the sixth preset end state.

[0033] In one implementation, determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes:

[0034] When the target action is the swing action, the remote control device is controlled to trigger a swing action instruction corresponding to the swing action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the swing action instruction.

[0035] In one implementation, the performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes:

[0036] Determining whether the action performed by the user holding the remote control device meets a preset rotation action determination condition based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device;

[0037] When the action performed by the user while holding the remote control device meets the preset rotation action determination condition, it is determined that the target action performed by the user while holding the remote control device is a rotation action;

[0038] The preset rotation action determination conditions include a left rotation action determination condition, a right rotation action determination condition, and a back-and-forth rotation action determination condition;

[0039] Furthermore, the left-hand rotation action determination condition is:

[0040] When the user holds the remote control device to perform an action, the angle of the remote control device rotating counterclockwise around the y-axis exceeds a third preset angle value, and the angle of the remote control device rotating around the x-axis and the z-axis does not exceed a fourth preset angle value, and the rotation time during the rotation around the axis is less than the first preset rotation time interval;

[0041] The right-hand action determination condition is:

[0042] When the user holds the remote control device to perform an action, the angle of the clockwise rotation of the remote control device around the y-axis exceeds the fifth preset angle value, and the angle of rotation around the x-axis and the z-axis does not exceed the sixth preset angle value, and the rotation time during the rotation around the axis is less than the second preset rotation time interval;

[0043] The back and forth rotation action determination condition is:

[0044] When the user holds the remote control device to perform an action, the time interval from the left rotation action to the right rotation action is less than the third preset rotation time interval, or the time interval from the right rotation action to the left rotation action is less than the third preset rotation time interval.

[0045] In one implementation, determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes:

[0046] When the target action is the rotation action, the remote control device is controlled to trigger a rotation action instruction corresponding to the rotation action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the rotation action instruction.

[0047] The present invention also discloses a remote control interaction device, wherein the device comprises:

[0048] An action determination module, for determining the target action performed by the user holding the remote control device when the control mode of the remote control device is detected to be the action control mode, based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device;

[0049] The instruction triggering module is used to determine whether to trigger a corresponding action instruction based on the target action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction.

[0050] The present invention also discloses a terminal, which includes: a memory, a processor, and a remote control interaction program stored in the memory and executable on the processor, wherein the remote control interaction program implements the steps of the remote control interaction method described above when executed by the processor.

[0051] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the remote control interaction method as described above.

[0052] The present invention provides a remote control interaction method, device, terminal and storage medium, the remote control interaction method comprising: when it is detected that the control mode of the remote control device is the motion control mode, the action judgment is performed according to the three-dimensional motion data collected by the built-in motion sensor of the remote control device to determine the target action performed when the user holds the remote control device; based on the target action, it is determined whether to trigger the corresponding action instruction, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction. It can be seen that the present invention expands more interactive operations by being compatible with the action control mode on the basis of the numerical control mode of the remote control device, and in the action control mode, the corresponding action instruction is triggered by the target action performed when the user holds the remote control device, and different action instructions correspond to different control commands, thereby achieving a richer interactive experience, and increasing the interactive mode and interactive fun based on action control achieved by the execution of the target action when the user holds the remote control device. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of a preferred embodiment of the remote control interaction method of the present invention;

[0054] Figure 2 It is a flow chart of a specific compatible system control method disclosed in the present invention;

[0055] Figure 3 It is a specific flow chart of the remote control interaction method disclosed in the present invention;

[0056] Figure 4 It is a schematic diagram of a three-dimensional rectangular coordinate system corresponding to the remote control device disclosed in the present invention;

[0057] Figure 5 It is a schematic diagram of a hook action disclosed in the present invention;

[0058] Figure 6 It is a specific hook action judgment flow chart disclosed in the present invention;

[0059] Figure 7 It is a specific flow chart of the remote control interaction method disclosed in the present invention;

[0060] Figure 8 It is a schematic diagram of a swinging action disclosed in the present invention;

[0061] Fig. 9 It is a specific flow chart of the remote control interaction method disclosed in the present invention;

[0062] Fig.10 It is a schematic diagram of a rotation action disclosed in the present invention;

[0063] Fig.11It is a functional principle block diagram of a preferred embodiment of the remote control interaction device of the present invention;

[0064] Fig.12 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0066] See also Figure 1 , Figure 1 Flowchart of the remote control interaction method in the present invention. Figure 1 As shown, the remote control interaction method described in the embodiment of the present invention includes:

[0067] Step S11: when it is detected that the control mode of the remote control device is the motion control mode, motion determination is performed based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device to determine the target motion performed by the user holding the remote control device.

[0068] It should be pointed out that action control is based on the change of motion sensor values. When the user wants to trigger action control without triggering numerical control, it is necessary to achieve compatibility between existing numerical control and action control. For example, a button can be added to achieve compatible interactive behavior. When the button is not pressed, the sensor numerical control is performed normally. When the button is pressed, the action control is started and the numerical control is suspended as needed. In other words, the switching between the numerical control mode and the action control mode of the remote control device is controlled by the pressing and lifting settings of the mode control button. It can be understood that by agreeing on the pressing or lifting state of the mode control button on the remote control device to switch the numerical control mode and the action control mode, the compatibility of action control and numerical control in a system is achieved. At the same time, by designing the button position, the user's hand-held device posture can be guided and adjusted to make it more in line with the interaction expectations, so as to improve the accuracy of action judgment. And through this compatible setting, the sending of instructions in the numerical control mode can be suspended during the action control mode, thereby saving unnecessary data transmission resources.

[0069] In this embodiment, when it is detected that the control mode of the remote control device is in the motion control mode, the motion determination is performed based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device to determine the target motion performed by the user when holding the remote control device. It can be understood that based on the pre-defined motion standard, by obtaining the data of the motion sensor, the motion performed by the user when holding the remote control device is identified and determined, so that the user's holding motion can be standardized to a certain extent, so that the user's behavior judgment is more in line with expectations.

[0070] Step S12: determining whether to trigger a corresponding action instruction based on the target action, so that the remote control device interacts with the device to be interacted with based on a control command corresponding to the action instruction.

[0071] It is understandable that, based on the existing numerical control, motion control can be achieved through a series of predefined action standards, that is, the user presses the button of the remote control device to switch the motion control mode, and then in the motion control mode, the user can trigger the corresponding action command by making an action through the handheld remote control device, and the triggered action command can be used for human-computer interaction. For example, the user can trigger the corresponding action command by making a hook action, a swing action, or a rotation action through the handheld remote control device, and then according to the agreement that different action commands correspond to different control commands, the remote control device can interact with the device to be interacted based on the control command corresponding to the action command, thereby expanding more interactive operations without adding hardware buttons. In actual scene applications, the remote control device can be a remote control or smart device with a built-in motion sensor, such as a gyroscope or accelerometer, so that new interaction methods can be expanded in human-computer interaction fields such as remote control or gaming. For example, without hardware changes, motion control expansion can be achieved on the basis of existing numerical control systems such as air mice.

[0072] For example, when the remote control device is a TV remote control, the action command can generate and send a specific remote control command. When the remote control device is a handheld game device, the action command can represent a specific command in an electronic game or serve as a scene trigger condition. For another example, in a smart TV system, it can be agreed that the hook and shake actions trigger detection, and it can be agreed that the hook action corresponds to calling up the system settings page, and the shake action corresponds to folding the system settings page. This can realize directly calling up or hiding the system settings page in any application, or combining with the existing TV focus system to realize the operation of the action command focus point, that is, agreeing to trigger detection with left or right rotation, and agreeing on the application list page that the left rotation command is pinned to the top, and the right rotation command is canceled to the top, so that when the focus is dropped to the application icon, the left rotation action or the right rotation action interaction can be used to operate the application pinning and cancel the application pinning. In other words, the agreed action can be corresponded to other control operations to achieve the corresponding action control expansion.

[0073] It can be seen that in the embodiments of the present invention, by making the motion control mode compatible with the numerical control mode of the remote control device, the compatibility of the motion control mode and the numerical control mode is achieved, thereby being able to expand more interactive operations, and in the motion control mode, the corresponding action instructions are triggered by the target action performed when the user holds the remote control device, and different action instructions correspond to different control commands, thereby achieving a richer interactive experience and increasing the interactive method and interactive fun based on motion control achieved by the execution of the target action when the user holds the remote control device.

[0074] For example, see Figure 2 As shown, the gravity direction information, three-axis acceleration information or trigonometric Euler value information of the remote control device itself is determined based on the three-dimensional attributes and basic motion data sources obtained from the outside, and then based on this information, it is detected whether the user presses the behavior detection button set on the remote control device. When the user presses the button, the action is judged based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device, and then it is determined whether a certain action instruction is triggered. If so, an action instruction is generated, and the outside responds based on the action instruction; when the user does not press the button, the existing numerical control method is turned on to implement the existing motion sensor numerical control, that is, the outside responds directly based on the sensor data.

[0075] See also Figure 3 As shown, the embodiment of the present invention discloses a specific remote control interaction method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution.

[0076] Step S21: when it is detected that the control mode on the remote control device is the motion control mode, determine whether the action performed by the user holding the remote control device meets the preset hook action judgment condition based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device.

[0077] In this embodiment, according to the predefined hook action standard (i.e., the preset hook action judgment condition), the action performed by the user when holding the remote control device is identified and judged by acquiring the data of the motion sensor to determine whether the action performed by the user when holding the remote control device is a hook action.

[0078] It should be noted that see Figure 4As shown, a three-dimensional rectangular coordinate system consisting of an x-axis, a y-axis and a z-axis is established with the center of the remote control device as the origin, the x-axis is a coordinate axis that points from the left side to the right side when the remote control device is in a horizontal position with the front side facing upward and the back side facing downward, the y-axis is a coordinate axis that points from the rear end to the front end when the remote control device is in a horizontal position with the front side facing upward and the back side facing downward, the z-axis is a coordinate axis that is perpendicular to the x-axis and the y-axis and points from the back end to the front end, and the sum of the gravity forces on the x-axis, the y-axis and the z-axis is 1, and the gravity direction of the remote control device is vertically downward from the ground.

[0079] It should also be pointed out that the preset hook action judgment condition is: the starting action state and the ending action state corresponding to the action performed when the user holds the remote control device satisfy the first preset starting state and the first preset ending state, and the time interval for the action performed when the user holds the remote control device to transition from the starting action state to the ending action state is less than the first preset time interval, and in the process of transitioning from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle change between the gravity direction and the x-axis direction is less than the first preset angle value.

[0080] Furthermore, the first preset starting state is: the angle between the x-axis direction of the remote control device and the gravity direction meets the angle range of the first preset starting state, the angle between the y-axis direction and the gravity direction meets the angle range of the second preset starting state, and the angle between the z-axis direction and the gravity direction meets the angle range of the third preset starting state; the first preset ending state is: the angle between the x-axis direction of the remote control device and the gravity direction meets the angle range of the first preset ending state, the angle between the y-axis direction and the gravity direction meets the angle range of the second preset ending state, and the angle between the z-axis direction and the gravity direction meets the angle range of the third preset ending state. For example, the angle range of the first preset starting state can be 60 degrees to 120 degrees, the angle range of the second preset starting state can be 120 degrees to 160 degrees, the angle range of the third preset starting state can be 120 degrees to 90 degrees, and the angle range of the first preset ending state can be 60 degrees to 120 degrees, the angle range of the second preset ending state can be 180 degrees to 150 degrees, and the angle range of the third preset ending state can be 90 degrees to 60 degrees.

[0081] For example, see Figure 5As shown, when the user holds the remote control device to perform a standard hook action, the remote control device will transition from state 1 to state 2 in a short time, such as transitioning from state 1 to state 2 within 0.4 seconds, and during the transition from state 1 to state 2, the main rotation axis is the x-axis, the gravity value of the remote control device on the y-axis is continuously less than 0, and the angle change between the gravity direction and the x-axis direction is less than a first preset angle value, such as the gravity angle change on the x-axis is less than 30 degrees, that is, the first preset angle value can be set to 30 degrees, that is, the hook action standard is as shown in Table 1:

[0082] Table 1

[0083]

[0084] In addition, for the specific judgment process of the hook action, see Figure 6 As shown, that is, according to the three-dimensional motion data collected by the built-in motion sensor on the remote control device, it is first determined whether the starting action state corresponding to the action performed by the user holding the remote control device meets the state 1. If it meets, the cached timestamp corresponding to the state 1 is updated, and at the same time, it is determined whether the ending action state corresponding to the action performed by the user holding the remote control device meets the state 1. If it meets, it is determined whether there is a cached timestamp corresponding to the state 1. If it exists, it is determined whether the time interval between the timestamp corresponding to the state 1 and the current timestamp corresponding to the state 2 is less than 0.4 seconds. If it is not less than 0.4 seconds, the cached timestamp corresponding to the state 1 is cleared; if it is less than 0.4 seconds, it is further determined whether the absolute value of the remote control device on the x-axis is greater than sin30° during the transition from state 1 to state 2, that is, whether the angle change of gravity on the x-axis is less than 30 degrees or whether the angle change of the remote control device on the y-axis and z-axis is less than 30 degrees. If this condition is met, it is determined that the action performed by the user holding the remote control device is a hook action, and the cached timestamp corresponding to the state 1 is cleared.

[0085] The state details table of the first preset starting state (state 1) and the first preset ending state (state 2) is shown in Table 2:

[0086] Table 2

[0087]

[0088] Step S22: When the action performed by the user while holding the remote control device meets the preset hook action determination condition, it is determined that the target action performed by the user while holding the remote control device is a hook action.

[0089] Step S23: When the target action is the hook action, the remote control device is controlled to trigger a hook action instruction corresponding to the hook action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the hook action instruction.

[0090] For the specific contents of the above-mentioned step S22 and step S23, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be repeated here.

[0091] It can be seen that in the embodiments of the present invention, by making the motion control mode compatible with the numerical control mode of the remote control device, the compatibility of the motion control mode and the numerical control mode is achieved, thereby being able to expand more interactive operations, and in the motion control mode, the corresponding hook action instruction is triggered by the hook action performed by the user when holding the remote control device, and the hook action instruction corresponds to the corresponding control command, thereby achieving a richer interactive experience and increasing the interactive method and interactive fun based on motion control achieved by the execution of the target action when the user holds the remote control device.

[0092] See also Figure 7 As shown, the embodiment of the present invention discloses a specific remote control interaction method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution.

[0093] Step S31: when it is detected that the control mode on the remote control device is the motion control mode, it is determined whether the action performed by the user holding the remote control device meets the preset swing action determination condition according to the three-dimensional motion data collected by the built-in motion sensor of the remote control device.

[0094] In this embodiment, according to the predefined swinging action standard (i.e., the preset swinging action determination condition), the action performed by the user when holding the remote control device is identified and judged by acquiring the data of the motion sensor to determine whether the action performed by the user when holding the remote control device is a swinging action.

[0095] See above Figure 4 As shown, the specific contents about the x-axis, y-axis and z-axis can refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0096] It should be pointed out that the preset swing action judgment condition is: the starting action state and the ending action state corresponding to the action performed when the user holds the remote control device satisfy the second preset starting state and the second preset ending state, and the time interval for the action performed when the user holds the remote control device to transition from the starting action state to the ending action state is less than the second preset time interval, and in the process of transitioning from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle change between the gravity direction and the x-axis direction is less than the second preset angle value.

[0097] Furthermore, the second preset starting state is: the angle between the x-axis direction of the remote control device and the gravity direction meets the angle range of the fourth preset starting state, the angle between the y-axis direction and the gravity direction meets the angle range of the fifth preset starting state, and the angle between the z-axis direction and the gravity direction meets the angle range of the sixth preset starting state; the second preset ending state is: the angle between the x-axis direction of the remote control device and the gravity direction meets the angle range of the fourth preset ending state, the angle between the y-axis direction and the gravity direction meets the angle range of the fifth preset ending state, and the angle between the z-axis direction and the gravity direction meets the angle range of the sixth preset ending state. For example, the angle range of the fourth preset starting state may be 60 degrees to 120 degrees, the angle range of the fifth preset starting state may be 120 degrees to 160 degrees, the angle range of the sixth preset starting state may be 30 degrees to 70 degrees, the angle range of the fourth preset ending state may be 60 degrees to 120 degrees, the angle range of the fifth preset ending state may be 120 degrees to 160 degrees, and the angle range of the sixth preset ending state may be 120 degrees to 90 degrees.

[0098] For example, see Figure 8 As shown, when the user holds the remote control device and performs a standard swinging action, the remote control device will transition from state 3 to state 1 in a short time, such as transitioning from state 3 to state 1 within 0.4 seconds, and during the transition from state 3 to state 1, the main rotation axis is the x-axis, the gravity value of the remote control device on the y-axis is continuously less than 0, and the angle change between the gravity direction and the x-axis direction is less than the first preset angle value, such as the gravity angle change on the x-axis is less than 30 degrees, that is, the second preset angle value can be set to 30 degrees, that is, the swinging action standard is as shown in Table 3:

[0099] Table 3

[0100]

[0101]

[0102] The state details table of the second preset starting state (state 3) and the second preset ending state (state 1) is shown in Table 4:

[0103] Table 4

[0104]

[0105] Step S32: When the action performed by the user while holding the remote control device meets the preset swing action determination condition, it is determined that the target action performed by the user while holding the remote control device is a swing action.

[0106] Step S33: When the target action is the swing action, the remote control device is controlled to trigger a swing action instruction corresponding to the swing action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the swing action instruction.

[0107] For the specific contents of the above-mentioned step S32 and step S33, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0108] It can be seen that in the embodiments of the present invention, by making the motion control mode compatible with the numerical control mode of the remote control device, the compatibility of the motion control mode and the numerical control mode is achieved, thereby being able to expand more interactive operations, and in the motion control mode, the corresponding swinging action instruction is triggered by the swinging action performed by the user when holding the remote control device, and the swinging action instruction corresponds to the corresponding control command, thereby achieving a richer interactive experience and increasing the interactive method and interactive fun based on motion control achieved by the execution of the target action when the user holds the remote control device.

[0109] See also Fig. 9 As shown, the embodiment of the present invention discloses a specific remote control interaction method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution.

[0110] Step S41: when it is detected that the control mode on the remote control device is the motion control mode, it is determined whether the action performed by the user holding the remote control device meets the preset rotation action determination condition according to the three-dimensional motion data collected by the built-in motion sensor of the remote control device.

[0111] In this embodiment, according to the predefined rotation action standard (i.e., the preset rotation action judgment condition), the action performed by the user when holding the remote control device is identified and judged by acquiring the data of the motion sensor to determine whether the action performed by the user when holding the remote control device is a rotation action.

[0112] See above Figure 4 As shown, the specific contents about the x-axis, y-axis and z-axis can refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0113] It should be noted that the preset rotation action determination conditions include a left rotation action determination condition, a right rotation action determination condition and a back-and-forth rotation action determination condition; and the left rotation action determination condition is: when the user holds the remote control device to perform an action, the angle of the remote control device rotating counterclockwise around the y-axis exceeds the third preset angle value, and the angle of rotation around the x-axis and the z-axis does not exceed the fourth preset angle value, and the rotation time during the rotation around the axis is less than the first preset rotation time interval; the right rotation action determination condition is: when the user holds the remote control device to perform an action, the angle of the remote control device rotating clockwise around the y-axis exceeds the fifth preset angle value, and the angle of rotation around the x-axis and the z-axis does not exceed the sixth preset angle value, and the rotation time during the rotation around the axis is less than the second preset rotation time interval; the back-and-forth rotation action determination condition is: when the user holds the remote control device to perform an action, the time interval from performing a left rotation action to a right rotation action is less than the third preset rotation time interval, or the time interval from a right rotation action to a left rotation action is less than the third preset rotation time interval. Among them, the third preset angle value can be 45 degrees, the fourth preset angle value can be 30 degrees, the first preset rotation time interval can be 0.5 seconds, among them, the fifth preset angle value can be 45 degrees, the sixth preset angle value can be 30 degrees, the second preset rotation time interval can be 0.5 seconds, and the third preset rotation time interval can be 0.8 seconds.

[0114] For example, see Fig.10 As shown, when the user holds the remote control device to perform a standard left-hand rotation, the angle of the remote control device's counterclockwise rotation around the y-axis exceeds 45 degrees, and the angle of rotation around the x-axis and z-axis does not exceed 30 degrees, and the rotation time during the rotation around the axis is less than 0.5 seconds, that is, the gravity value of the remote control device on the x-axis decreases by more than sin(45*π / 180) in a short period of time (within 0.5 seconds) and during the left-hand rotation, the gravity value change on the y and z axes is less than sin(30*π / 180); when the user holds the remote control device to perform a standard right-hand rotation, The angle of clockwise rotation of the remote control device around the y-axis exceeds 45 degrees, and the angle of rotation around the x-axis and z-axis does not exceed 30 degrees, and the rotation time during the rotation around the axis is less than 0.5 seconds, that is, the gravity value of the remote control device on the x-axis increases by more than sin(45*π / 180) in a short period of time (within 0.5 seconds) and during the right-hand rotation, the gravity value change on the y and z axes is less than sin(30*π / 180); when the user holds the remote control device to perform a back-and-forth rotation, at least one left rotation and one right rotation are triggered within 0.8 seconds. Accordingly, the rotation action standards are shown in Table 5:

[0115] Table 5

[0116]

[0117]

[0118] Step S42: When the action performed by the user while holding the remote control device meets the preset rotation action determination condition, it is determined that the target action performed by the user while holding the remote control device is a rotation action.

[0119] Step S43: When the target action is the rotation action, the remote control device is controlled to trigger a rotation action instruction corresponding to the rotation action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the rotation action instruction.

[0120] For the specific contents of the above steps S42 to S43, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be repeated here.

[0121] It can be seen that in the embodiments of the present invention, by making the motion control mode compatible with the numerical control mode of the remote control device, the compatibility of the motion control mode and the numerical control mode is achieved, thereby being able to expand more interactive operations, and in the motion control mode, the corresponding rotation action instruction is triggered by the rotation action performed by the user when holding the remote control device, and the rotation action instruction corresponds to the corresponding control command, thereby achieving a richer interactive experience and increasing the interaction method and interaction fun based on motion control achieved by the execution of the target action when the user holds the remote control device.

[0122] In one embodiment, if Fig.11 As shown, based on the above remote control interaction method, the present invention also provides a remote control interaction device, including:

[0123] The action determination module 100 is used to determine the target action performed by the user holding the remote control device when the control mode on the remote control device is detected to be the action control mode based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device;

[0124] The instruction triggering module 200 is used to determine whether to trigger a corresponding action instruction based on the target action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction.

[0125] Fig.12 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0126] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0127] When the processor 502 executes the program, the remote control interaction method provided in the above embodiment is implemented.

[0128] Furthermore, the terminal further includes:

[0129] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0130] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0131] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0132] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0133] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0134] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0135] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the remote control interaction method as described above is implemented.

[0136] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0137] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0138] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, in which the order shown or discussed may not be followed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can read instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary and then storing it in a computer memory.

[0140] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0141] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0143] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0144] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A remote control interaction method, characterized in that: The method comprises: When it is detected that the control mode on the remote control device is the motion control mode, motion determination is performed based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device to determine the target motion performed by the user when holding the remote control device; Based on the target action, it is determined whether to trigger a corresponding action instruction, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction.

2. The remote control interaction method according to claim 1, characterized in that: The performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes: Determine whether the action performed by the user holding the remote control device meets a preset hook action determination condition according to the three-dimensional motion data collected by the built-in motion sensor on the remote control device; When the action performed by the user holding the remote control device meets the preset hook action determination condition, it is determined that the target action performed by the user holding the remote control device is a hook action; Among them, the preset hook action determination condition is: The starting action state and the ending action state corresponding to the action performed by the user holding the remote control device satisfy the first preset starting state and the first preset ending state, and the time interval of the action performed by the user holding the remote control device during the transition from the starting action state to the ending action state is less than the first preset time interval, and in the process of the transition from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle between the gravity direction and the x-axis direction changes less than the first preset angle value; Furthermore, the first preset starting state is: The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the first preset starting state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the second preset starting state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the third preset starting state; The first preset end state is: The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the first preset end state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the second preset end state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the third preset end state; Wherein, a three-dimensional rectangular coordinate system consisting of an x-axis, a y-axis and a z-axis is established with the center of the remote control device as the origin; The x-axis is a coordinate axis that points from the left side to the right side when the remote control device is placed horizontally with the front side facing upward and the back side facing downward. The y-axis is a coordinate axis that points from the rear end to the front end when the remote control device is placed horizontally with the front side facing upward and the back side facing downward. The z-axis is a coordinate axis that is perpendicular to the x-axis and the y-axis and points from the back end to the front end. Furthermore, the sum of the gravity on the x-axis, y-axis and z-axis is 1, and the gravity direction of the remote control device is vertically downward.

3. The remote control interaction method according to claim 2, characterized in that: The determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes: When the target action is the hook action, the remote control device is controlled to trigger a hook action instruction corresponding to the hook action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the hook action instruction.

4. The remote control interaction method according to claim 3, characterized in that: The performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes: Determining whether the action performed by the user holding the remote control device meets a preset swing action determination condition based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device; When the action performed by the user while holding the remote control device meets the preset swing action determination condition, it is determined that the target action performed by the user while holding the remote control device is a swing action; Wherein, the preset swing action determination condition is: The starting action state and the ending action state corresponding to the action performed by the user holding the remote control device satisfy the second preset starting state and the second preset ending state, and the time interval between the transition from the starting action state to the ending action state of the action performed by the user holding the remote control device is less than the second preset time interval, and in the process of the transition from the starting action state to the ending action state, with the x-axis as the main rotation axis, the gravity value of the remote control device on the y-axis is continuously less than zero and the angle between the gravity direction and the x-axis direction changes less than the second preset angle value; Furthermore, the second preset starting state is: The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the fourth preset starting state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the fifth preset starting state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the sixth preset starting state; The second preset end state is: The angle between the x-axis direction of the remote control device and the gravity direction satisfies the angle range of the fourth preset end state, the angle between the y-axis direction and the gravity direction satisfies the angle range of the fifth preset end state, and the angle between the z-axis direction and the gravity direction satisfies the angle range of the sixth preset end state.

5. The remote control interaction method according to claim 4, characterized in that: The determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes: When the target action is the swing action, the remote control device is controlled to trigger a swing action instruction corresponding to the swing action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the swing action instruction.

6. The remote control interaction method according to any one of claims 2 to 5, characterized in that: The performing action determination based on the three-dimensional motion data collected by the built-in motion sensor on the remote control device to determine the target action performed by the user when holding the remote control device includes: Determine whether the action performed by the user holding the remote control device meets a preset rotation action determination condition according to the three-dimensional motion data collected by the built-in motion sensor on the remote control device; When the action performed by the user holding the remote control device meets the preset rotation action determination condition, it is determined that the target action performed by the user holding the remote control device is a rotation action; The preset rotation action determination conditions include a left rotation action determination condition, a right rotation action determination condition, and a back-and-forth rotation action determination condition; Furthermore, the left-hand rotation action determination condition is: When the user holds the remote control device to perform an action, the angle of the remote control device rotating counterclockwise around the y-axis exceeds a third preset angle value, and the angle of the remote control device rotating around the x-axis and the z-axis does not exceed a fourth preset angle value, and the rotation time during the rotation around the axis is less than the first preset rotation time interval; The right-hand action determination condition is: When the user holds the remote control device to perform an action, the angle of the clockwise rotation of the remote control device around the y-axis exceeds the fifth preset angle value, and the angle of rotation around the x-axis and the z-axis does not exceed the sixth preset angle value, and the rotation time during the rotation around the axis is less than the second preset rotation time interval; The back and forth rotation action determination condition is: When the user holds the remote control device to perform an action, the time interval from the left rotation action to the right rotation action is less than the third preset rotation time interval, or the time interval from the right rotation action to the left rotation action is less than the third preset rotation time interval.

7. The remote control interaction method according to claim 6, characterized in that: The determining whether to trigger a corresponding action instruction based on the target action so that the remote control device interacts with the to-be-interacted device based on a control command corresponding to the action instruction includes: When the target action is the rotation action, the remote control device is controlled to trigger a rotation action instruction corresponding to the rotation action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the rotation action instruction.

8. A remote control interactive device, characterized in that: The device comprises: An action determination module, for determining the target action performed by the user holding the remote control device when the control mode of the remote control device is detected to be the action control mode, based on the three-dimensional motion data collected by the built-in motion sensor of the remote control device; The instruction triggering module is used to determine whether to trigger a corresponding action instruction based on the target action, so that the remote control device interacts with the device to be interacted based on the control command corresponding to the action instruction.

9. A terminal, characterized in that: include: A memory, a processor, and a remote control interaction program stored in the memory and executable on the processor, wherein the remote control interaction program, when executed by the processor, implements the steps of the remote control interaction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the remote control interaction method according to any one of claims 1 to 7.

Images