Interface interaction control method and device, wearable equipment and storage medium

Abstract

The invention provides an interface interaction control method and device, wearable equipment and a computer readable storage medium. The interface interaction control method comprises the steps that a virtual scene picture is displayed, and the virtual scene picture comprises a cursor interaction interface and a virtual ray of cursor control equipment; based on a current ray interaction point, a cursor of the cursor control equipment is displayed on the cursor interaction interface, and the current ray interaction point is an intersection point between the virtual ray located at the current display position and the cursor interaction interface; and when the cursor control equipment moves, controlling the cursor to move from the current ray interaction point to a target ray interaction point. According to the method, the interface interaction intuition can be improved, and the interface interaction operation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of interface interaction processing, and specifically to an interface interaction control method, apparatus, wearable device and computer-readable storage medium. Background Art

[0002] In recent years, wearable devices have been loved by the general public. In order to better meet people's interactive needs, the cursor position in the screen of wearable devices such as smart glasses can be controlled by moving a cursor control device (such as a ring). However, in the related art, it is difficult for users to perceive the changing rules of the cursor position, the intuitiveness of the interface interaction is low, and therefore the efficiency of the interface interaction operation is relatively low. Summary of the invention

[0003] The present application provides an interface interaction control method, apparatus, wearable device and computer-readable storage medium, which can improve the intuitiveness of interface interaction and improve the efficiency of interface interaction operations.

[0004] In a first aspect, the present application provides an interface interaction control method, the method comprising:

[0005] Displaying a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0006] Based on a current ray interaction point, displaying a cursor of the cursor control device on the cursor interaction interface, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface;

[0007] When the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point.

[0008] In a second aspect, the present application provides an interface interaction control device, the interface interaction control device comprising:

[0009] A display unit, used to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0010] A control unit, configured to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface;

[0011] The control unit is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

[0012] In a third aspect, the present application also provides a wearable device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, any one of the interface interaction control methods provided in the present application is executed.

[0013] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the interface interaction control method.

[0014] In the present application, a virtual scene screen including a cursor of a cursor control device is displayed, and the cursor of the cursor control device is displayed on the cursor interaction interface based on the current ray interaction point; when the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point, so that the cursor can be presented at the intersection between the virtual ray and the cursor interaction interface, so that the user can perceive that the virtual ray moves with the cursor control device and the cursor position changes with the ray interaction point, and then the user can perceive the changing pattern of the cursor position, thereby improving the intuitiveness of the interface interaction and the efficiency of the interface interaction operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 is a schematic block diagram of the structure of a wearable device provided in an embodiment of the present application;

[0017] Figure 2 This is a flow chart of the interface interaction control method provided in the embodiment of the present application;

[0018] Figure 3 is a schematic diagram of a virtual scene screen provided in an embodiment of the present application;

[0019] Figure 4 is another schematic diagram of a virtual scene screen provided in an embodiment of the present application;

[0020] Figure 5 is a schematic diagram for explaining a ray reference coordinate system provided in an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of a scene of a cursor movement process provided in an embodiment of the present application;

[0022] Figure 7It is a schematic diagram illustrating the state of the target interactive object when the ray interaction point is at different positions provided in the embodiment of the present application;

[0023] Figure 8 This is a schematic diagram illustrating that the interface interaction cursor is automatically adsorbed onto the interactive object provided in an embodiment of the present application;

[0024] Fig. 9 It is a schematic diagram of the structure of an embodiment of the interface interaction control device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0028] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0029] The embodiments of the present application provide an interface interaction control method, device, wearable device, and computer-readable storage medium. The interface interaction control device can be integrated in a wearable device. The wearable device can be smart glasses, smart helmets, etc. The smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, etc. The smart helmet can be an AR helmet, etc.

[0030] The executor of the interface interaction control method of the embodiment of the present application may be the interface interaction control device provided in the embodiment of the present application, or a wearable device integrating the interface interaction control device, wherein the interface interaction control device may be implemented in hardware or software.

[0031] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0032] Figure 1 It is a schematic block diagram of the structure of a wearable device provided in an embodiment of the present application.

[0033] like Figure 1 As shown, the wearable device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0034] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire wearable device 100. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0035] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0036] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment scheme of the present application, and does not constitute a limitation on the wearable device to which the embodiment scheme of the present application is applied. The specific wearable device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0037] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the interface interaction control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0038] Display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; based on a current ray interaction point, display a cursor of the cursor control device on the cursor interaction interface, wherein the current ray interaction point is an intersection between the virtual ray at a current display position and the cursor interaction interface; when the cursor control device moves, control the cursor to move from the current ray interaction point to a target ray interaction point.

[0039] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0040] When the cursor control device moves, a first moving angle and a first moving direction of the cursor control device are obtained; based on the first moving angle and a preset angle relationship, a second moving angle of a virtual ray of the cursor control device is determined; based on the first moving direction and a preset direction relationship, a second moving direction of the virtual ray is determined; based on the second moving angle and the second moving direction, the movement of the virtual ray is controlled.

[0041] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0042] The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is an intersection point between the virtual ray at the moved position and the cursor interaction interface.

[0043] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0044] According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second moving angle.

[0045] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0046] When a reset instruction of the cursor control device is triggered, the cursor is moved to a preset initial position of the cursor interaction interface, wherein when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0047] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0048] Determine the target interaction object on the cursor interaction interface that is closest to the target ray interaction point; if the distance between the target ray interaction point and the target interaction object is less than a preset distance threshold, control the cursor to move to the target interaction object, wherein the target interaction object includes at least one of a control, text, and a function area.

[0049] In some embodiments, the target interaction object is a function area, the function area includes multiple sub-function areas, and the processor 101 is used to run a computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0050] From the sub-functional areas of the functional area, determine the target sub-functional area that is closest to the target ray interaction point; and control the cursor to move to the target sub-functional area.

[0051] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0052] When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stop position of the cursor interaction interface.

[0053] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0054] When the ray interaction point is not on the interactive object of the cursor interaction interface, the interactive object is displayed in a first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface for a time period greater than a preset hovering time period, the interactive object is displayed in a second display state, wherein in the second display state, at least one of the size, color, and style of the interactive object is different from that in the first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein in the third display state, at least one of the size, color, and style of the interactive object is different from that in the second display state.

[0055] It should be noted that technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the wearable device described above can refer to the corresponding process in the following interface interaction control method embodiment, and will not be repeated here.

[0056] In the following, Figure 1 The wearable device shown in is used as an example to execute the interface interaction control method. The interface interaction control method provided by the embodiment of the present application is described in detail. For the sake of simplicity and convenience of description, the execution subject will be omitted in the subsequent method embodiments. Figure 1 The scenarios are only used to explain the interface interaction control method provided in the embodiments of the present application, but do not constitute a limitation on the application scenarios of the interface interaction control method provided in the embodiments of the present application.

[0057] See also Figure 2 , Figure 2 201 to 203, wherein:

[0058] 201. Display a virtual scene screen.

[0059] The virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device.

[0060] Among them, the cursor interaction interface is a user interface that can be interacted with by a cursor. In some embodiments, the cursor interaction interface can be the entire virtual scene screen; for example, Figure 3 As shown, Figure 3 The rectangular area in the middle represents the virtual scene screen displayed by the AR glasses. The entire virtual scene screen can be interacted with by the cursor, so the entire virtual scene screen can be used as the cursor interaction interface. In other embodiments, the cursor interaction interface can be a part of the virtual scene screen; for example, Figure 4As shown, Figure 4 A schematic diagram of a virtual scene screen. Figure 4 Only the dotted rectangular frame area can be interacted with using the cursor, so the dotted rectangular frame area in the virtual scene screen is used as the cursor interaction interface.

[0061] The cursor control device is a device that generates a cursor on a cursor interaction interface for interaction. For example, the cursor control device may be a finger ring, a watch, or the like.

[0062] For example, a virtual scene screen can be displayed by a wearable device, and the virtual scene screen can be a game screen, a video screen, etc., and the specific screen content of the virtual scene screen is not limited in this embodiment. For example, a game screen can be displayed by AR glasses.

[0063] Among them, the wearable device can be smart glasses, smart helmets, etc., the smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, etc., and the smart helmet can be an AR helmet, etc.

[0064] 202. Display a cursor of the cursor control device on the cursor interaction interface based on the current ray interaction point.

[0065] In order to better understand the embodiments of the present application, some names involved in the embodiments are introduced below:

[0066] 1. Ray starting point: the starting point of the virtual ray, which can be any position point outside the plane where the cursor interaction interface is located. In this embodiment, in order to more accurately simulate the movement of the cursor, the ray starting point is a preset fixed position point. For example, Figure 3 As shown, Figure 3 The point O in the middle indicates the starting point of the ray.

[0067] 2. Virtual ray: Any ray starting from the ray starting point and pointing to any position point on the cursor interaction interface is a virtual ray of the cursor control device. Figure 3 As shown, the ray OA intersecting the cursor interaction interface at point A can be used as a virtual ray, and the ray OB intersecting the cursor interaction interface at point B can also be used as a virtual ray. In this embodiment, the movement of the virtual ray is controlled according to the movement of the cursor control device, thereby changing the position of the intersection between the virtual ray and the cursor interaction interface to achieve control of the cursor.

[0068] 3. Ray interaction point: the intersection point between the virtual ray and the cursor interaction interface. In this embodiment, the display position of the cursor on the cursor interaction interface is the location of the ray interaction point. Therefore, the display position of the cursor on the cursor interaction interface changes with the change of the ray interaction point. The movement of the cursor can be controlled by controlling the movement of the virtual ray. For example, Figure 3 As shown, due to Figure 3 The intersection of ray OA and the cursor interface is point A, so when the virtual ray is at the position of ray OA, the cursor is at point A; Figure 3 The intersection point of the middle ray OB and the cursor interaction interface is point B, so when the virtual ray is at the position of ray OB, the cursor position point is point B. Therefore, the cursor position point can be continuously adjusted according to the ray interaction point (i.e., the intersection point between the virtual ray and the cursor interaction interface).

[0069] 4. The preset initial position of the cursor: that is, the default position of the cursor in the cursor interaction interface.

[0070] 5. Default position of virtual ray: the position of virtual ray when the cursor is at the preset initial position. Figure 3 As shown, assuming that the preset initial position of the cursor is the center point P of the cursor interaction interface, the default position of the virtual ray can be set to the position of the ray from the ray starting point O to the center point P of the cursor interaction interface.

[0071] The current ray interaction point is the intersection point between the virtual ray at the current display position and the cursor interaction interface.

[0072] 203. When the cursor control device moves, control the cursor to move from the current ray interaction point to a target ray interaction point.

[0073] The virtual ray moves based on the movement data of the cursor control device, and the target ray interaction point is the intersection point between the virtual ray at the moved position and the cursor interaction interface. Exemplarily, when the cursor control device moves, the virtual ray is controlled to move with the cursor control device so that the intersection point between the virtual ray and the cursor interaction interface changes accordingly, thereby controlling the cursor to move from the current ray interaction point to the target ray interaction point.

[0074] The interface interaction control method may further include: controlling the movement of the virtual ray based on the movement data of the cursor control device. Exemplarily, there is a preset angle relationship between the movement angle of the virtual ray and the movement angle of the cursor control device, and there is a preset direction relationship between the movement direction of the virtual ray and the movement direction of the cursor control device. At this time, the movement of the virtual ray may be controlled by referring to the following steps A1 to A4:

[0075] A1. When the cursor control device moves, obtain a first moving angle and a first moving direction of the cursor control device.

[0076] The first movement angle refers to the movement angle of the cursor control device. For example, taking the cursor control device as a finger ring, if the finger of the user wearing the finger ring rotates 1° to the left, the first movement angle of the finger ring is 1°.

[0077] The first moving direction refers to the moving direction of the cursor control device. For example, taking the cursor control device as a finger ring, if the finger of the user wearing the finger ring rotates 1° to the left, the first moving direction of the finger ring is to the left.

[0078] Here, "to the left" is an example made for the convenience of understanding. It can be understood that an inertial measurement unit (IMU) can be built into the cursor control device to measure and detect whether the cursor control device moves. When the cursor control device moves, the three-axis angular velocity and three-axis acceleration of the cursor control device are measured by the IMU built into the cursor control device, so as to calculate the first movement angle and the first movement direction of the cursor control device. Specifically, the first movement angle and the first movement direction of the cursor control device can be represented by the IMU sensor coordinate system built into the cursor control device. For example, in the IMU sensor coordinate system built into the cursor control device, the first movement angle and the first movement direction of the cursor control device are represented by the rotation angle of the cursor control device relative to the x-axis, y-axis, and z-axis.

[0079] A2. Determine a second moving angle of the virtual ray of the cursor control device based on the first moving angle and a preset angle relationship.

[0080] The virtual ray is used to reflect the movement of the cursor control device. The position of the virtual ray changes as the cursor control device moves, thereby changing the position of the intersection between the virtual ray and the cursor interaction interface to achieve control of the cursor.

[0081] The second moving angle refers to the moving angle of the virtual ray. For example, if the virtual ray rotates 1 degree to the left, the second moving angle of the virtual ray is 1 degree.

[0082] Among them, a coordinate system can be established by a preset ray reference coordinate system (such as a coordinate system with the ray starting point as the coordinate origin, a horizontal straight line parallel to the cursor interaction interface as the x-axis direction, a horizontal straight line perpendicular to the cursor interaction interface as the y-axis direction, and a vertical straight line parallel to the cursor interaction interface as the z-axis direction, such as Figure 5 ) to indicate the moving angle and moving direction of the virtual ray. Figure 5As shown, in a preset ray reference coordinate system (wherein the origin of the ray reference coordinate system is the ray starting point O, and the yoz plane of the ray reference coordinate system is parallel to the plane where the cursor interaction interface is located), the second movement angle and second movement direction of the cursor are represented by the rotation angle of the virtual ray relative to the x-axis, y-axis, and z-axis.

[0083] The preset angle relationship is used to indicate the relationship between the first moving angle of the cursor control device and the second moving angle of the virtual ray. The preset angle relationship can be set according to the actual business scenario requirements. The specific value relationship of the preset angle relationship is not limited here. For example, the preset angle relationship can be: the first moving angle rotates 1 degree, and the second moving angle rotates 3 degrees.

[0084] A3. Determine a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship.

[0085] The preset direction relationship is used to indicate the relationship between the first moving direction of the cursor control device and the second moving direction of the virtual ray. The preset direction relationship can be set according to the actual business scenario requirements. There is no restriction on the setting of the preset direction relationship here. For example, the preset direction relationship can be: when the first moving direction is the clockwise direction of the x-axis relative to the sensor coordinate system, the second moving direction is the clockwise direction of the x-axis relative to the ray reference coordinate system.

[0086] For example, the preset direction relationship indicates: when the cursor control device rotates counterclockwise relative to the x-axis of the sensor coordinate system, the virtual ray rotates counterclockwise relative to the x-axis of the ray reference coordinate system; when the cursor control device rotates clockwise relative to the x-axis of the sensor coordinate system, the virtual ray rotates clockwise relative to the x-axis of the ray reference coordinate system; when the cursor control device rotates counterclockwise relative to the y-axis of the sensor coordinate system, the virtual ray rotates counterclockwise relative to the y-axis of the ray reference coordinate system; when the cursor control device rotates clockwise relative to the y-axis of the sensor coordinate system, the virtual ray rotates clockwise relative to the y-axis of the ray reference coordinate system. Therefore, when the first moving direction is counterclockwise relative to the x-axis of the sensor coordinate system, the counterclockwise direction relative to the x-axis of the ray reference coordinate system is used as the second moving direction of the cursor.

[0087] A4. Control the movement of the virtual ray based on the second movement angle and the second movement direction.

[0088] Specifically, according to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray is the second moving angle. Figure 5, the second moving direction is "counterclockwise relative to the x-axis of the ray reference coordinate system", and the second moving angle is "3 degrees", then the virtual ray is controlled to rotate counterclockwise from the current display position relative to the x-axis of the ray reference coordinate system until the angle between the moved position of the virtual ray and the current display position of the virtual ray is 3 degrees.

[0089] In order to better understand how to complete interface interaction through virtual rays, a specific example is given below. Figure 6 ( Figure 6 The middle ring represents the cursor), for example, taking the cursor control device as a ring and the cursor interaction interface as the display interface of the AR glasses as an example, when the user rotates the ring-wearing finger 1 degree to the left, step A1 detects that the ring rotates 1 degree counterclockwise relative to the x-axis of the sensor coordinate system (that is, the first movement angle is 1 degree, and the first movement direction is the counterclockwise direction of the x-axis relative to the sensor coordinate system). Through steps A2 to A3, it can be determined that the second movement angle is 3 degrees, and the second movement direction is the counterclockwise direction of the x-axis relative to the ray reference coordinate system. Then, the virtual ray can be rotated 3 degrees counterclockwise from the current display position toward the x-axis relative to the ray reference coordinate system, and then the cursor is moved from the current ray interaction point (such as the intersection P1 between the virtual ray at the current display position and the cursor interaction interface) to the target ray interaction point (such as the intersection P2 between the virtual ray at the moved position and the cursor interaction interface); thereby, when the user rotates the ring-wearing finger 1 degree to the left, the cursor moves 3 degrees on the display interface of the AR glasses. At this time, if the user presses the confirmation button on the ring, the "confirmation" interaction process will be completed.

[0090] Therefore, by determining the second moving angle of the virtual ray according to the first moving angle of the cursor control device and the preset angle relationship, the cursor movement is controlled, so that when the moving angle of the cursor control device is relatively small, the moving angle of the virtual ray can be larger, thereby increasing the movement amplitude of the cursor, thereby reducing the movement amplitude of the cursor control device and reducing the movement amplitude of the cursor control device by the user (for example, taking the cursor control device as a finger ring as an example, in order to reduce the movement amplitude of the wrist or arm), thereby reducing the discomfort of controlling the cursor control device and improving the interactive convenience of the cursor control device.

[0091] It can be seen that, on the first hand, by displaying a virtual scene screen including the cursor of the cursor control device, when the reset command of the cursor control device is triggered, the cursor of the cursor control device is moved to the preset initial position of the cursor interaction interface; thus, when the cursor is in a position that is not conducive to user movement, the cursor is quickly restored to the default position through the reset command (for example, when the user holds the cursor control device and places it on the thigh, the cursor is at the bottom of the cursor interaction interface, and it is originally necessary to raise the arm to raise the cursor control device to raise the cursor to the center point of the interface. The reset command can directly restore the cursor to the center point of the interface), so that there is no need to move the arm and wrist to a specified height or angle to achieve cursor movement, avoiding the problem that the user needs to move the cursor for a long time and in a large range to achieve cursor movement, thereby improving the convenience of interactive operations. On the second hand, when the cursor control device moves, controlling the cursor to move from the preset initial position to other positions of the cursor interaction interface can enable the user to move the cursor control device in a relatively comfortable position to move the cursor for interaction, thereby improving the convenience of interactive operations of the cursor.

[0092] Furthermore, if Figure 5 As shown, in order to improve the intuitiveness of the interaction and the visibility of the interactive operation, the virtual scene screen also includes a virtual wearable object of the cursor control device, the direction of the virtual ray is the same as the pointing direction of the virtual wearable object, the starting point of the virtual ray (i.e., the ray starting point) is the same as the position of the virtual wearable object corresponding to the cursor control device, and the intersection between the virtual ray and the cursor interaction interface is displayed as the cursor; when the pointing direction of the actual wearable object of the cursor control device (such as when the cursor control device is a ring, the pointing direction of the user's finger wearing the ring) moves, the direction of the virtual ray moves accordingly, thereby visually providing the user with an intuitive interactive feeling and improving the visibility of the interactive operation, so that the user can clearly understand the current interactive behavior.

[0093] Furthermore, in order to improve the convenience of interactive operations of the cursor, the interface interaction control method may further include: when a reset instruction of the cursor control device is triggered, moving the cursor to a preset initial position of the cursor interaction interface.

[0094] The preset initial position is the default position of the cursor of the cursor control device in the cursor interaction interface. For example, the preset initial position may be the center point of the cursor interaction interface.

[0095] When the cursor is at a preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0096] The reset instruction is used to control the cursor of the cursor control device to return to a preset initial position. There are many ways to trigger the reset instruction, for example, when the reset button of the cursor control device is pressed, the reset instruction of the cursor control device is triggered, and when the user gesture is detected to be the same as the preset gesture, the reset instruction of the cursor control device is triggered.

[0097] For example, Figure 3 As shown, assuming that the preset initial position of the cursor is the center point P of the cursor interaction interface, before the reset instruction of the cursor control device is triggered, the position point of the cursor is point A of the cursor interaction interface; when the reset instruction of the cursor control device is triggered, the position point of the cursor is restored to the preset initial position of the cursor interaction interface (i.e., the center point P of the cursor interaction interface).

[0098] There are many ways to implement “triggering the reset instruction of the cursor control device”, which include, for example:

[0099] (1) When a user makes a preset gesture to reset the cursor position, a reset instruction of the cursor control device is triggered, and the cursor is reset to the preset initial position of the cursor interaction interface. In this case, "when the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface" may specifically include:

[0100] 2021A. Obtain user interaction images.

[0101] 2022A. Perform gesture recognition on the user interaction image to obtain the gesture of the user interaction image.

[0102] 2023A. When the gesture of the user interaction image is the same as the preset gesture, trigger a reset instruction of the cursor control device.

[0103] 2024A. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0104] (2) By setting a reset button on the cursor control device, when the user presses the reset button of the cursor control device, a reset instruction of the cursor control device is triggered, and the cursor is reset to the preset initial position of the cursor interaction interface. In this case, "when the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface" may specifically include:

[0105] 2021B. When a cursor reset request sent by the cursor control device is received, trigger a reset instruction of the cursor control device, wherein the cursor control device sends the cursor reset request when a touch operation of a reset button is detected.

[0106] 2022B. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0107] (3) Collecting user gestures through a cursor control device (such as a finger ring), triggering a reset instruction of the cursor control device when the user makes a preset gesture to reset the cursor position, and resetting the cursor back to the preset initial position of the cursor interaction interface. At this time, step 202 may specifically include:

[0108] 2021C. When a cursor reset request sent by the cursor control device is received, trigger a reset instruction of the cursor control device, wherein the cursor control device sends the cursor reset request when a preset gesture is detected.

[0109] 2022C. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0110] Furthermore, in order to improve the interaction efficiency, when the cursor corresponding to the virtual ray approaches an interactive object such as a clickable control, text, or function area, the cursor can be controlled to automatically adsorb to the interactive object. That is, the interface interaction control method can further include: determining the target interactive object on the cursor interaction interface that is closest to the target ray interaction point; if the distance between the target ray interaction point and the target interactive object on the cursor interaction interface is less than a preset distance threshold, controlling the cursor to move to the target interactive object.

[0111] The interactive object includes at least one of a control, text, and a function area.

[0112] The target interaction object refers to the interaction object that is closest to the target ray interaction point among the interaction objects on the cursor interaction interface.

[0113] In one embodiment, after the target interactive object is determined, the cursor can be controlled to be directly adsorbed onto the target interactive object.

[0114] In another embodiment, if Figure 8 As shown, taking the interactive object as a clickable control as an example, assuming that there are 4 clickable controls on the cursor interaction interface (such as Figure 8 From left to right, they are: control 1, control 2, control 3, control 4). The control (such as control 1) closest to the target ray interaction point is determined from the four clickable controls as the target interaction object. When the distance between the target ray interaction point and the clickable control 1 on the cursor interaction interface is less than a preset distance threshold, the cursor is controlled to be adsorbed onto the clickable control 1, thereby improving the efficiency of control selection.

[0115] For another example, taking the interactive object as text, assuming that there are 4 texts that can be selected and copied on the cursor interaction interface (such as: "Text 1", "Text 2", "Text 3", "Text 4"), the text closest to the target ray interaction point is determined from the 4 texts that can be selected and copied (such as "Text 1") as the target interactive object. When the distance between the target ray interaction point and "Text 1" on the cursor interaction interface is less than the preset distance threshold, the cursor is controlled to be adsorbed on "Text 1" (such as the characters or words of the text). This can improve the efficiency of text selection and make it easier to edit documents, select text or position the cursor.

[0116] For another example, taking the interactive object as a function area, assuming that there are 4 clickable function areas on the cursor interaction interface (such as: function area 1, function area 2, function area 3, and function area 4), the function area closest to the target ray interaction point (such as clickable function area 4) is determined from the 4 clickable function areas as the target interactive object. When the distance between the target ray interaction point and function area 4 on the cursor interaction interface is less than a preset distance threshold, the cursor is controlled to be adsorbed onto function area 4, thereby improving the selection efficiency of the function area. Furthermore, if the function area includes multiple sub-function areas, the target sub-function area closest to the target ray interaction point can be determined from the sub-function areas of the function area; and the cursor is controlled to move to the target sub-function area. Among them, the distance between the middle pixel point of the sub-function area and the target ray interaction point can be obtained as the distance between the sub-function area and the target ray interaction point.

[0117] Furthermore, in order to highlight the adsorption effect of the cursor, the target interactive object can be highlighted when the cursor is adsorbed on it. In addition, different highlighting methods can be used for different types of target interactive objects. For example, when the cursor is adsorbed to an icon or tab without a background, it can be highlighted with a highlight effect; when the cursor is adsorbed to a button or list with a background, it can be highlighted with a lifting effect; when the cursor is adsorbed to an application icon, it can be highlighted with a floating effect.

[0118] Furthermore, in order to avoid the cursor from being lost, when the user does not move the cursor for a period of time, the cursor can be automatically adsorbed to the preset stop position of the cursor interaction interface. That is, the interface interaction control method can further include: when the duration of the cursor being at the target ray interaction point is greater than the preset duration, controlling the cursor to move to the preset stop position of the cursor interaction interface. For example, taking the preset stop position as the edge of the cursor interaction interface as an example, when the cursor stays at the target ray interaction point for more than 20 seconds, the cursor is automatically adsorbed to the edge of the cursor interaction interface.

[0119] Furthermore, in order to improve the intuitiveness of the interactive display and let the user know the current interactive situation, when the intersection between the virtual ray and the cursor interactive interface is on an interactive control, the interactive control can also be highlighted (such as enlarged or changed in color). That is, the interface interaction control method can further include: if the target ray interaction point is on an interactive control, highlighting the interactive control on the cursor interactive interface. For example, when the intersection between the virtual ray and the cursor interactive interface falls on a button, the button is enlarged and displayed in a different color.

[0120] Further, the cursor control device is a wearable device, such as Figure 7 As shown, after wearing a cursor control device (such as a ring), the user can use the ray interaction point to interact with the interface. For example, when the ray interaction point is on the target that the user wants to select (such as a button, function area, text, etc.), the user can press or slide the ring touch bar to perform various interactive operations such as confirmation, return, scroll, etc.

[0121] Furthermore, in order to improve the interactive visibility of the ray interaction point, the states of the interactive object include an idle state, a hovering state, and a clicked state, and the display style of the interactive object is different in each state. That is, the interface interaction control method may further include: when the ray interaction point is not on the interactive object of the cursor interaction interface, the interactive object is displayed in a first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface for a time period longer than a preset hovering time period, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

[0122] For example, Figure 7 As shown, when the ray interaction point (i.e., the cursor) is not on the interactive object (such as a button), the interactive object is in an idle state, and the control is displayed according to the first display state; when the ray interaction point (i.e., the cursor) is on the interactive object (such as a button), but no operation instruction of the interactive object is issued (such as when "Confirm" is not pressed), the interactive object is in a hovering state, and the control is displayed according to the second display state; when the ray interaction point (i.e., the cursor) is on the interactive object (such as a button) and an operation instruction of the interactive object is issued (such as when the physical "Confirm" button is pressed on the ring), the interactive object is in a click state, and the control is displayed according to the third display state.

[0123] From the above content, it can be seen that in this embodiment, by displaying a virtual scene screen including the cursor of the cursor control device, the cursor of the cursor control device is displayed on the cursor interaction interface based on the current ray interaction point; when the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point, so that the cursor can be presented at the intersection between the virtual ray and the cursor interaction interface, so that the user can perceive that the virtual ray moves with the cursor control device and the cursor position changes with the ray interaction point, and then the user can perceive the changing pattern of the cursor position, thereby improving the intuitiveness of the interface interaction and the efficiency of the interface interaction operation.

[0124] In addition, in order to better implement the interface interaction control method in the embodiment of the present application, based on the interface interaction control method, an interface interaction control device is also provided in the embodiment of the present application, such as Fig. 9 FIG. 1 is a schematic diagram of a structure of an embodiment of an interface interaction control device provided in an embodiment of the present application. The interface interaction control device 900 includes:

[0125] Display unit 901, a display unit, used to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0126] A control unit 902 is used to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface;

[0127] The control unit 902 is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

[0128] In some embodiments, the control unit 902 is specifically configured to:

[0129] The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is the intersection point between the virtual ray at the moved position and the cursor interaction interface.

[0130] In some embodiments, the control unit 902 is specifically configured to:

[0131] When the cursor control device moves, obtaining a first moving angle and a first moving direction of the cursor control device;

[0132] Determining a second moving angle of a virtual ray of the cursor control device based on the first moving angle and a preset angle relationship;

[0133] Determining a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship;

[0134] The movement of the virtual ray is controlled based on the second movement angle and the second movement direction.

[0135] In some embodiments, the control unit 902 is specifically configured to:

[0136] According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second moving angle.

[0137] In some embodiments, the control unit 902 is specifically configured to:

[0138] When a reset instruction of the cursor control device is triggered, the cursor is moved to a preset initial position of the cursor interaction interface, wherein when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0139] In some embodiments, the control unit 902 is specifically configured to:

[0140] Determine the target interaction object on the cursor interaction interface that is closest to the target ray interaction point;

[0141] If the distance between the target ray interaction point and the target interaction object is less than a preset distance threshold, the cursor is controlled to move to the target interaction object, wherein the target interaction object includes at least one of a control, a text, and a function area.

[0142] In some embodiments, the target interaction object is a function area, and the function area includes multiple sub-function areas. The control unit 902 is specifically used to:

[0143] Determine, from among the sub-functional areas of the functional area, a target sub-functional area that is closest to the target ray interaction point;

[0144] Control the cursor to move to the target sub-function area.

[0145] In some embodiments, the control unit 902 is specifically configured to:

[0146] When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stop position of the cursor interaction interface.

[0147] In some embodiments, the control unit 902 is specifically configured to:

[0148] When the ray interaction point is not on the interactive object of the cursor interaction interface, displaying the interactive object in a first display state;

[0149] Alternatively, when the duration of the ray interaction point being on the interactive object of the cursor interaction interface is greater than a preset hovering duration, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state;

[0150] Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

[0151] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous interface interaction control method embodiment, which will not be repeated here.

[0152] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned interface interaction control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0153] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any interface interaction control method provided in the embodiment of the present application.

[0154] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0155] In the above-mentioned interface interaction control device, computer-readable storage medium, and wearable device embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described interface interaction control device, computer-readable storage medium, wearable device and its corresponding units can refer to the description of the interface interaction control method in the above embodiment, and will not be repeated here.

[0156] The above is a detailed introduction to an interface interaction control method, device, wearable device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An interface interactive control method, It is characterized in that The method comprises: Displaying a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; Based on a current ray interaction point, displaying a cursor of the cursor control device on the cursor interaction interface, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; When the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point.

2. The interface interaction control method according to claim 1, It is characterized in that The method further comprises: The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is an intersection point between the virtual ray at the moved position and the cursor interaction interface.

3. The interface interaction control method according to claim 2, It is characterized in that The controlling the movement of the virtual ray based on the movement data of the cursor control device comprises: When the cursor control device moves, obtaining a first moving angle and a first moving direction of the cursor control device; Determining a second moving angle of a virtual ray of the cursor control device based on the first moving angle and a preset angle relationship; Determining a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship; The movement of the virtual ray is controlled based on the second movement angle and the second movement direction.

4. The interface interaction control method according to claim 3, It is characterized in that The controlling the movement of the virtual ray based on the second movement angle and the second movement direction includes: According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second moving angle.

5. The interface interaction control method according to claim 1, It is characterized in that The method further comprises: When a reset instruction of the cursor control device is triggered, the cursor is moved to a preset initial position of the cursor interaction interface, wherein when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

6. The interface interaction control method according to claim 1, It is characterized in that The method further comprises: Determine the target interaction object on the cursor interaction interface that is closest to the target ray interaction point; If the distance between the target ray interaction point and the target interaction object is less than a preset distance threshold, the cursor is controlled to move to the target interaction object, wherein the target interaction object includes at least one of a control, a text, and a function area.

7. The interface interaction control method according to claim 6, It is characterized in that The target interactive object is a function area, the function area includes a plurality of sub-function areas, and controlling the cursor to move to the target interactive object includes: Determine, from among the sub-functional areas of the functional area, a target sub-functional area that is closest to the target ray interaction point; Control the cursor to move to the target sub-function area.

8. The interface interaction control method according to claim 1, It is characterized in that The method further comprises: When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stop position of the cursor interaction interface.

9. The interface interaction control method according to claim 1, It is characterized in that The method further comprises: When the ray interaction point is not on the interactive object of the cursor interaction interface, displaying the interactive object in a first display state; Alternatively, when the duration of the ray interaction point being on the interactive object of the cursor interaction interface is greater than a preset hovering duration, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state; Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

10. An interface interactive control device, It is characterized in that The interface interaction control device comprises: A display unit, used to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; A control unit, configured to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; The control unit is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

11. A wearable device, It is characterized in that It comprises a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the interface interaction control method as claimed in any one of claims 1 to 9 is executed.

12. A computer-readable storage medium, It is characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the interface interaction control method according to any one of claims 1 to 9.

Images