Interface interaction control method and apparatus, wearable device, and storage medium

Abstract

The present application provides an interface interaction control method and apparatus, a wearable device, and a computer-readable storage medium. The method comprises: displaying a virtual scene screen (201); on the basis of a current ray interaction point, displaying a cursor of a cursor control device on a cursor interaction interface of the screen (202); and when the cursor control device moves, controlling the cursor to move from the current ray interaction point to a target ray interaction point (203). The present application can improve the intuitiveness of interface interaction and increase interface interaction operation efficiency.

Description

Interface interaction control method, device, wearable device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 6, 2023, with application number 202311670972.7 and invention name “Interface interaction control method, device, wearable device and storage medium”, the entire contents of which are incorporated by reference in this disclosure. Technical Field

[0003] The present application relates to the technical field of interface interaction processing, and specifically to an interface interaction control method, apparatus, wearable device, and computer-readable storage medium. Background Art

[0004] Wearable devices have become increasingly popular in recent years. To better meet people's interactive needs, they can control the cursor position on wearable devices like smart glasses by moving a cursor control device (such as a finger ring). However, in related technologies, users have difficulty perceiving the changing patterns of the cursor position, resulting in less intuitive interface interaction and relatively low efficiency.

[0005] Summary of the Invention

[0006] The present application provides an interface interaction control method, apparatus, wearable device and computer-readable storage medium, which can improve the intuitiveness of interface interaction and improve the efficiency of interface interaction operations.

[0007] In a first aspect, the present application provides an interface interaction control method, the method comprising:

[0008] Displaying a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0009] displaying a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; and

[0010] When the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point.

[0011] In a second aspect, the present application provides an interface interaction control device, the interface interaction control device comprising:

[0012] A display unit, configured to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0013] a control unit, configured to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; and

[0014] The control unit is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

[0015] In a third aspect, the present application also provides a wearable device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, it executes any one of the interface interaction control methods provided in the present application.

[0016] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the interface interaction control method.

[0017] In this application, a virtual scene screen including the cursor of the cursor control device is displayed, and the cursor of the cursor control device is displayed on the cursor interaction interface based on the current ray interaction point; when the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point, so that the cursor can be presented at the intersection between the virtual ray and the cursor interaction interface, so that the user can perceive that the virtual ray moves with the cursor control device and the cursor position changes with the ray interaction point, and then the user can perceive the changing pattern of the cursor position, thereby improving the intuitiveness of the interface interaction and the efficiency of the interface interaction operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] FIG1 is a schematic block diagram of the structure of a wearable device provided in an embodiment of the present application;

[0020] FIG2 is a flow chart of an interface interaction control method provided in an embodiment of the present application;

[0021] FIG3 is a schematic diagram of a virtual scene screen provided in an embodiment of the present application;

[0022] FIG4 is another schematic diagram of a virtual scene screen provided in an embodiment of the present application;

[0023] FIG5 is a schematic diagram illustrating a ray reference coordinate system provided in an embodiment of the present application;

[0024] FIG6 is a schematic diagram of a scene of a cursor movement process provided in an embodiment of the present application;

[0025] FIG7 is a schematic diagram illustrating the states of the target interactive object when the ray interaction point is at different positions provided in an embodiment of the present application;

[0026] FIG8 is a schematic diagram illustrating an interface interaction control cursor automatically adsorbed onto an interactive object provided by an embodiment of the present application;

[0027] FIG9 is a schematic structural diagram of an embodiment of an interface interaction control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0030] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0031] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, well-known processes will not be elaborated in detail to avoid obscuring the description of the embodiments of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0032] Embodiments of the present application provide an interface interaction control method, apparatus, wearable device, and computer-readable storage medium. The interface interaction control apparatus can be integrated into a wearable device. The wearable device can be smart glasses, a smart helmet, or the like. The smart glasses can be AR (augmented reality) glasses, VR (virtual reality) glasses, or the like. The smart helmet can be an AR helmet, or the like.

[0033] The executor of the interface interaction control method of the embodiment of the present application may be the interface interaction control device provided in the embodiment of the present application, or a wearable device integrating the interface interaction control device, wherein the interface interaction control device may be implemented in hardware or software.

[0034] The following embodiments of the present application are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0035] FIG1 is a schematic block diagram of the structure of a wearable device provided in an embodiment of the present application.

[0036] As shown in FIG1 , a wearable device 100 includes a processor 101 and a memory 102 . The processor 101 and the memory 102 are connected via a bus 103 , such as an I 2 C (Inter-Integrated Circuit) bus.

[0037] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire wearable device 100. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0038] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0039] Those skilled in the art will understand that the structure shown in Figure 1 is merely a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the wearable device to which the embodiment of the present application is applied. The specific wearable device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0040] The processor 101 is configured to run a computer program stored in the memory 102 and implement any one of the interface interaction control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is configured to run a computer program stored in the memory 102 and implement the following steps when executing the computer program:

[0041] Display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; based on a current ray interaction point, display a cursor of the cursor control device on the cursor interaction interface, wherein the current ray interaction point is the intersection between the virtual ray at the current display position and the cursor interaction interface; when the cursor control device moves, control the cursor to move from the current ray interaction point to the target ray interaction point.

[0042] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0043] When the cursor control device moves, a first movement angle and a first movement direction of the cursor control device are obtained; based on the first movement angle and a preset angle relationship, a second movement angle of a virtual ray of the cursor control device is determined; based on the first movement direction and a preset direction relationship, a second movement direction of the virtual ray is determined; and based on the second movement angle and the second movement direction, the movement of the virtual ray is controlled.

[0044] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0045] The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is an intersection point between the virtual ray at the moved position and the cursor interaction interface.

[0046] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0047] According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray is the second moving angle.

[0048] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0049] When the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface, wherein, when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0050] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0051] Determine the target interactive object on the cursor interaction interface that is closest to the target ray interaction point; if the distance between the target ray interaction point and the target interactive object is less than a preset distance threshold, control the cursor to move to the target interactive object, wherein the target interactive object includes at least one of a control, text, and a function area.

[0052] In some embodiments, the target interaction object is a functional area, which includes multiple sub-functional areas. The processor 101 is configured to run a computer program stored in the memory 102 and implement the following steps when executing the computer program:

[0053] From the sub-functional areas of the functional area, determine the target sub-functional area that is closest to the target ray interaction point; and control the cursor to move to the target sub-functional area.

[0054] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0055] When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stay position of the cursor interaction interface.

[0056] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, the processor 101 may implement the following steps:

[0057] When the ray interaction point is not on the interactive object of the cursor interaction interface, the interactive object is displayed in a first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface for a time period greater than a preset hovering time period, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

[0058] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the wearable device described above can refer to the corresponding process in the following interface interaction control method embodiment, and will not be repeated here.

[0059] Below, the interface interaction control method provided in the embodiment of the present application will be described in detail using the wearable device shown in Figure 1 as the execution subject of the interface interaction control method. For the sake of simplicity and ease of description, this execution subject will be omitted in the subsequent method embodiments. It should be noted that the scenario in Figure 1 is only used to explain the interface interaction control method provided in the embodiment of the present application, and does not constitute a limitation on the application scenario of the interface interaction control method provided in the embodiment of the present application.

[0060] Please refer to Figure 2, which is a flow chart of an interface interaction control method provided by an embodiment of the present application. The interface interaction control method includes steps 201 to 203, wherein:

[0061] 201. Display a virtual scene image.

[0062] The virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device.

[0063] Among them, the cursor interaction interface is a user interface that can be interacted with by the cursor. In some embodiments, the cursor interaction interface can be the entire virtual scene screen; for example, as shown in Figure 3, the rectangular area in Figure 3 represents the virtual scene screen displayed by the AR glasses, and the entire virtual scene screen can be interacted with by the cursor, so the entire virtual scene screen can be used as the cursor interaction interface. In other embodiments, the cursor interaction interface can be a part of the virtual scene screen; for example, as shown in Figure 4, Figure 4 is a scene diagram of the virtual scene screen, and only the dotted rectangular box area in Figure 4 can be interacted with by the cursor, so the dotted rectangular box area in the virtual scene screen is used as the cursor interaction interface.

[0064] The cursor control device is a device that generates a cursor on a cursor interaction interface for interaction. For example, the cursor control device may be a finger ring, a watch, or the like.

[0065] For example, a wearable device may display a virtual scene image, which may be a game image, a video image, etc., and the specific image content of the virtual scene image is not limited in this embodiment. For example, a game image may be displayed through AR glasses.

[0066] Among them, the wearable device can be smart glasses, smart helmets, etc., the smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, etc., and the smart helmet can be an AR helmet, etc.

[0067] 202. Display a cursor of the cursor control device on the cursor interaction interface based on the current ray interaction point.

[0068] In order to better understand the embodiments of the present application, the following first introduces some of the names involved in this embodiment:

[0069] 1. Ray starting point: This is the starting point of the virtual ray, which can be any point outside the plane of the cursor interface. In this embodiment, to more accurately simulate cursor movement, the ray starting point is a preset fixed point. For example, as shown in Figure 3, point O in Figure 3 represents the ray starting point.

[0070] 2. Virtual Rays: Any ray originating from the ray origin and directed toward any point on the cursor interface is considered a virtual ray for the cursor control device. For example, as shown in Figure 3, ray OA intersecting the cursor interface at point A can be considered a virtual ray, and ray OB intersecting the cursor interface at point B can also be considered a virtual ray. In this embodiment, the movement of the virtual ray is controlled based on the movement of the cursor control device, thereby changing the position of the intersection point between the virtual ray and the cursor interface to achieve cursor control.

[0071] 3. Ray interaction point: the intersection point between the virtual ray and the cursor interaction interface. In this embodiment, the display position of the cursor on the cursor interaction interface is the location of the ray interaction point. Therefore, the display position of the cursor on the cursor interaction interface changes with the change of the ray interaction point. The movement of the cursor can be controlled by controlling the movement of the virtual ray. For example, as shown in Figure 3, since the intersection point of ray OA and the cursor interaction interface in Figure 3 is point A, the location point of the cursor when the virtual ray is at the position of ray OA is point A; since the intersection point of ray OB and the cursor interaction interface in Figure 3 is point B, the location point of the cursor when the virtual ray is at the position of ray OB is point B. Therefore, the location point of the cursor can be continuously adjusted according to the ray interaction point (i.e., the intersection point between the virtual ray and the cursor interaction interface).

[0072] 4. The default initial position of the cursor: that is, the default position of the cursor in the cursor interaction interface.

[0073] 5. Default position of the virtual ray: This refers to the position of the virtual ray when the cursor is at the preset initial position. For example, as shown in Figure 3, assuming that the preset initial position of the cursor is the center point P of the cursor interaction interface, the default position of the virtual ray can be set to the position of the ray from the ray starting point O to the center point P of the cursor interaction interface.

[0074] The current ray interaction point is the intersection between the virtual ray at the current display position and the cursor interaction interface.

[0075] 203. When the cursor control device moves, control the cursor to move from the current ray interaction point to a target ray interaction point.

[0076] The virtual ray moves based on the movement data of the cursor control device, and the target ray interaction point is the intersection point between the virtual ray at the moved position and the cursor interaction interface. Exemplarily, when the cursor control device moves, the virtual ray is controlled to move with the cursor control device so that the intersection point between the virtual ray and the cursor interaction interface changes accordingly, thereby controlling the cursor to move from the current ray interaction point to the target ray interaction point.

[0077] The interface interaction control method may further include: controlling the movement of the virtual ray based on the movement data of the cursor control device. For example, there is a preset angular relationship between the movement angle of the virtual ray and the movement angle of the cursor control device, and there is a preset directional relationship between the movement direction of the virtual ray and the movement direction of the cursor control device. At this point, the movement of the virtual ray may be controlled by referring to the following steps A1 to A4:

[0078] A1. When the cursor control device moves, obtain a first movement angle and a first movement direction of the cursor control device.

[0079] The first movement angle refers to the movement angle of the cursor control device. For example, if the cursor control device is a ring, and the finger of the user wearing the ring rotates 1° to the left, the first movement angle of the ring is 1°.

[0080] The first movement direction refers to the movement direction of the cursor control device. For example, if the cursor control device is a ring, and the finger of the user wearing the ring rotates 1° to the left, the first movement direction of the ring is left.

[0081] Here, "left" is an example made for the convenience of understanding. It can be understood that an inertial measurement unit (IMU) can be built into the cursor control device to measure and detect whether the cursor control device has moved. When the cursor control device moves, the three-axis angular velocity and three-axis acceleration of the cursor control device are measured by the IMU built into the cursor control device, so as to calculate the first movement angle and the first movement direction of the cursor control device. Specifically, the first movement angle and the first movement direction of the cursor control device can be represented by the IMU sensor coordinate system built into the cursor control device. For example, in the IMU sensor coordinate system built into the cursor control device, the first movement angle and the first movement direction of the cursor control device are represented by the rotation angle of the cursor control device relative to the x-axis, y-axis, and z-axis.

[0082] A2. Determine a second moving angle of the virtual ray of the cursor control device based on the first moving angle and a preset angle relationship.

[0083] The virtual ray is used to reflect the movement of the cursor control device. The position of the virtual ray changes as the cursor control device moves, thereby changing the position of the intersection between the virtual ray and the cursor interaction interface to achieve control of the cursor.

[0084] The second movement angle refers to the movement angle of the virtual ray. For example, if the virtual ray rotates 1 degree to the left, the second movement angle of the virtual ray is 1 degree.

[0085] The movement angle and movement direction of the virtual ray can be represented by a preset ray reference coordinate system (such as a coordinate system established with the ray starting point as the coordinate origin, the horizontal straight line parallel to the cursor interaction interface as the x-axis direction, the horizontal straight line perpendicular to the cursor interaction interface as the y-axis direction, and the vertical straight line parallel to the cursor interaction interface as the z-axis direction, as shown in FIG5 ). For example, as shown in FIG5 , in a preset ray reference coordinate system (wherein the origin of the ray reference coordinate system is the ray starting point O, and the yoz plane of the ray reference coordinate system is parallel to the plane of the cursor interaction interface), the second movement angle and second movement direction of the cursor are represented by the rotation angle of the virtual ray relative to the x-axis, y-axis, and z-axis.

[0086] The preset angle relationship indicates the relationship between the first movement angle of the cursor control device and the second movement angle of the virtual ray. The preset angle relationship can be set based on actual business scenario requirements. The specific value relationship of the preset angle relationship is not limited here. For example, the preset angle relationship may be: for every 1-degree rotation of the first movement angle, the second movement angle rotates 3 degrees.

[0087] A3. Determine a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship.

[0088] The preset directional relationship is used to indicate the relationship between the first movement direction of the cursor control device and the second movement direction of the virtual ray. The preset directional relationship can be set according to actual business scenario requirements and is not limited to the setting of the preset directional relationship. For example, the preset directional relationship may be: when the first movement direction is clockwise relative to the x-axis of the sensor coordinate system, the second movement direction is clockwise relative to the x-axis of the ray reference coordinate system.

[0089] For example, the preset direction relationship indicates that: when the cursor control device rotates counterclockwise relative to the x-axis of the sensor coordinate system, the virtual ray rotates counterclockwise relative to the x-axis of the ray reference coordinate system; when the cursor control device rotates clockwise relative to the x-axis of the sensor coordinate system, the virtual ray rotates clockwise relative to the x-axis of the ray reference coordinate system; when the cursor control device rotates counterclockwise relative to the y-axis of the sensor coordinate system, the virtual ray rotates counterclockwise relative to the y-axis of the ray reference coordinate system; and when the cursor control device rotates clockwise relative to the y-axis of the sensor coordinate system, the virtual ray rotates clockwise relative to the y-axis of the ray reference coordinate system. Therefore, when the first movement direction is counterclockwise relative to the x-axis of the sensor coordinate system, the counterclockwise direction relative to the x-axis of the ray reference coordinate system is used as the second movement direction of the cursor.

[0090] A4. Control the movement of the virtual ray based on the second movement angle and the second movement direction.

[0091] Specifically, the virtual ray is controlled to move from the current display position according to the second movement direction until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second movement angle. For example, referring to Figure 5, if the second movement direction is "counterclockwise relative to the x-axis of the ray reference coordinate system" and the second movement angle is "3 degrees", the virtual ray is controlled to rotate counterclockwise relative to the x-axis of the ray reference coordinate system from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms 3 degrees.

[0092] To better understand how to complete interface interaction through virtual rays, a specific example is given below to illustrate. Please refer to Figure 6 (the circle in Figure 6 represents a cursor). For example, taking the cursor control device as a ring and the cursor interaction interface as the display interface of the AR glasses, when the user rotates the ring-wearing finger 1 degree to the left, step A1 detects that the ring is rotated 1 degree counterclockwise relative to the x-axis of the sensor coordinate system (i.e., the first movement angle is a rotation of 1 degree, and the first movement direction is the counterclockwise direction relative to the x-axis of the sensor coordinate system). Through steps A2 to A3, it can be determined that the second movement angle is a rotation of 3 degrees, and the second movement direction is the counterclockwise direction relative to the x-axis of the ray reference coordinate system. Then, the virtual ray can be rotated 3 degrees counterclockwise relative to the x-axis of the ray reference coordinate system from the current display position, and then the cursor is moved from the current ray interaction point (e.g., the intersection P1 between the virtual ray at the current display position and the cursor interaction interface) to the target ray interaction point (e.g., the intersection P2 between the virtual ray at the moved position and the cursor interaction interface); thereby, when the user rotates the ring-wearing finger 1 degree to the left, the cursor moves 3 degrees on the display interface of the AR glasses. At this time, if the user presses the confirmation button on the ring, the "confirmation" interaction process will be completed.

[0093] Therefore, by determining the second movement angle of the virtual ray based on the first movement angle of the cursor control device and the preset angle relationship, the cursor movement can be controlled, so that the movement angle of the virtual ray can be larger when the movement angle of the cursor control device is relatively small, thereby increasing the movement amplitude of the cursor, thereby reducing the movement amplitude of the cursor control device, thereby reducing the user's movement amplitude of the cursor control device (for example, taking the cursor control device as a finger ring as an example, in order to reduce the movement amplitude of the wrist or arm), thereby reducing the discomfort of controlling the cursor control device and improving the interactive convenience of the cursor control device.

[0094] Thus, it can be seen that, firstly, by displaying a virtual scene screen containing the cursor of the cursor control device, when the reset command of the cursor control device is triggered, the cursor of the cursor control device is moved to the preset initial position of the cursor interaction interface; thus, when the cursor is in a position that is not conducive to user movement, the reset command can quickly restore the cursor to the default position (for example, when the user holds the cursor control device on the thigh, the cursor is at the bottom of the cursor interaction interface. Originally, it is necessary to raise the arm to raise the cursor control device to raise the cursor to the center point of the interface. The reset command can directly restore the cursor to the center point of the interface). This makes it unnecessary to move the arm and wrist to a specified height or angle to achieve cursor movement, avoiding the problem of the user having to move the cursor for a long time and with a large amplitude to achieve cursor movement, thereby improving the convenience of interactive operations. Secondly, when the cursor control device moves, controlling the cursor to move from the preset initial position to other positions of the cursor interaction interface can enable the user to move the cursor control device in a relatively comfortable position to move the cursor for interaction, thereby improving the convenience of interactive operations of the cursor.

[0095] Furthermore, as shown in FIG5 , in order to improve the intuitiveness of the interaction and the visibility of the interactive operation, the virtual scene screen also includes a virtual wearable object of the cursor control device, the direction of the virtual ray is the same as the direction of the virtual wearable object, the starting point of the virtual ray (i.e., the ray starting point) is the same as the position of the virtual wearable object corresponding to the cursor control device, and the intersection between the virtual ray and the cursor interaction interface is displayed as the cursor; when the direction of the actual wearable object of the cursor control device (such as when the cursor control device is a ring, the direction of the user's finger wearing the ring) moves, the direction of the virtual ray moves accordingly, thereby providing the user with an intuitive interactive feeling visually and improving the visibility of the interactive operation, so that the user can clearly understand the current interactive behavior.

[0096] Furthermore, in order to improve the convenience of interactive operations of the cursor, the interface interaction control method may further include: when a reset instruction of the cursor control device is triggered, moving the cursor to a preset initial position of the cursor interaction interface.

[0097] The preset initial position is the default position of the cursor of the cursor control device in the cursor interaction interface. For example, the preset initial position may be the center point of the cursor interaction interface.

[0098] When the cursor is at a preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0099] The reset command is used to control the cursor of the cursor control device to return to a preset initial position. The reset command can be triggered in various ways, for example, when a reset button of the cursor control device is pressed, or when a user gesture is detected that matches a preset gesture.

[0100] For example, as shown in Figure 3, assuming that the preset initial position of the cursor is the center point P of the cursor interaction interface, before the reset instruction of the cursor control device is triggered, the position point of the cursor is point A of the cursor interaction interface; when the reset instruction of the cursor control device is triggered, the position point of the cursor is restored to the preset initial position of the cursor interaction interface (that is, the center point P of the cursor interaction interface).

[0101] There are many ways to implement “triggering the reset instruction of the cursor control device”, illustratively including:

[0102] (1) When a user makes a preset gesture to reset the cursor position, a reset instruction of the cursor control device is triggered, and the cursor is reset to the preset initial position of the cursor interaction interface. In this case, "when the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface" may specifically include:

[0103] 2021A, obtain user interaction images.

[0104] 2022A. Perform gesture recognition on the user interaction image to obtain the gesture of the user interaction image.

[0105] 2023A. When the gesture of the user interaction image is the same as the preset gesture, trigger a reset instruction of the cursor control device.

[0106] 2024A. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0107] (2) By providing a reset button on the cursor control device, when the user presses the reset button of the cursor control device, a reset instruction of the cursor control device is triggered, and the cursor is reset to the preset initial position of the cursor interaction interface. In this case, "when the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface" may specifically include:

[0108] 2021B. When a cursor reset request is received from the cursor control device, a reset instruction of the cursor control device is triggered, wherein the cursor control device sends the cursor reset request when a touch operation of a reset button is detected.

[0109] 2022B. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0110] (3) Collecting user gestures through a cursor control device (such as a finger ring), triggering a reset instruction of the cursor control device when the user makes a preset gesture to reset the cursor position, and resetting the cursor to the preset initial position of the cursor interaction interface. In this case, step 202 may specifically include:

[0111] 2021C. When a cursor reset request is received from the cursor control device, trigger a reset instruction of the cursor control device, wherein the cursor control device sends the cursor reset request when a preset gesture is detected.

[0112] 2022C. When a reset instruction of the cursor control device is triggered, move the cursor of the cursor control device to a preset initial position of the cursor interaction interface.

[0113] Furthermore, to improve interaction efficiency, when the cursor corresponding to the virtual ray approaches an interactive object such as a clickable control, text, or function area, the cursor can be automatically attached to the interactive object. Specifically, the interface interaction control method can further include: determining the target interactive object on the cursor interaction interface that is closest to the target ray interaction point; and if the distance between the target ray interaction point and the target interactive object on the cursor interaction interface is less than a preset distance threshold, controlling the cursor to move to the target interactive object.

[0114] The interactive object includes at least one of a control, text, and a function area.

[0115] The target interaction object refers to the interaction object closest to the target ray interaction point among the interaction objects on the cursor interaction interface.

[0116] In one embodiment, after the target interactive object is determined, the cursor can be controlled to directly adsorb onto the target interactive object.

[0117] In another embodiment, as shown in FIG8 , taking the interactive object being a clickable control as an example, assuming that there are four clickable controls on the cursor interaction interface (as shown in FIG8 , from left to right are: control 1, control 2, control 3, and control 4), the control closest to the target ray interaction point (such as control 1) is determined from the four clickable controls as the target interactive object. When the distance between the target ray interaction point and the clickable control 1 on the cursor interaction interface is less than a preset distance threshold, the cursor is controlled to be adsorbed onto the clickable control 1, thereby improving the efficiency of control selection.

[0118] For example, taking the interactive object as text, assuming that there are 4 texts that can be selected and copied on the cursor interaction interface (such as: "Text 1", "Text 2", "Text 3", and "Text 4"), the text closest to the target ray interaction point is determined from the 4 texts that can be selected and copied (such as "Text 1") as the target interactive object. When the distance between the target ray interaction point and "Text 1" on the cursor interaction interface is less than the preset distance threshold, the cursor is controlled to be adsorbed on "Text 1" (such as the characters or words of the text). This can improve the efficiency of text selection and make it easier to edit documents, select text, or position the cursor.

[0119] For another example, taking the interactive object as a function area, assuming that there are four clickable function areas on the cursor interaction interface (such as: function area 1, function area 2, function area 3, and function area 4), the function area closest to the target ray interaction point (such as clickable function area 4) is determined from the four clickable function areas as the target interactive object. When the distance between the target ray interaction point and function area 4 on the cursor interaction interface is less than a preset distance threshold, the cursor is controlled to be adsorbed on function area 4, thereby improving the selection efficiency of the function area. Furthermore, if the function area includes multiple sub-function areas, the target sub-function area closest to the target ray interaction point can be determined from the sub-function areas of the function area; and the cursor is controlled to move to the target sub-function area. The distance between the middle pixel point of the sub-function area and the target ray interaction point can be obtained as the distance between the sub-function area and the target ray interaction point.

[0120] Furthermore, in order to highlight the adsorption effect of the cursor, the target interactive object can be highlighted when the cursor is adsorbed on it. In addition, different highlighting methods can be used for different types of target interactive objects. For example, when the cursor is adsorbed to an icon or label (tab) without a background, it can be highlighted with a highlight effect; when the cursor is adsorbed to a button or list with a background, it can be highlighted with a lifting effect; when the cursor is adsorbed to an application icon, it can be highlighted with a floating effect.

[0121] Furthermore, to prevent the cursor from being lost, the cursor can be automatically adsorbed to a preset rest position on the cursor interaction interface when the user does not move the cursor for a period of time. That is, the interface interaction control method can further include: when the cursor remains at the target ray interaction point for a duration greater than a preset duration, controlling the cursor to move to the preset rest position on the cursor interaction interface. For example, taking the edge of the cursor interaction interface as an example, if the cursor remains at the target ray interaction point for more than 20 seconds, the cursor automatically adsorbs to the edge of the cursor interaction interface.

[0122] Furthermore, to enhance the intuitiveness of the interactive display and allow the user to clearly understand the current interactive situation, when the intersection between the virtual ray and the cursor interaction interface is on an interactive control, the interactive control may be highlighted (e.g., enlarged or changed in color). That is, the interface interaction control method may further include: if the target ray interaction point is on an interactive control, highlighting the interactive control on the cursor interaction interface. For example, when the intersection between the virtual ray and the cursor interaction interface falls on a key, the key is enlarged and displayed in a different color.

[0123] Furthermore, the cursor control device is a wearable device, as shown in Figure 7. After wearing the cursor control device (such as a ring), the user can use the ray interaction point to interact with the interface. For example, when the ray interaction point is on the target that the user wants to select (such as a button, function area, text, etc.), the user can press or slide the ring touch bar to perform various interactive operations such as confirmation, return, scroll, etc.

[0124] Furthermore, in order to improve the interactive visibility of the ray interaction point, the states of the interactive object include an idle state, a hovering state, and a clicked state, and the display style of the interactive object is different in each state. That is, the interface interaction control method may further include: when the ray interaction point is not on the interactive object of the cursor interaction interface, the interactive object is displayed in a first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface for a time period longer than a preset hovering time period, the interactive object is displayed in a second display state, wherein in the second display state, at least one of the size, color, and style of the interactive object is different from that in the first display state; or, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein in the third display state, at least one of the size, color, and style of the interactive object is different from that in the second display state.

[0125] For example, as shown in Figure 7, when the ray interaction point (i.e., the cursor) is not on the interactive object (such as a button), the interactive object is in an idle state, and the control is displayed according to the first display state; when the ray interaction point (i.e., the cursor) is on the interactive object (such as a button) but no operation instruction of the interactive object is issued (such as when "Confirm" is not pressed), the interactive object is in a hovering state, and the control is displayed according to the second display state; when the ray interaction point (i.e., the cursor) is on the interactive object (such as a button) and an operation instruction of the interactive object is issued (such as when the physical "Confirm" button is pressed on the ring), the interactive object is in a click state, and the control is displayed according to the third display state.

[0126] From the above content, it can be seen that in this embodiment, by displaying a virtual scene screen including the cursor of the cursor control device, the cursor of the cursor control device is displayed on the cursor interaction interface based on the current ray interaction point; when the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point, so that the cursor can be presented at the intersection between the virtual ray and the cursor interaction interface, so that the user can perceive that the virtual ray moves with the cursor control device and the cursor position changes with the ray interaction point, and then the user can perceive the changing pattern of the cursor position, thereby improving the intuitiveness of the interface interaction and the efficiency of the interface interaction operation.

[0127] In addition, in order to better implement the interface interaction control method in the embodiment of the present application, based on the interface interaction control method, the embodiment of the present application further provides an interface interaction control device. As shown in FIG9 , which is a schematic structural diagram of an embodiment of the interface interaction control device provided in the embodiment of the present application, the interface interaction control device 900 includes:

[0128] A display unit 901 is used to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device;

[0129] A control unit 902 is configured to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface;

[0130] The control unit 902 is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

[0131] In some embodiments, the control unit 902 is configured to:

[0132] The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is the intersection between the virtual ray at the moved position and the cursor interaction interface.

[0133] In some embodiments, the control unit 902 is configured to:

[0134] When the cursor control device moves, obtaining a first movement angle and a first movement direction of the cursor control device;

[0135] determining a second movement angle of the virtual ray of the cursor control device based on the first movement angle and a preset angle relationship;

[0136] determining a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship;

[0137] The movement of the virtual ray is controlled based on the second movement angle and the second movement direction.

[0138] In some embodiments, the control unit 902 is configured to:

[0139] According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray is the second moving angle.

[0140] In some embodiments, the control unit 902 is configured to:

[0141] When the reset instruction of the cursor control device is triggered, the cursor is moved to the preset initial position of the cursor interaction interface, wherein, when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

[0142] In some embodiments, the control unit 902 is configured to:

[0143] Determine a target interactive object on the cursor interactive interface that is closest to the target ray interaction point;

[0144] If the distance between the target ray interaction point and the target interactive object is less than a preset distance threshold, the cursor is controlled to move to the target interactive object, wherein the target interactive object includes at least one of a control, text, and a function area.

[0145] In some embodiments, the target interaction object is a function area, which includes multiple sub-function areas. The control unit 902 is configured to:

[0146] Determine, from among the sub-functional areas of the functional area, a target sub-functional area that is closest to the target ray interaction point;

[0147] Control the cursor to move to the target sub-function area.

[0148] In some embodiments, the control unit 902 is configured to:

[0149] When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stay position of the cursor interaction interface.

[0150] In some embodiments, the control unit 902 is configured to:

[0151] When the ray interaction point is not on the interactive object of the cursor interaction interface, displaying the interactive object in a first display state;

[0152] Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface for a period longer than a preset hovering period, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state;

[0153] Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

[0154] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can be found in the previous interface interaction control method embodiment, which will not be repeated here.

[0155] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned interface interaction control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0156] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple computer programs. The computer programs can be loaded by a processor to execute any interface interaction control method provided in the embodiment of the present application.

[0157] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0158] In the above-mentioned embodiments of the interface interaction control device, computer-readable storage medium, and wearable device, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes and beneficial effects of the above-mentioned interface interaction control device, computer-readable storage medium, wearable device, and their corresponding units can be referred to the description of the interface interaction control method in the above embodiment, and the details will not be repeated here.

[0159] The above is a detailed introduction to an interface interaction control method, device, wearable device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An interface interaction control method, the method comprising: Displaying a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; Based on a current ray interaction point, displaying a cursor of the cursor control device on the cursor interaction interface, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; and When the cursor control device moves, the cursor is controlled to move from the current ray interaction point to the target ray interaction point.

2. The interface interaction control method according to claim 1, wherein: The method further comprises: The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is the intersection point between the virtual ray at the moved position and the cursor interaction interface.

3. The interface interaction control method according to claim 2, wherein: The controlling the movement of the virtual ray based on the movement data of the cursor control device comprises: When the cursor control device moves, obtaining a first moving angle and a first moving direction of the cursor control device; Determining a second moving angle of a virtual ray of the cursor control device based on the first moving angle and a preset angle relationship; Determining a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship; and The movement of the virtual ray is controlled based on the second movement angle and the second movement direction.

4. The interface interaction control method according to claim 3, wherein: The controlling the movement of the virtual ray based on the second movement angle and the second movement direction includes: According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second moving angle.

5. The interface interaction control method according to claim 1, wherein: The method further comprises: When a reset instruction of the cursor control device is triggered, the cursor is moved to a preset initial position of the cursor interaction interface, wherein when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

6. The interface interaction control method according to claim 1, wherein: The method further comprises: Determine a target interactive object on the cursor interactive interface that is closest to the target ray interactive point; and If the distance between the target ray interaction point and the target interactive object is less than a preset distance threshold, the cursor is controlled to move to the target interactive object, wherein the target interactive object includes at least one of a control, a text, and a function area.

7. The interface interaction control method according to claim 6, wherein: The target interactive object is a function area, the function area includes a plurality of sub-function areas, and controlling the cursor to move to the target interactive object includes: Determine, from among the sub-functional areas of the functional area, a target sub-functional area that is closest to the target ray interaction point; and Control the cursor to move to the target sub-function area.

8. The interface interaction control method according to claim 1, wherein: The method further comprises: When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stop position of the cursor interaction interface.

9. The interface interaction control method according to claim 1, wherein: The method further comprises: When the ray interaction point is not on the interactive object of the cursor interaction interface, displaying the interactive object in a first display state; Alternatively, when the duration of the ray interaction point being on the interactive object of the cursor interaction interface is greater than a preset hovering duration, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state; Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

10. An interface interaction control device, wherein: The interface interaction control device comprises: A display unit, used to display a virtual scene screen, wherein the virtual scene screen includes a cursor interaction interface and a virtual ray of a cursor control device; a control unit, configured to display a cursor of the cursor control device on the cursor interaction interface based on a current ray interaction point, wherein the current ray interaction point is an intersection point between the virtual ray at a current display position and the cursor interaction interface; and The control unit is further configured to control the cursor to move from the current ray interaction point to a target ray interaction point when the cursor control device moves.

11. The interface interaction control device according to claim 10, wherein: The control unit is used for: The movement of the virtual ray is controlled based on the movement data of the cursor control device, wherein the target ray interaction point is the intersection point between the virtual ray at the moved position and the cursor interaction interface.

12. The interface interaction control device according to claim 11, wherein: The control unit is used for: When the cursor control device moves, obtaining a first moving angle and a first moving direction of the cursor control device; Determining a second moving angle of a virtual ray of the cursor control device based on the first moving angle and a preset angle relationship; Determining a second moving direction of the virtual ray based on the first moving direction and a preset direction relationship; The movement of the virtual ray is controlled based on the second movement angle and the second movement direction.

13. The interface interaction control device according to claim 12, wherein: The control unit is used for: According to the second moving direction, the virtual ray is controlled to move from the current display position until the angle between the moved position of the virtual ray and the current display position of the virtual ray forms the second moving angle.

14. The interface interaction control device according to claim 10, wherein: The control unit is used for: When a reset instruction of the cursor control device is triggered, the cursor is moved to a preset initial position of the cursor interaction interface, wherein when the cursor is at the preset initial position, the virtual ray is at a preset default position, and the intersection between the virtual ray at the default position and the cursor interaction interface is the preset initial position.

15. The interface interaction control device according to claim 10, wherein: The control unit is used for: Determine the target interaction object on the cursor interaction interface that is closest to the target ray interaction point; If the distance between the target ray interaction point and the target interactive object is less than a preset distance threshold, the cursor is controlled to move to the target interactive object, wherein the target interactive object includes at least one of a control, a text, and a function area.

16. The interface interaction control device according to claim 15, wherein: The target interaction object is a function area, the function area includes a plurality of sub-function areas, and the control unit is used for: Determine, from among the sub-functional areas of the functional area, a target sub-functional area that is closest to the target ray interaction point; Control the cursor to move to the target sub-function area.

17. The interface interaction control device according to claim 10, wherein: The control unit is used for: When the duration of the cursor being at the target ray interaction point is greater than a preset duration, the cursor is controlled to move to a preset stop position of the cursor interaction interface.

18. The interface interaction control device according to claim 10, wherein: The control unit is used for: When the ray interaction point is not on the interactive object of the cursor interaction interface, displaying the interactive object in a first display state; Alternatively, when the duration of the ray interaction point being on the interactive object of the cursor interaction interface is greater than a preset hovering duration, the interactive object is displayed in a second display state, wherein at least one of the size, color, and style of the interactive object in the second display state is different from that in the first display state; Alternatively, when the ray interaction point is on the interactive object of the cursor interaction interface and the interactive object is triggered, the interactive object is displayed in a third display state, wherein at least one of the size, color, and style of the interactive object in the third display state is different from that in the second display state.

19. A wearable device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the interface interaction control method according to any one of claims 1 to 9 is executed.

20. A computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the interface interaction control method according to any one of claims 1 to 9.

Images