Virtual rocker control method and computer program product, electronic device

Abstract

The present disclosure provides a virtual rocker control method and product and device, relates to the technical field of unmanned aerial vehicles, and comprises the following steps: displaying a virtual rocker on an operation interface, wherein the virtual rocker has an operation area; determining a first contact point coordinate of a pressing event acting on the virtual rocker and a second contact point coordinate of a moving event acting on the virtual rocker; determining a first offset in a first direction and a second offset in a second direction according to the first contact point coordinate and the second contact point coordinate; determining a main motion direction and a secondary motion direction based on the first offset and the second offset; determining a main direction screen coordinate of a center of the virtual rocker in the main motion direction and a secondary direction screen coordinate of the center of the virtual rocker in the secondary motion direction, updating a display position of the virtual rocker based on the main direction screen coordinate and the secondary direction screen coordinate, and controlling a target object corresponding to the virtual rocker. The present disclosure can accurately control the virtual rocker.

Description

Technical Field

[0001] This disclosure relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically, to a virtual joystick control method, computer program products, and electronic devices. Background Technology

[0002] Drones can be operated via virtual joysticks. In related technologies, two touch areas are typically drawn on the screen. Based on the distance and direction of finger movement within these areas, two sets of X / Y directional control values ​​are calculated and output. These are then combined to form four control values ​​(roll / pitch / throttle / yaw) which are sent to the drone.

[0003] In the above methods, touchscreens typically have a high sampling frequency and a relatively large contact area between the finger and the screen. Therefore, the output coordinate values ​​of finger movement are highly random, and the two-dimensional movement of the finger on the screen lacks constraints. The output values ​​are prone to large fluctuations, which may lead to large errors and make it impossible to achieve precise control of the target object. Summary of the Invention

[0004] The purpose of this disclosure is to provide a virtual joystick control method, computer program product, and electronic device, thereby overcoming, at least to some extent, the problem of inability to achieve precise control due to limitations and defects in related technologies.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a virtual joystick control method is provided, comprising: displaying a virtual joystick on an operation interface, the virtual joystick having an operation area; determining the coordinates of a first contact point of a press event acting on the virtual joystick and the coordinates of a second contact point of a movement event acting on the virtual joystick, the press event and the movement event belonging to the same operation cycle; determining a first offset in a first direction and a second offset in a second direction based on the first contact point coordinates and the second contact point coordinates; determining a primary movement direction and a secondary movement direction based on the first offset and the second offset; determining the screen coordinates of the center of the virtual joystick in the primary direction of the primary movement direction and the screen coordinates in the secondary direction of the secondary movement direction, wherein the determination of the screen coordinates in the secondary direction is related to an operation error threshold; updating the display position of the virtual joystick based on the screen coordinates in the primary direction and the screen coordinates in the secondary direction, and controlling a target object corresponding to the virtual joystick.

[0007] In one exemplary embodiment of this disclosure, the absolute value of the first offset is greater than or equal to the absolute value of the second offset, and the step of determining the primary motion direction and the secondary motion direction based on the first offset and the second offset includes: determining the first direction as the primary motion direction and determining the second direction as the secondary motion direction.

[0008] In an exemplary embodiment of this disclosure, determining the screen coordinates of the center of the virtual joystick in the primary direction of the primary motion direction and the screen coordinates in the secondary direction of the secondary motion direction includes: summing the component of the screen coordinates of the center point of the operation area in the first direction with the first offset to determine a first summation result; confining the first summation result within a first closed interval, the endpoints of the first closed interval being 0 and the side length of the operation area, respectively; and using the confined first summation result as the screen coordinates in the primary direction.

[0009] In one exemplary embodiment of this disclosure, determining the screen coordinates of the center of the virtual joystick in the primary direction of the primary motion direction and the screen coordinates in the secondary direction of the secondary motion direction includes: when the second offset is less than or equal to the operation error threshold, determining the component of the screen coordinates of the center point of the operation area in the second direction as the secondary direction screen coordinates; when the second offset is greater than the operation error threshold, determining the secondary direction screen coordinates based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the second direction.

[0010] In one exemplary embodiment of this disclosure, determining the secondary direction screen coordinates based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the second direction includes: determining the error between the absolute value of the second offset and the operation error threshold; determining the sign of the error based on the comparison result of the second offset and 0, wherein the error is positive when the second offset is greater than 0 and negative when the second offset is less than 0; summing the component of the screen coordinates of the center point of the operation area in the second direction with the error to determine a second summation result; limiting the second summation result to a second closed interval, the endpoints of the second closed interval being 0 and the side length of the operation area, respectively; and using the limited second summation result as the secondary direction screen coordinates.

[0011] In one exemplary embodiment of this disclosure, the method further includes: normalizing the primary direction screen coordinates and the secondary direction screen coordinates to determine the primary direction control quantity and the secondary direction control quantity; determining the vibration degree of the vibration motor based on the primary direction control quantity and the secondary direction control quantity, and controlling the vibration motor to vibrate according to the vibration degree.

[0012] In one exemplary embodiment of this disclosure, the method further includes: responding to a lift event acting on the virtual joystick, controlling the virtual joystick to return at a constant speed from the display position to the center point of the operation area.

[0013] In one exemplary embodiment of this disclosure, controlling the virtual joystick to return at a constant speed from the display position to the center point of the operation area includes: determining a first return value for the return operation based on the primary direction screen coordinates and a preset number of return attempts; determining a second return value for the return operation based on the secondary direction screen coordinates and the number of return attempts; subtracting the first return value from the primary direction screen coordinates to obtain a first temporary coordinate component, and subtracting the second return value from the secondary direction screen coordinates to obtain a second temporary coordinate component, thereby performing the return operation and updating the display position based on the first and second temporary coordinate components until the number of times the return operation is performed reaches the preset number of return attempts.

[0014] According to one aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the virtual joystick control method described in any of the preceding claims.

[0015] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions; wherein the processor is configured to perform the virtual joystick control method described in any of the preceding claims by executing the executable instructions.

[0016] In the technical solution provided by this disclosure, on the one hand, the primary and secondary movement directions of the virtual joystick are distinguished by a first offset and a second offset. The primary screen coordinates of the virtual joystick in the primary movement direction are determined, and the secondary screen coordinates in the secondary movement direction are determined based on an operation error threshold. This enables precise determination of the virtual joystick's position during movement, avoiding the randomness of the coordinates output by movement events and achieving precise control of the target object corresponding to the virtual joystick. On the other hand, the screen coordinates of the secondary direction can be determined based on an operation error threshold. This threshold can filter out some operations in the secondary movement direction, achieving filtering of the secondary movement direction and avoiding unreasonable jitter. This improves the stability and reliability of triggering the virtual joystick through movement events and enhances the user experience.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 The diagram illustrates a virtual joystick control method according to an embodiment of the present disclosure.

[0020] Figure 2 The diagram illustrates a virtual joystick and its operating area in an embodiment of this disclosure.

[0021] Figure 3 The schematic diagram illustrates the process of determining the screen coordinates of the secondary orientation in an embodiment of this disclosure.

[0022] Figure 4 This diagram schematically illustrates the processing operations triggered by the type of touch event according to an embodiment of the present disclosure.

[0023] Figure 5 The schematic diagram illustrates the process of determining the primary and secondary screen coordinates according to an embodiment of the present disclosure. Detailed Implementation

[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0025] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] In some embodiments, virtual joystick control has the following problems: Touchscreens typically have a high sampling frequency, and the contact area between the finger and the screen is relatively large, so the output coordinate values ​​of finger movement have a large degree of randomness, which is not conducive to precise control of the drone's attitude; the two-dimensional movement of the finger on the touchscreen lacks constraints (physical joysticks usually have spring damping), and the output value is prone to large fluctuations; touchscreens cannot simulate the damping feel of physical joysticks, and when users operate with their eyes off the screen, there is a lack of mechanical feedback, which can easily lead to accidental touches.

[0027] In order to solve the technical problems in the related art, some embodiments of this disclosure provide a virtual joystick control method, which can be applied to scenarios where a target object is controlled by a virtual joystick.

[0028] Figure 1 The flowchart of the virtual joystick control method is illustrated in the figure. (Refer to...) Figure 1 As shown, the main steps include:

[0029] In step S110, a virtual joystick is displayed on the operation interface, the virtual joystick having an operation area;

[0030] In step S120, the coordinates of the first contact point of a press event acting on the virtual joystick and the coordinates of the second contact point of a move event acting on the virtual joystick are determined, wherein the press event and the move event belong to the same operation cycle;

[0031] In step S130, a first offset in the first direction and a second offset in the second direction are determined based on the coordinates of the first contact point and the coordinates of the second contact point.

[0032] In step S140, the primary motion direction and the secondary motion direction are determined based on the first offset and the second offset.

[0033] In step S150, the screen coordinates of the center of the virtual joystick in the primary direction of the main motion direction and the screen coordinates in the secondary direction of the secondary motion direction are determined, wherein the determination of the screen coordinates in the secondary direction is related to the operation error threshold.

[0034] In step S160, the display position of the virtual joystick is updated based on the primary direction screen coordinates and the secondary direction screen coordinates, and the target object corresponding to the virtual joystick is controlled.

[0035] In this embodiment, on the one hand, by distinguishing the primary and secondary movement directions of the virtual joystick using a first offset and a second offset, the primary direction screen coordinates of the virtual joystick in the primary movement direction are determined, and the secondary direction screen coordinates in the secondary movement direction are determined based on an operation error threshold. This improves the accuracy of the virtual joystick's position, avoids the randomness of the output coordinates of movement events, and achieves precise control of the target object corresponding to the virtual joystick. On the other hand, since the screen coordinates of the secondary direction can be determined based on the operation error threshold, some operations in the secondary movement direction can be filtered out based on the operation error threshold, achieving filtering of the secondary movement direction, avoiding unreasonable jitter, improving the stability and reliability of movement events, and enhancing the user experience.

[0036] Next, refer to Figure 1 As shown, each step in the virtual joystick control method in the embodiments of this disclosure will be described in detail.

[0037] In step S110, a virtual joystick is displayed on the operation interface, the virtual joystick having an operation area.

[0038] In some embodiments of this disclosure, a virtual joystick can be used to control a target object, which can be a game object in a game or a drone. Here, we will use a drone as an example to illustrate the concept. The drone can be any type of drone, such as a drone for delivering goods, like food delivery or other items; it can also be a drone for aerial photography; or it can be a drone for operating crops or performing other tasks. There are no specific limitations here.

[0039] A virtual joystick can be displayed on the user interface used to control the target object. This virtual joystick has a corresponding operating area. The virtual joystick, also known as a virtual joystick or controller, is the area where the user actually operates. The virtual joystick can be a circle with a variable radius, specifically represented as a stick. A variable radius circle means that before receiving a touch operation, the virtual joystick can be displayed as a first circle; after receiving a touch operation, it can be displayed as a second circle, with a radius larger than the first circle. Throughout the entire movement event, the radius of the second circle is determined by the distance from the touch point of the movement event to the center of the virtual joystick area, and the radius of the second circle does not exceed the side length of the virtual joystick's operating area.

[0040] The operating area of ​​a virtual joystick refers to the movable range of the virtual joystick. The operating area can be square, with a side length denoted as w. The movement range of the virtual joystick is limited by its operating area; that is, the virtual joystick can only move within the operating area. For example, a virtual joystick and its corresponding operating area can be provided at any location on the user interface.

[0041] Figure 2 The diagram illustrates the virtual joystick and the control area. (See reference) Figure 2 As shown, the virtual joystick 201 can be circular, and the operating area 202 can be square. The virtual joystick is a circle located inside the operating area. It should be noted that the virtual joystick can be a circle with a variable radius, used to represent the real-time projection position of the virtual joystick within the operating area. The operating area represents the movable range of the virtual joystick.

[0042] In step S120, the coordinates of the first contact point of a press event acting on the virtual joystick and the coordinates of the second contact point of a movement event acting on the virtual joystick are determined, wherein the press event and the movement event belong to the same operation cycle.

[0043] In this embodiment of the disclosure, when a touch event is detected acting on the virtual joystick, the type of the touch event can first be determined. The type of touch event can include any one of a press event, a move event, and a release event. A press event refers to a finger pressing down, used to trigger control of the virtual joystick. A move event refers to a finger moving, used to control the movement of the virtual joystick. A release event can be a finger lifting event, used to indicate the end of control of the virtual joystick. Specifically, a press event can be represented as ACTION_DOWN; a move event can be represented as ACTION_MOVE; and a release event can be represented as ACTION_UP.

[0044] Within the operation cycle of each touch event, the sequence of actions includes a press event, a move event, and a release event. Each touch event begins with a press event and ends with a release event. Furthermore, when each action, such as a press event, a move event, or a release event, occurs, the coordinates of the current touch point are simultaneously determined. These coordinates can be the screen coordinates of the current touch point, specifically represented as endX / endY.

[0045] Based on this, several parameters related to touch events can be determined. These include multiple variables, constants, and enumeration types. For example, the screen coordinates of the finger when it presses down can be represented as startX / startY. The offset between the screen coordinates endX / endY of the current contact point when the finger moves with the ACTION_MOVE event and the screen coordinates startX / startY of the finger when it presses down can be represented as deltaX / deltaY. stickX / stickY represent the screen coordinates of the center of the virtual joystick to be drawn. Center records the screen coordinates of the center point of the operation area; since the RockerView is a square, the actual coordinates are [center, center]. The constant can be the operation error threshold, which can be represented as THRESHOLD, representing the threshold of allowed finger operation error. The operation error threshold can be 50 and can be adjusted by the user according to actual needs. The enumeration type MainDirection{Direction_X,Direction_Y} represents the main direction of movement.

[0046] Upon receiving a press event (ACTION_DOWN) on the virtual joystick, the current contact point coordinates of the press event can be used as the first contact point coordinates. It can be determined whether the distances from the first contact point coordinates (endX / endY) to the center point (screen coordinates) of the operation area in both directions exceed the operation error threshold (THRESHOLD). If the operation error threshold is exceeded, the current operation cycle ends; otherwise, the first contact point coordinates (startX and startY) of the press event are updated. The values ​​of the first contact point coordinates (startX and startY) can be the current contact point coordinates (endX / endY), i.e., startX = endX, startY = endY. Here, the current contact point coordinates refer to the screen coordinates of the current contact point when the press event occurs, and the first contact point coordinates refer to the screen coordinates of the finger's contact point when the press event occurs.

[0047] When a movement event ACTION_MOVE is received that acts on the virtual joystick, the second contact point coordinates endX / endY of the movement event can be used as the second contact point coordinates.

[0048] It should be noted that the virtual joystick's press event and move event belong to the same operation cycle. That is, the press event and the move event belong to a set of operations within a certain time period, and have the same task start time and task end time.

[0049] In step S130, a first offset in the first direction and a second offset in the second direction are determined based on the coordinates of the first contact point and the coordinates of the second contact point.

[0050] In this embodiment of the disclosure, after receiving a press event, if a movement event acting on the virtual joystick is received, a first offset in the first direction and a second offset in the second direction are determined based on the coordinates of the second contact point of the movement event and the coordinates of the first touch point of the press event. If a movement event ACTION_MOVE is received, it is first determined whether the coordinates of the second contact point of the movement event are both 0; if so, the current triggering operation ends; if not, the first offset in the first direction and the second offset in the second direction are calculated.

[0051] For example, the first direction can be set perpendicular to the second direction; for instance, the first direction can be the X direction and the second direction can be the Y direction. Alternatively, the first direction can be the Y direction and the second direction can be the X direction. When a movement event ACTION_MOVE is detected, the second contact point coordinates endX / endY of the movement event can be received. Further, the difference between the second contact point coordinates endX / endY of the movement event and the first touch point coordinates startX / startY of the press event can be calculated in the first and second directions respectively to obtain a first offset in the first direction and a second offset in the second direction. For example, when the first direction is the X direction and the second direction is the Y direction, the first offset can be represented as deltaX and the second offset can be represented as deltaY.

[0052] In step S130, the primary motion direction and the secondary motion direction are determined based on the first offset and the second offset.

[0053] In this embodiment of the disclosure, after determining the first offset and the second offset, the primary motion direction and the secondary motion direction can be determined based on the first offset and the second offset. For example, the first offset and the second offset can be compared to determine the maximum offset and the minimum offset. Further, the direction corresponding to the maximum offset can be determined as the primary motion direction, and the direction corresponding to the minimum offset can be determined as the secondary motion direction. For example, if the absolute value of the first offset is greater than or equal to the absolute value of the second offset, the first direction is determined as the primary motion direction, and the second direction is determined as the secondary motion direction.

[0054] For example, when the absolute value of the first offset is greater than the second offset, the X direction corresponding to the first offset is determined as the primary direction of motion, and the Y direction corresponding to the second offset is determined as the secondary direction of motion.

[0055] In step S140, the screen coordinates of the center of the virtual joystick in the primary direction of motion and the screen coordinates in the secondary direction of motion are determined, wherein the determination of the screen coordinates in the secondary direction is related to the operation error threshold.

[0056] In this embodiment of the disclosure, after determining the main direction of motion, the screen coordinates of the center of the virtual joystick in the main direction of motion can be determined. Taking the X-direction as the main direction of motion as an example, the screen coordinates of the main direction of motion can be represented by the main direction screen coordinate stickX.

[0057] In some embodiments, the main direction screen coordinates can be determined based on the components of the second contact point coordinates of a movement event in the main direction, the components of the first touch point coordinates of a press event in the main direction, the components of the center point screen coordinates of the operation area in the main direction, and the side length of the operation area. For example, the difference between the current contact point coordinates of the movement event in the main direction and the touch point coordinates of the press event in the main direction is determined, i.e., the difference endX-startX is determined, and this difference is the first offset. After determining the first offset, the first summation result is determined by summing the first direction component centerPos of the center point screen coordinates of the virtual joystick's operation area with the first offset endX-startX. The first summation result is then confined to a first closed interval, with endpoints 0 and the side length w of the operation area, respectively. The first closed interval can be represented as [0, w]. Based on this, the confined first summation result can be used as the main direction screen coordinates. For example, when the first summation result is less than 0, the main direction screen coordinates are 0. When the first summation result is greater than or equal to 0 and less than or equal to the side length w of the operating area, the screen coordinates in the main direction are the first summation result. When the first summation result is greater than the side length w of the operating area, the screen coordinates in the main direction are the side length w of the operating area.

[0058] After determining the secondary direction of motion, the screen coordinates of the virtual joystick's center in the secondary direction of motion can be determined based on the second offset and the operation error threshold. For example, when the absolute value of the first offset is greater than or equal to the absolute value of the second offset, the minimum offset represented by the second offset can be compared with the operation error threshold to obtain a comparison result. Based on the comparison result, different methods can be selected to determine the screen coordinates of the virtual joystick's center in the secondary direction of motion.

[0059] For example, the comparison result can be that the minimum offset is less than or equal to the operation error threshold, or that the minimum offset is greater than the operation error threshold. Different comparison results lead to different methods for determining the screen coordinates in the secondary direction. In some embodiments, when the second offset is less than or equal to the operation error threshold, the component of the screen coordinates of the center point of the operation area in the secondary direction can be directly determined as the screen coordinates in the secondary direction; that is, stickY is directly equal to center.

[0060] If the second offset is greater than the operation error threshold, the screen coordinates in the secondary direction are determined based on the second offset, the operation error threshold, and the screen coordinates of the center point of the operation area in the secondary direction.

[0061] Figure 3 The flowchart illustrating the determination of secondary orientation screen coordinates is shown in the image. Figure 3 As shown, the main steps include:

[0062] In step S310, the error between the absolute value of the second offset and the operation error threshold is determined.

[0063] In this step, the absolute value of the second offset can be subtracted from the operation error threshold to obtain the error between the two. The error between the two can be specifically expressed as Math.abs(endY-startY)–THRESHOLD.

[0064] In step S320, the sign of the error is determined based on the comparison result between the second offset and 0, wherein the error is positive when the second offset is greater than 0, and the error is negative when the second offset is less than 0.

[0065] In this embodiment, the second offset endY-startY can be compared with 0, and a fixed parameter can be determined based on the comparison result. The fixed parameter can be 1 or -1. Then, the sign of the error is determined by multiplying the fixed parameter by the error. When the second offset is greater than 0, the error is positive; when the second offset is less than 0, the error is negative.

[0066] In step S330, the component of the screen coordinates of the center point of the operation area in the second direction is summed with the error to determine the second summation result.

[0067] The second summation result can be obtained by adding the components of the screen coordinates of the center point of the operation area in the secondary direction to the error.

[0068] In step S340, the second summation result is confined to a second closed interval, the endpoints of which are 0 and the side length of the operation region, respectively.

[0069] In this step, the second summation result can be confined to a second closed interval, with endpoints 0 and the side length w of the operating region, respectively. The second closed interval can be represented as [0, w]. Based on this, the second summation result can be restricted to the second closed interval [0, w].

[0070] In step S350, the second summation result after the limit is used as the secondary direction screen coordinate.

[0071] Furthermore, the second summation result after the limit can be used as the secondary direction screen coordinate. For example, when the second summation result is less than 0, the secondary direction screen coordinate is 0. When the second summation result is within the second closed interval [0, w], the secondary direction screen coordinate is the second summation result. When the second summation result is greater than w, the secondary direction screen coordinate is the side length w of the operating area.

[0072] In this embodiment, by distinguishing between primary and secondary motion directions, the primary direction screen coordinates of the primary motion direction and the secondary direction screen coordinates of the secondary motion direction are determined, thereby accurately determining the position of the virtual joystick's center and improving precise control. Furthermore, the secondary direction screen coordinates in the secondary motion direction can be determined based on a second offset and an operation error threshold. An operation error threshold is introduced when determining the secondary direction screen coordinates, achieving motion filtering in the secondary motion direction and avoiding jitter in that direction.

[0073] In step S160, the display position of the virtual joystick is updated based on the primary and secondary screen coordinates, and the target object corresponding to the virtual joystick is controlled.

[0074] In this embodiment of the disclosure, after determining the primary direction screen coordinates of the primary motion direction and the secondary direction screen coordinates of the secondary motion direction, the display position of the virtual joystick can be updated according to the primary and secondary direction screen coordinates to draw the virtual joystick graphic at that display position. Based on this, the movement of the target object controlled by the virtual joystick can be controlled according to the display position.

[0075] For example, the primary directional screen coordinate can be stickX, and the secondary directional screen coordinate can be stickY. The display position of the virtual joystick can be updated based on (stickX, stickY).

[0076] It should be noted that when the main direction of motion is the Y direction, the method for determining the screen coordinates of the main direction and the screen coordinates of the secondary direction is the same as the step in step S150 above, and will not be repeated here.

[0077] In this embodiment, the screen coordinates of the secondary direction are determined based on the operation error threshold, and de-jitter calculation is performed on the secondary motion direction to avoid the problem of large fluctuations in the output value, thereby improving the control accuracy and stability.

[0078] After determining the display position of the virtual joystick, a vibration motor can be driven to vibrate, simulating the operation of a real joystick and improving the user experience. For example, the vibration intensity can be determined based on the primary and secondary screen coordinates, and the vibration motor can be controlled to vibrate according to this intensity. In some embodiments, the primary and secondary screen coordinates can be normalized to obtain primary and secondary control values. Specifically, the difference between the primary screen coordinates and the screen coordinates of the center point of the operating area in the primary direction can be calculated, and the ratio of this difference to the screen coordinates of the center point of the operating area in the primary direction can be used as the primary control value. The determination of the secondary control value is the same as that of the primary control value, and will not be repeated here. Based on the normalization process, the primary and secondary screen coordinates stickX / stickY can be normalized and converted into primary and secondary control values ​​within the range [-1,1] for output to control the vibration intensity of the vibration motor.

[0079] If neither the primary nor secondary screen coordinates are zero, the vibration intensity can be determined based on the maximum value of the primary and secondary control values. If either the primary or secondary screen coordinates is zero, the vibration intensity can be determined based on the non-zero value of either the primary or secondary control value. By determining the vibration intensity of the vibration motor based on the primary and secondary screen coordinates, the virtual joystick based on the user interface can achieve a control feel close to that of a physical joystick, enhancing the realism of operation and improving the user experience.

[0080] After detecting a movement event acting on the operating area, a lift event can trigger the termination of control over the virtual joystick. A lift event can be a finger lift event, represented by `ACTION_UP`. Upon detecting a lift event, the virtual joystick can be controlled to return at a constant speed from its position indicated by the primary and secondary screen coordinates to the center point of the operating area; that is, the virtual joystick can be controlled to return at a constant speed from its display position to the center point of the operating area. It should be noted that the virtual joystick can return to the center point of the operating area through multi-frame animation.

[0081] For example, a first return value for the return operation can be determined based on the primary direction screen coordinates and the number of return cycles, and a second return value can be determined based on the secondary direction screen coordinates and the number of return cycles. The number of return cycles refers to the preset number of coordinate updates required to uniformly return from the positions represented by the primary and secondary direction screen coordinates to the center point of the operation area. The number of return cycles describes the number of animation frames required to return to the center point of the operation area. The number of return cycles can be determined according to actual needs, such as 10 or 20 times. The first return value represents the value that needs to be subtracted from each frame of animation during the return from the primary direction screen coordinates to the center point of the operation area, i.e., the value that needs to be subtracted for each return operation. The second return value represents the value that needs to be subtracted from each frame of animation during the return from the secondary direction screen coordinates to the center point of the operation area.

[0082] Based on this, the first return value is subtracted from the screen coordinates of the primary direction to obtain the first temporary coordinate component; the second return value is subtracted from the screen coordinates of the secondary direction to obtain the second temporary coordinate component, so as to perform a return operation and update the display position according to the first temporary coordinate component and the second temporary coordinate component, until the number of times the return operation is executed reaches the return count, so as to return the virtual joystick from that position to the center point of the operation area.

[0083] For example, after subtracting the first return value from the primary direction screen coordinates to obtain the first temporary coordinate component, and subtracting the second return value from the secondary direction screen coordinates to obtain the second temporary coordinate component, the display position of the virtual joystick's center can be updated to the first and second temporary coordinate components. Further, the first return value can be subtracted from the first temporary coordinate component, and the second temporary coordinate component can be subtracted from the second temporary coordinate component, continuing to update the display position of the virtual joystick's center until the number of return operations reaches the required number of returns, at which point the virtual joystick is returned from the display position determined based on the movement event to the center point of the operating area.

[0084] It should be noted that after performing a return operation, there can be a preset delay before the next return operation. The preset delay can be 16ms or other values; no specific limitation is made here. By controlling the virtual joystick to automatically and uniformly return to the center point of the operating area, the automatic centering operation of a real physical joystick is simulated, improving the user experience.

[0085] Figure 4 The flowchart illustrating the processing operations corresponding to the type of touch event is shown in the image. (See reference...) Figure 4 As shown, the main steps include:

[0086] Step S410: A touch event is received. The touch event includes the event type and the current touch point coordinates endX / endY.

[0087] Step S420: Determine the type of touch event.

[0088] Step S430: When the touch event is a press event, execute step S432.

[0089] Step S432: Determine whether the distance from the first contact point coordinates endX / endY in both directions to the screen coordinates center of the operation area when the press event occurs exceeds the operation error threshold THRESHOLD; if not, proceed to step S434; if yes, end.

[0090] Step S434: The coordinates of the first contact point of the press event are the coordinates of the current contact point of the press event, endX / endY.

[0091] Step S440: When the touch event is a movement event, execute step S442.

[0092] Step S442: Determine whether the coordinates startX and startY of the first contact point of the pressed event are both 0; if not, proceed to step S444; if yes, end.

[0093] Step S444: Calculate the primary direction screen coordinate stickX for the primary motion direction and the secondary direction screen coordinate stickY for the secondary motion direction corresponding to the movement event.

[0094] Step S450: When the touch event is a lift event, execute step S452.

[0095] Step S452: Control the virtual joystick to return to the center point of the operation area at a constant speed within a certain time range.

[0096] Figure 4 The technical solution provided in the article can accurately determine the position of the virtual joystick by determining the primary direction screen coordinates of the main direction of movement and the secondary direction screen coordinates of the secondary direction of movement when the touch event is a movement event.

[0097] Figure 5 The diagram illustrates a flowchart for determining the primary direction screen coordinates of the primary motion direction and the secondary direction screen coordinates of the secondary motion direction based on movement events. (See reference...) Figure 5 As shown, the main steps include:

[0098] Step S502: Determine whether the coordinates startX and startY of the first contact point of the pressed event are both 0; if not, proceed to step S504; if yes, end.

[0099] Step S504: Calculate the first offset in the first direction and the second offset in the second direction.

[0100] Step S506: Determine the main direction of motion based on the first offset and the second offset.

[0101] Step S508: Determine the main motion direction type. If the main motion direction type is X-direction, proceed to step S510. If the main motion direction type is Y-direction, proceed to step S520.

[0102] Step S510: Calculate the main direction screen coordinates of the main motion direction Direction_X, for example, calculate stickX.

[0103] Step S512: If the absolute value of the first offset is greater than or equal to the absolute value of the second offset, determine whether the second offset is less than the operation error threshold; if yes, proceed to step S514; if no, proceed to step S516.

[0104] Step S514: Take the screen coordinates of the center point of the operation area in the secondary direction as the screen coordinate stickY in the secondary direction.

[0105] Step S516: Determine the secondary direction screen coordinate stickY based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the secondary direction.

[0106] Step S520: Calculate the main direction screen coordinates of the main motion direction Direction_Y, for example, calculate stickY.

[0107] Step S522: If the absolute value of the first offset is less than the absolute value of the second offset, determine whether the first offset is less than the operation error threshold; if yes, proceed to step S524; if no, proceed to step S526.

[0108] Step S524: Take the screen coordinate component of the center point of the operation area in the secondary direction as the screen coordinate stickX in the secondary direction.

[0109] Step S526: Determine the secondary direction screen coordinate stickX based on the first offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the secondary direction.

[0110] Step S530: Determine the display position of the virtual joystick based on the primary direction screen coordinates and the secondary direction screen coordinates.

[0111] Step S532: Normalize the screen coordinates of the primary direction and the screen coordinates of the secondary direction to obtain the primary direction control quantity and the secondary direction control quantity X / Y.

[0112] Step S534: Determine the vibration level based on the primary direction control value and the secondary direction control value, and control the vibration motor to vibrate according to the vibration level.

[0113] The technical solution in this embodiment of the present disclosure, by distinguishing between primary and secondary motion directions, outputs higher precision and smoother primary and secondary direction control quantities, and adds vibration feedback from the vibration motor, filtering out unreasonable jitter in finger touch control, simulating the operating feel and automatic centering action of a physical joystick, improving the accuracy and safety of controlling the drone through the virtual joystick, and enabling precise control of the drone's attitude.

[0114] In some embodiments of this disclosure, a virtual joystick control device is also provided, comprising: a joystick providing module for displaying a virtual joystick on an operating interface, the virtual joystick having an operating area; a contact point coordinate determining module for determining a first contact point coordinate of a press event acting on the virtual joystick and a second contact point coordinate of a movement event acting on the virtual joystick, the press event and the movement event belonging to the same operating cycle; an offset determining module for determining a first offset in a first direction and a second offset in a second direction based on the first contact point coordinate and the second contact point coordinate; a motion direction determining module for determining a primary motion direction and a secondary motion direction based on the first offset and the second offset; a screen coordinate determining module for determining the screen coordinates of the center of the virtual joystick in the primary direction of the primary motion direction and the screen coordinates in the secondary direction of the secondary motion direction, wherein the determination of the screen coordinates in the secondary direction is related to an operating error threshold; and a position determining module for updating the display position of the virtual joystick based on the screen coordinates in the primary direction and the screen coordinates in the secondary direction, and controlling the target object corresponding to the virtual joystick.

[0115] In one exemplary embodiment of this disclosure, the absolute value of the first offset is greater than or equal to the absolute value of the second offset, and the motion direction determination module is configured to perform: determining the first direction as the primary motion direction and determining the second direction as the secondary motion direction.

[0116] In an exemplary embodiment of this disclosure, the screen coordinate determination module includes: a first summation result determination module, configured to sum the component of the screen coordinates of the center point of the operation area in the first direction with the first offset to determine a first summation result; a limiting module, configured to limit the first summation result to a first closed interval, the endpoints of the first closed interval being 0 and the side length of the operation area, respectively; and a primary coordinate determination module, configured to use the limited first summation result as the screen coordinates in the primary direction.

[0117] In one exemplary embodiment of this disclosure, the screen coordinate determination module includes: a first determination module, configured to determine the component of the screen coordinates of the center point of the operation area in the second direction as the secondary direction screen coordinates when the second offset is less than or equal to the operation error threshold; and a second determination module, configured to determine the secondary direction screen coordinates based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the second direction when the second offset is greater than the operation error threshold.

[0118] In one exemplary embodiment of this disclosure, the second determining module includes: an error determining module, configured to determine the error between the absolute value of the second offset and the operation error threshold; an error sign determining module, configured to determine the sign of the error based on a comparison between the second offset and 0, wherein the error is positive when the second offset is greater than 0 and negative when the second offset is less than 0; a second summation result determining module, configured to sum the component of the screen coordinates of the center point of the operation area in the second direction with the error to determine a second summation result; a limiting module, configured to limit the second summation result to a second closed interval, the endpoints of the second closed interval being 0 and the side length of the operation area, respectively; and a secondary coordinate determining module, configured to use the limited second summation result as the secondary direction screen coordinates.

[0119] In one exemplary embodiment of this disclosure, the device further includes: a normalization module, configured to normalize the primary direction screen coordinates and the secondary direction screen coordinates to determine the primary direction control quantity and the secondary direction control quantity; and a vibration module, configured to determine the vibration degree of the vibration motor based on the primary direction control quantity and the secondary direction control quantity, and control the vibration motor to vibrate according to the vibration degree.

[0120] In one exemplary embodiment of this disclosure, the device further includes: a centering module, configured to control the virtual joystick to return at a constant speed from the display position to the center point of the operation area in response to a touch event acting on the virtual joystick being a lift event.

[0121] In an exemplary embodiment of this disclosure, the centering module includes: a return value determination module, configured to determine a first return value for the return operation based on the primary direction screen coordinates and a preset number of return operations, and to determine a second return value for the return operation based on the secondary direction screen coordinates and the number of return operations; and a virtual joystick return module, configured to subtract the first return value from the primary direction screen coordinates to obtain a first temporary coordinate component, and subtract the second return value from the secondary direction screen coordinates to obtain a second temporary coordinate component, so as to perform the return operation, and update the display position based on the first temporary coordinate component and the second temporary coordinate component, until the number of times the return operation is executed reaches the number of return operations.

[0122] It should be noted that the specific details of each part of the virtual joystick control device have been described in detail in some embodiments of the corresponding method. For details that are not disclosed, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0123] Exemplary embodiments of this disclosure also provide an electronic device. This electronic device can be the aforementioned terminal device, on which software for performing the aforementioned methods can be installed. Generally, the electronic device may include a processor and a memory, the memory storing executable instructions of the processor, and the processor configured to execute the aforementioned virtual joystick control method by executing the executable instructions. Furthermore, the electronic device may also include a display for displaying an operating interface.

[0124] The electronic device is described below as an example in the form of a general-purpose computing device. This electronic device is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0125] The components of an electronic device may include, but are not limited to: at least one processing unit, at least one storage unit, a bus connecting different system components (including storage units and processing units), and a display unit.

[0126] The storage unit stores program code that can be executed by the processing unit, causing the processing unit to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit can perform actions such as... Figure 1 The steps are shown in the figure.

[0127] The storage unit may include readable media in the form of volatile storage units, such as random access memory (RAM) and / or cache storage units, and may further include read-only memory (ROM).

[0128] The storage unit may also include a program / utility having a set (at least one) of program modules, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0129] A bus can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus that uses any of the various bus structures.

[0130] The electronic device can also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be achieved through input / output (I / O) interfaces. Furthermore, the electronic device can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0131] It should be noted that some embodiments of this disclosure also provide a computer program product, which includes a computer program that implements the above-described method when executed by a processor.

[0132] In one embodiment, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing the computer program, such as read-only memory, NAND flash memory, etc.

[0133] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0134] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0135] Computer programs can be carried or transmitted via signals such as electrical, magnetic, optical, electromagnetic, and infrared rays. Electronic devices can convert signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to be executed by the processor of the electronic device) the method steps of various exemplary embodiments of this disclosure.

[0136] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0137] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0138] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A virtual rocker control method, characterized by, include: A virtual joystick is displayed on the user interface, and the virtual joystick has an operating area; Determine the coordinates of the first contact point of a press event acting on the virtual joystick and the coordinates of the second contact point of a move event acting on the virtual joystick, wherein the press event and the move event belong to the same operation cycle; Based on the coordinates of the first contact point and the coordinates of the second contact point, determine the first offset in the first direction and the second offset in the second direction; Based on the first offset and the second offset, the primary motion direction and the secondary motion direction are determined; The screen coordinates of the center of the virtual joystick in the primary direction of the main movement direction and the screen coordinates in the secondary direction of the secondary movement direction are determined, wherein the determination of the screen coordinates in the secondary direction is related to the operation error threshold. The display position of the virtual joystick is updated based on the primary and secondary screen coordinates, and the target object corresponding to the virtual joystick is controlled.

2. The virtual joystick control method according to claim 1, characterized in that, The absolute value of the first offset is greater than or equal to the absolute value of the second offset. Determining the primary and secondary motion directions based on the first and second offsets includes: The first direction is determined as the primary direction of motion, and the second direction is determined as the secondary direction of motion.

3. The virtual joystick control method according to claim 2, characterized in that, Determining the screen coordinates of the center of the virtual joystick in the primary direction of motion and the secondary direction of motion includes: The screen coordinates of the center point of the operation area in the first direction are summed with the first offset to determine the first summation result; The first summation result is confined to a first closed interval, the endpoints of which are 0 and the side length of the operation region, respectively. The first summation result after the limit is used as the screen coordinate of the main direction.

4. The virtual joystick control method according to claim 2, characterized in that, Determining the screen coordinates of the center of the virtual joystick in the primary direction of motion and the secondary direction of motion includes: If the second offset is less than or equal to the operation error threshold, the component of the screen coordinates of the center point of the operation area in the second direction is determined as the screen coordinates in the secondary direction. If the second offset is greater than the operation error threshold, the secondary direction screen coordinates are determined based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the second direction.

5. The virtual joystick control method according to claim 4, characterized in that, Determining the secondary direction screen coordinates based on the second offset, the operation error threshold, and the component of the screen coordinates of the center point of the operation area in the second direction includes: Determine the error between the absolute value of the second offset and the operational error threshold; The sign of the error is determined based on the comparison between the second offset and 0, wherein the error is positive when the second offset is greater than 0, and negative when the second offset is less than 0. The component of the screen coordinates of the center point of the operation area in the second direction is summed with the error to determine the second summation result; The second summation result is confined to a second closed interval, the endpoints of which are 0 and the side length of the operation region, respectively. The second summation result after the limit is used as the screen coordinate of the secondary direction.

6. The virtual joystick control method according to claim 1, characterized in that, The method further includes: The primary direction screen coordinates and the secondary direction screen coordinates are normalized to determine the primary direction control quantity and the secondary direction control quantity. The vibration level of the vibration motor is determined based on the primary and secondary directional control values, and the vibration motor is controlled to vibrate according to the vibration level.

7. The virtual joystick control method according to claim 1, characterized in that, The method further includes: In response to a lift event of the virtual joystick, the virtual joystick is controlled to return at a constant speed from the display position to the center point of the operation area.

8. The virtual joystick control method according to claim 7, characterized in that, The control of the virtual joystick to return at a constant speed from the display position to the center point of the operation area includes: The first return value for the return operation is determined based on the main orientation screen coordinates and the preset number of return attempts. The second return value of the return operation is determined based on the secondary orientation screen coordinates and the number of return attempts. Subtract the first returned value from the primary orientation screen coordinates to obtain a first temporary coordinate component, and subtract the second returned value from the secondary orientation screen coordinates to obtain a second temporary coordinate component. Perform the return operation and update the display position according to the first and second temporary coordinate components until the number of times the return operation is performed reaches the number of returns.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the virtual joystick control method according to any one of claims 1-8.

10. An electronic device, characterized in that, include: processor; as well as Memory, used to store executable instructions; The processor is configured to execute the virtual joystick control method of any one of claims 1-8 by executing the executable instructions.

Images