An interactive control method and electronic device

Abstract

The application relates to the technical field of interactive control, in particular to an interactive control method and an electronic device. The method comprises the following steps: in the case that an interaction mode is a multi-screen linkage interaction mode, detecting a touch point of a first finger on a first screen to obtain first touch information; detecting a touch point of at least one second finger on a touch module to obtain second touch information; and controlling a controlled object displayed on the first screen to execute a preset operation task based on the first touch information and the second touch information. Therefore, the application can provide a more natural, efficient and low-burden large-screen human-computer interaction experience.

Description

Technical Field

[0001] This application relates to the field of interactive control technology, and in particular to an interactive control method and electronic device. Background Technology

[0002] With the evolution of smartphone form factors, X-shaped foldable phones have become a significant market trend due to their large screen size when unfolded (typically over 8 inches diagonally). While offering a wider display space and multitasking capabilities, these devices also present significant challenges in human-computer interaction.

[0003] Traditional candybar phones primarily rely on single-handed or double-handed thumb operation for touch interaction, with the "hot zone" concentrated in the lower half of the screen, allowing users to easily reach most of the area with their thumbs. However, when foldable phones are fully unfolded, users typically hold them with both hands, making it difficult for the thumbs to reach the upper middle area of ​​the screen, thus significantly limiting the original interaction mode. (1) Difficulty in accessing: The screen is too large, making some functional areas untouchable. Users must change their grip or use the other hand to operate, which reduces convenience. (2) Decreased efficiency: Frequent large hand movements affect the smoothness of operation and disrupt the continuity of use; (3) Interaction incompatibility: The original thumb-centered operation logic is no longer compatible with the current operation habits that mainly involve the index finger and other forefingers; (4) Limited function expansion: Traditional single-point touch methods such as tapping and swiping have a limited input dimension and are difficult to meet the high-precision control requirements in complex scenarios. Summary of the Invention

[0004] In view of this, embodiments of this application provide an interactive control method and electronic device, which can effectively solve the problems of low efficiency and heavy burden in human-computer interaction in the prior art.

[0005] In a first aspect, embodiments of this application provide an interactive control method applicable to an electronic device, the electronic device including a first screen and a touch module; the first screen and the touch module are set based on a preset angle; the method includes: When the interaction mode is a multi-screen linkage interaction mode, the touch point of the first finger on the first screen is detected to obtain the first touch information; the touch point of at least one second finger on the touch module is detected to obtain the second touch information. Based on the first touch information and the second touch information, the controlled object displayed on the first screen is controlled to perform a preset operation task.

[0006] Secondly, embodiments of this application provide an electronic device, which includes a first screen and a touch module; the first screen and the touch module are set at a preset angle; the electronic device also includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement an interactive control method provided in the first aspect of this application and interact with the user.

[0007] The embodiments of this application have the following beneficial effects: This application relates to an electronic device, which includes a first screen and a touch module. The first screen and the touch module are set at a preset angle. In a multi-screen interactive mode, this application detects the touch point of a first finger on the first screen to obtain first touch information; detects the touch point of at least one second finger on the touch module to obtain second touch information; and controls the controlled object displayed on the first screen to perform a preset operation task based on the first and second touch information. Therefore, this application can provide a more natural, efficient, and less burdensome large-screen human-computer interaction experience. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This diagram illustrates a structural schematic of an electronic device in the unfolded state of its folded screen, according to an embodiment of this application. Figure 2 A flowchart of an interactive control method according to an embodiment of this application is shown; Figure 3 Flowchart of a click process in a cursor-linked scenario in the interactive control method of this application embodiment; Figure 4 Flowchart of the cursor linkage scenario in the interactive control method of this application embodiment; Figure 5 A flowchart illustrating the smearing function implemented by the forward-finger dragging operation in a cursor-linked scenario in the interactive control method of this application embodiment; Figure 6 A flowchart of the cursor linkage scenario drag operation in the interactive control method of this application embodiment; Figure 7 A schematic diagram of push-pull operation gestures in the interactive control method of this application embodiment; Figure 8A schematic diagram of the anchor point sliding gesture in the interactive control method of this application embodiment; Figure 9 A schematic diagram of a two-finger movement gesture in the interactive control method of this application embodiment; Figure 10 A schematic diagram of the operation steps of the split-screen task in the interactive control method of this application embodiment; Figure 11 A schematic diagram of one operation step of the split-screen task in the interactive control method of this application embodiment; Figure 12 A schematic diagram illustrating another operation step of the split-screen task in the interactive control method of this application embodiment; Figure 13 A schematic diagram of an operation scenario for UI control interaction tasks in the interactive control method of this application embodiment; Figure 14 A schematic diagram of an operation scenario for a text interaction task in the interactive control method of this application embodiment; Figure 15 This application's embodiment of the interactive control method illustrates another operational scenario for text interaction tasks. Detailed Implementation

[0010] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0011] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0012] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0013] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0014] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0015] Figure 1 This illustration shows a structural diagram of an electronic device in its unfolded state according to an embodiment of this application. Exemplarily, the electronic device includes a first screen and a touch module; the first screen and the touch module are set at a preset angle. This preset angle setting includes, but is not limited to, a back-to-back orientation, a perpendicular orientation, an obtuse angle, an acute angle, etc. The touch module is a touch-sensing device, such as, but not limited to, a touchscreen. The first screen includes, but is not limited to, a foldable screen, a single screen, etc.

[0016] In this embodiment, the foldable screen is combined into a first screen when unfolded, and the first screen and the touch module are arranged back to back; the electronic device also includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the computer program to implement the interactive control method in the embodiments of this application.

[0017] This electronic device includes, but is not limited to, foldable phones, foldable tablets, and single-screen phones.

[0018] The interactive control method will be described below with reference to some specific embodiments. In this embodiment, the first screen is a front screen, and the touch module is a rear touch module that is positioned opposite to the front screen.

[0019] Figure 2 A flowchart of an interactive control method according to an embodiment of this application is shown. Exemplarily, the interactive control method includes the following steps: S100, in the case of multi-screen linkage interaction mode, detects the touch point of the first finger on the first screen to obtain the first touch information; detects the touch point of at least one second finger on the touch module to obtain the second touch information.

[0020] This application uses a foldable screen phone as an example to illustrate the interactive control method of this application embodiment. The first finger is the thumb, and the second finger is any finger other than the thumb. Based on the angle of the finger operating the first screen and the touch module, the first finger is also called the forward finger, and the second finger is also called the backward finger. The second finger is the finger that performs operations on the touch module; the first finger includes, but is not limited to, the ring finger, thumb, index finger, and middle finger.

[0021] like Figure 1 As shown, the first screen faces the user, while the touch module is the touchscreen facing away from the user and accessible to the fingers. In the unfolded state, the user holds the foldable phone with both hands, with the thumb naturally touching the first screen and the index finger (or second finger) naturally touching the touch module. In essence, the thumb touches the first screen, and the index finger touches the touch module.

[0022] The first finger is the finger that performs operations on the first screen. The first finger includes, but is not limited to, the thumb, index finger, middle finger, and ring finger.

[0023] To facilitate the calculation of the position coordinates of the first finger and the second finger in the embodiments of this application, the coordinates are all referred to in the preset projection coordinate system.

[0024] Exemplary interaction modes include single-screen interaction mode and multi-screen linkage interaction mode; the multi-screen linkage interaction mode receives the user's touch operation through two screens, the first screen and the touch module; the single-screen interaction mode receives the user's touch operation through the first screen.

[0025] In one implementation, the method further includes: S010, detect the touch point of the first finger on the first screen and obtain the first touch information; detect the touch point of the second finger on the touch module and obtain the second touch information; S020, determine the user's interaction mode based on the first touch information and the second touch information.

[0026] Further, in step S020, determining the user's interaction mode based on the first touch information and the second touch information includes: S021, if it is determined from the first touch information and the second touch information that the user has performed a single pinch operation, then the multi-screen linkage interaction mode is enabled and the single-screen interaction mode is disabled.

[0027] If, within the first time window, both the first and second fingers are detected to switch from a lifted state to a pressed state, then a single pinch operation is confirmed. The single-screen interaction mode is then deactivated after entering the multi-screen interactive mode.

[0028] In other words, this application also provides an anti-accidental touch mechanism to eliminate ambiguous misoperations and special situations involving gestures during daily use, in order to accurately identify user intent. When activating the multi-screen interactive mode, it is first necessary to ensure that the leading and trailing fingers simultaneously undergo a "from lift to press" state change, and the maximum time threshold for this state change is 0.2 seconds. This is to avoid accidental touches caused by the user already having the leading or trailing finger in a pressed state before activating the multi-screen interactive mode, or other scenarios that may lead to pinch-to-activate. Therefore, the multi-screen interactive mode is activated by a gesture with higher operational cost and clear differentiation (leading and trailing fingers lift and fall simultaneously).

[0029] Furthermore, this application embodiment also provides a conflict reconciliation function. During the multi-screen interactive mode, conflicts may exist between operation gestures (clicking and dragging in a cursor-linked scenario) and other conventional single-screen gestures (clicking, dragging, etc.). Therefore, in this application embodiment, when the multi-screen interactive mode is enabled, it is determined that the pinch gesture is enabled, and other single-screen interactive functions are disabled. When the function is disabled in the multi-screen interactive mode, it is determined that the pinch gesture is disabled, and other single-screen interactive functions are restored. This mechanism ensures that when using the multi-screen interactive mode, it will not interfere with or be accidentally triggered by other single-screen interactive functions, ensuring the independence and priority of the multi-screen interactive mode. This allows the gesture to perform some system-level functions or functions that override specific software interactions.

[0030] S022, if the two-finger touch state is in the single-finger press state or the multi-finger press state, then exit the multi-screen linkage interaction mode and enter the single-screen interaction mode.

[0031] In other words, when a leading or trailing finger changes state from being raised to being pressed down, or any other additional finger changes state from being pressed down, the "sub-interaction function" is turned off and the multi-screen interaction mode is exited (exiting the pinch-and-swipe judgment logic). The sub-interaction function is the interaction function supported in the multi-screen interaction mode.

[0032] Furthermore, the two-finger touch state is identified using the following method: If the distance the first finger moves on the first screen is less than the first preset threshold (e.g., 2mm), and the distance the second finger moves on the touch module is less than the second preset threshold (e.g., 2mm), then the two-finger touch state is no movement; wherein, the first preset threshold and the second preset threshold may be equal or unequal.

[0033] If the distance the first finger moves on the first screen is greater than the first preset threshold, and the distance the second finger moves on the touch module is less than the second preset threshold, then the two-finger touch state is a single-finger movement (forward finger movement). If the distance the first finger moves on the first screen is less than the first preset threshold, and the distance the second finger moves on the touch module is greater than the second preset threshold, then the two-finger touch state is a single-finger movement (the second finger moves). If the distance the first finger moves on the first screen is greater than the first preset threshold, and the distance the second finger moves on the touch module is greater than the second preset threshold, then the two-finger touch state is two-finger movement.

[0034] The movement state of the two fingers in a two-finger touch state is determined by calculating the distance the first and second fingers move in the projected coordinate system. There are four states in total: no movement of both fingers, forward finger movement, backward finger movement, and both fingers moving.

[0035] Two-finger touch states also include a pressed state and a released state. Therefore, the two-finger touch state between the first and second fingers is calculated based on the first and second touch information: If both the first and second fingers switch from the lifted state to the pressed state within the first time window (e.g., 0.2s), then the two-finger touch state is confirmed to be the two-finger pressed state.

[0036] The pressed state includes, but is not limited to, a finger touching the first screen or the touch module. The released state includes, but is not limited to, a finger not touching the first screen or the touch module. If, within the first time window, both the first and second fingers are detected to change from a pressed state to a released state, then the two-finger touch state is confirmed to be the two-finger released state. If, within the first time window, either the first finger or the second finger is detected to change from a lifted state to a pressed state, then the two-finger touch state is confirmed to be a single-finger pressed state. If, within the first time window, the first, second, and third fingers all change from a raised state to a pressed state, then the two-finger touch state is confirmed to be a multi-finger pressed state. In other words, when a preceding or following finger changes from a raised state, or any other additional finger changes from a pressed state, the multi-screen interaction mode is exited. This exit mechanism strictly requires that, while maintaining the multi-screen interaction function, the preceding and following fingers must simultaneously remain in a pressed state, and no other fingers can participate. Once any finger changes from a raised state, or any other additional finger participates in the interaction, the multi-screen interaction mode is exited.

[0037] If, within the third time window (1 second), the following state changes are detected in the two-finger touch state: two-finger lifted state, two-finger pressed state, two-finger lifted state, two-finger pressed state, and two-finger lifted state, then it is confirmed that the user has performed a double pinch operation.

[0038] If, within the first time window (e.g., 0.2s), both the first and second fingers are detected to switch from the lifted state to the pressed state, then a single pinch operation is confirmed to have been performed.

[0039] S200, based on the first touch information and the second touch information, controls the controlled object displayed on the first screen to perform a preset operation task.

[0040] Both the first and second touch information include, but are not limited to, multi-dimensional information such as position, angle, and touch state.

[0041] Understandably, in single-screen interaction mode, the first touch information is continuously detected, and the controlled object displayed on the first screen is controlled to perform preset operation tasks based on the first touch information.

[0042] Understandably, in the multi-screen interactive mode, the controlled object displayed on the first screen is controlled to perform a preset operation task based on the first touch information and the second touch information.

[0043] Examplely, step 200 includes: S210, calculate the specified attribute between the first finger and at least one second finger based on the first touch information and the second touch information.

[0044] Exemplarily, the specified attributes include: position data, touch state, and movement angle. Understandably, for each second finger, the position data, touch state, and movement direction between the first and second fingers are calculated based on the first and second touch information. For example, the position data, touch state, and movement angle between the first finger and a second finger are calculated based on the first and second touch information respectively, thus obtaining the corresponding position data, touch state, and movement direction of the two fingers.

[0045] S220 determines the target function type of the preset operation task based on the number of the second finger.

[0046] In other words, the first screen can be controlled with one finger, and the touch module can be operated with one, two, three, or four fingers. Different numbers of second fingers can correspond to different functions. For example, the first finger on the first screen and one second finger on the touch module are used to perform the first type of operation task; the first finger on the first screen and two second fingers on the touch module are used to perform the second type of operation task; and the first finger on the first screen and three second fingers on the touch module are used to perform the third type of operation task.

[0047] S230, determine the interaction operation type and gesture recognition parameters based on at least some of the specified attributes.

[0048] As an example, the interaction type and gesture recognition parameters are determined based on at least one of two-finger position data, two-finger touch state, and two-finger movement direction.

[0049] S240 controls the controlled object displayed on the first screen to perform a preset operation task corresponding to the target function type based on the interaction operation type and gesture recognition parameters.

[0050] To reconcile conflicts between functions, in the multi-screen interactive mode, each of the three attributes used to identify gesture movement logic (two-finger position data, two-finger touch state, and two-finger movement direction) is mapped to different attributes of the controlled object to achieve different sub-interaction functions. This ensures that there are no conflicts between each combination of attributes.

[0051] In one implementation, the controlled object displayed on the first screen is controlled to perform a preset operation task corresponding to the target function type based on the interaction operation type and gesture recognition parameters, including: If the interaction type is a long pinch operation, the first screen is controlled to display a preset cursor to enter the cursor linkage scene. The display position of the preset cursor is controlled according to the displacement of the second finger. The attributes of the controlled object are controlled based on the single-finger interaction operation performed by the first finger on the first screen. The gesture recognition parameters include the displacement of the second finger. Cursor linkage refers to the interactive function achieved by combining the movement of the cursor with the movement of the cursor by the back finger and the single-finger interactive operation of the front finger.

[0052] In other words, in cursor-linked scenarios, the display position of the preset cursor is controlled by the displacement of the second finger, while the attributes of the controlled object are controlled by the single-finger interaction operation performed by the first finger on the first screen. Since the second finger is behind the screen, the cursor-linked function in cursor-linked scenarios is also called the virtual thumb function.

[0053] This means that the initial state of the cursor-linked scene is that both the forward and backward fingers are pressed down. The preset cursor includes, but is not limited to, a circular cursor.

[0054] The cursor-linked operation is activated via a long pinch gesture. Once activated, a virtual thumb (the default cursor) appears on the first screen to simulate the position of a finger on a standard single-screen device. Users primarily control the virtual thumb's movement by moving the back finger and output commands to control the device by clicking or swiping with the front finger. This combined operation of the front and back fingers allows for the completion of specific tasks, providing greater flexibility, convenience, and operational possibilities. The virtual thumb's control logic inherits the original single-screen operation logic, including single and double clicks, dragging, and swiping. Furthermore, it expands the scenarios and functions by utilizing the flexibility of the two controls (front and back fingers).

[0055] Single-finger interaction operations include single-click, double-click, forward-finger drag, backward-finger drag, and flick. Controlling the properties of a controlled object using single-finger interaction operations performed by the first finger on the first screen includes: (1) Clicking in a cursor-linked scenario: When performing a return task, such as Figure 3 As shown in 'a', by performing a long pinch operation, you enter the cursor-linked scene, displaying a virtual thumb (blue circle in the image); as... Figure 3 As shown in b, the position of the virtual thumb is controlled by moving the position of the index finger to reach the target position; for example... Figure 3 As shown in c, when the current finger undergoes a specific state change from "lifting up to pressing down and then lifting up again", a click operation is performed on the controlled object under the virtual thumb position to control the attributes of the controlled object, such as controlling the selection of the "back" button.

[0056] (2) Double-click in cursor linkage scenario: The position of the virtual thumb is controlled by moving the position of the next finger. When the current finger undergoes a specific state change of "from lifting to pressing, then lifting, then pressing, and then lifting", and the total duration of this specific state change is less than 1 second, a double-click operation is performed on the controlled object under the virtual thumb position to control the attributes of the controlled object.

[0057] (3) Forward dragging in cursor-linked scenarios: When using a virtual thumb to select text, such as Figure 4 As shown in 'a', by performing a long pinch operation, you enter the cursor-linked scene, displaying a virtual thumb (blue circle in the image); as... Figure 4 As shown in b, the position of the virtual thumb is controlled by moving the position of the index finger; for example... Figure 4 As shown in Figure 'c', when the rear finger is pressed, the movement vector of the front finger (drag operation of the front finger) is detected. The position of the virtual thumb follows the position change of the front finger (including direction and distance), and the change of the virtual thumb's position controls the attribute change of the controlled object (e.g., the range of selected text content). In other words, the position of the virtual thumb is controlled by moving the front finger, and the attribute of the controlled object can be controlled based on the position of the virtual thumb. When the front finger is lifted, the detection of the rear finger's movement stops, and the virtual thumb's position stops following the rear finger's movement.

[0058] For example, when using the forward finger drag to achieve the smearing function, such as Figure 5 The 'a' in the diagram represents the existing smearing method; a long pinch operation enters the cursor-linked scene, displaying a virtual thumb (blue circle in the image); as shown... Figure 5 As shown in b, the position of the virtual thumb is controlled by moving the position of the leading finger (leading finger drag operation) to adjust the cursor to the target position. Figure 5As shown in 'c', when the forehand finger performs a drag operation, a smearing line is generated based on the dragging marks, and then the forehand finger is dragged to smear the line.

[0059] (4) Cursor-linked dragging in scenarios: When performing a content selection task, such as... Figure 6 As shown in 'a', by performing a long pinch operation, you enter the cursor-linked scene, displaying a virtual thumb (blue circle in the image); as... Figure 6 As shown in b, the position of the virtual thumb is controlled by moving the position of the back finger (back finger drag operation) to adjust the cursor to the target position. Figure 6 As shown in c, when the current finger is in the pressed state, the position of the virtual thumb is detected, and the properties of the controlled object are controlled according to the position of the virtual thumb. For example, a rectangular selection box is drawn with the target position of the cursor as the starting point and the final position of the virtual thumb after the drag operation of the next finger as the ending point.

[0060] (5) Cursor-linked swiping: The position of the virtual thumb is controlled by moving the position of the next finger, and the list is selected by clicking. When the current finger is pressed, the movement speed of the previous finger is detected, and the movement speed of the previous finger is used to control the sliding speed of the preset list. Specifically, the position of the virtual thumb is controlled by moving the position of the previous finger, and the position of the virtual thumb can control the position of the list. At the same time, the movement speed of the previous finger will give the list the sliding speed. When the previous finger is lifted, the detection of the movement speed of the previous finger stops, the use of the movement speed of the previous finger to control the sliding speed of the preset list stops, and the position of the previous finger stops controlling the position of the virtual thumb. At the same time, the final movement speed of the previous finger will give the list the sliding speed. At this time, the speed of the list will change the position of the list items with the accumulation of time, and at the same time, the speed gradually tends to a stable state with the change of time.

[0061] Furthermore, if the interaction type is a rubbing operation, then the attributes of the controlled object within the operation task are controlled based on the distance index generated by the rubbing operation or the action of the rubbing operation; in other words, if the interaction type is a rubbing operation, then the attributes of the controlled object within the operation task are controlled based on the distance index generated by the rubbing operation, or if the interaction type is a rubbing operation, then the attributes of the controlled object within the operation task are controlled based on the action of the rubbing operation. The gesture recognition parameters include a distance index; the distance index includes a displacement difference vector or relative distance.

[0062] The attributes of the controlled object within the operation task are controlled based on the kneading action, including: when the kneading action ends, i.e., after the fingers are lifted, it indicates that the kneading action is complete, and the attributes of the controlled object within the operation task are triggered. The attributes of the controlled object within the operation task are controlled based on a distance indicator generated by the kneading action, including: if a set distance indicator is generated during the kneading process, the attributes of the controlled object within the operation task are triggered.

[0063] The displacement difference vector generated by the rubbing operation is the difference vector between the displacement vector of the first finger corresponding to the touch point and the displacement vector of the second finger corresponding to the touch point.

[0064] The relative distance generated by the rubbing operation is the relative distance between the touch point of the first finger and the touch point of the second finger after the rubbing is completed.

[0065] The properties of the controlled object within the operation task are controlled based on the displacement difference vector generated by the kneading operation, including: During the kneading process, the one-dimensional and two-dimensional properties of the controlled object are controlled based on the direction and magnitude of the displacement difference vector.

[0066] This method utilizes displacement difference vectors to control the properties of the controlled object. These properties include, but are not limited to, one-dimensional and two-dimensional variables. For example, when controlling a one-dimensional variable using a kneading operation, if the displacement difference vector is upward, the controlled object moves upward; if it is downward, it moves downward. When the controlled object's properties are discrete values, each time the displacement difference vector length increases or decreases by 1 cm, the controlled object's property increases or decreases by one unit.

[0067] For example, when a kneading operation controls a two-dimensional variable of a controlled object, the changes in the displacement difference vector along the X and Y axes correspond to the control of two one-dimensional attributes, thus forming a control relationship. This control method is similar to the control method for one-dimensional attributes. The change along the X-axis controls the first-dimensional attribute, and the change along the Y-axis controls the second-dimensional attribute.

[0068] The attributes of the controlled object within the operation task can be either continuous or discrete. For example, continuous attributes include drag-and-drop lists, contact lists, and volume adjustments; discrete attributes include page turning. Furthermore, if the interaction type is a two-way alternating tap operation, the attributes of the controlled object within the operation task are controlled based on this two-way alternating tap operation.

[0069] Furthermore, if the interaction operation type is a push-pull operation, then the attributes of the controlled object within the operation task are controlled based on the displacement vector generated by the push-pull operation or the action of the push-pull operation; in other words, if the interaction operation type is a push-pull operation, then the attributes of the controlled object within the operation task are controlled based on the displacement vector generated by the push-pull operation, or if the interaction operation type is a push-pull operation, then the attributes of the controlled object within the operation task are controlled based on the action of the push-pull operation. The gesture recognition parameters include the displacement vector.

[0070] Push-pull operations involve pressing down two fingers together (pinch state) and then simultaneously moving those two fingers to control the attributes of the controlled object. The movement judgment logic in push-pull operations is the same as that for single-finger movement, thus allowing for common displacement-type interactive gestures such as dragging and flicking.

[0071] Push-pull operations include dragging, page turning, and flicking.

[0072] The drag-and-drop process involves using the positions of the forward and backward fingers (which are in the same position when pinched) to directly control the position of the controlled object. The X position of the forward and backward fingers corresponds to the X position of the controlled object, and the Y position corresponds to the Y position of the controlled object.

[0073] The page-turning process includes: When the current and next fingers are in a pinch state and the two-finger touch state is in a stationary state, if the two-finger touch state is detected to switch to a two-finger movement state, the movement speed of the two fingers (the movement speed of the front and back fingers) is detected, and it is determined whether the movement distance of the two fingers is greater than a preset second distance (e.g., 5cm). When the two-finger touch state switches to a two-finger lift state, if the final detected movement distance of the two fingers is greater than the preset second distance (e.g., 5cm), the attribute of the controlled object will switch to the next state value. After the two-finger touch state switches to a two-finger lift state, the detection of the movement distance and movement speed of the two fingers stops, and the accumulation of time begins. The final detected movement speed of the two fingers and the accumulated time are used to control the attribute of the controlled object. The attribute of the controlled object can be the page movement speed. The page speed will change the page position with the accumulation of time, and at the same time, the speed will gradually approach a stable state over time.

[0074] The swinging process includes: When the two-finger touch state is in the "two-finger pressed" state, the movement speed of the two fingers is detected. It is checked whether the movement speed exceeds a preset speed threshold. Once the movement speed exceeds the preset threshold (5cm / s), the two-finger touch state is detected, and the two-finger release state is detected. The controlled object's attribute will then switch to the next state. After the two-finger touch state switches to the release state, the detection of the two-finger movement speed and whether it exceeds the preset speed threshold stops. Time accumulation begins, and the final two-finger movement speed and accumulated time are used to control the page's position change. Simultaneously, the two-finger movement speed gradually decreases over time until it reaches a stable state.

[0075] Furthermore, if the interaction operation type is an anchor point sliding operation, the first attribute of the controlled object within the control operation task is determined based on the position of the fixed finger, and the second attribute of the controlled object within the control operation task is determined based on the displacement vector of the moving finger; the fixed finger and the moving finger are one of the first finger and the second finger, respectively, and the two are different from each other; the gesture recognition parameters include the position of the fixed finger and the displacement vector of the moving finger.

[0076] Anchor point swipe is a gesture where one finger (either the first or second finger) remains anchored (fixed), while the other finger slides. The type of interaction performed by the other finger, determined by the anchored finger, controls the properties of the controlled object. The initial state of an anchor point swipe is two fingers together, both pressed (two fingers pressed). If the second finger is fixed, the sliding operation by the first finger is called a front anchor point swipe; if the first finger is fixed, the sliding operation by the second finger is called a back anchor point swipe.

[0077] Demonstrating the front anchor point sliding operation: A function wheel unfolds around the starting point of the two-finger pinch. Select a function on the function wheel by moving the position of the front finger, and release the front finger to confirm and select the function.

[0078] As an example, the front anchor point sliding operation: Calculates the angle of the line formed by the pinching start point of the two fingers and the position of the front finger (the pinching start point is the default starting point of the line). When the angle changes clockwise, the proportion of the controlled object increases with the change in angle; when it changes counterclockwise, the proportion decreases with the change in angle.

[0079] Back anchor point sliding operation: When the current finger is in the pressed state, the starting point of the selection box is formed according to the position of the previous finger. The position of the virtual thumb is controlled by moving the position of the back finger, and a rectangular selection box is drawn with the position of the virtual thumb as the endpoint.

[0080] Furthermore, the method also includes: In a cursor-linked scenario, if the first finger is detected to lift up and remain in the lifted state, the display position of the preset cursor is adjusted based on the touch position of the first finger. If a second finger is detected to sequentially perform a lift-up and press-down operation within the first time window (e.g., 1 second), the preset cursor display position is adjusted according to the touch position corresponding to the press-down operation. In other words, when entering the cursor linkage scenario (virtual thumb function enabled), if the first finger is in a lifted state, it is determined that the user is adjusting the position of the virtual thumb. If the second finger changes its lifted state at this time, the duration of the second finger's lifted state is calculated. If the duration is less than 1 second, and the second finger changes its pressed state, it is determined that the user is moving the second finger multiple times to adjust the position of the virtual thumb, and the cursor linkage scenario is not exited (virtual thumb function is not disabled). If the second finger's lifted state is maintained for more than 1 second, the linkage "virtual thumb" is disabled (virtual thumb function is disabled).

[0081] And / or, in a cursor-linked scenario, if the second finger is detected to be in a raised state and the first finger switches from a raised state to a pressed state, then it is determined that the user has entered the single-screen interaction mode and exited the multi-screen linkage interaction mode and the cursor-linked scenario; in other words, when the second finger is raised, the first finger undergoes a state change from "raised to pressed", and it is determined that the user intends to use the first finger to perform a normal single-screen interaction function, then the user enters the single-screen interaction mode and exits the multi-screen linkage interaction mode and the cursor-linked scenario.

[0082] And / or, in the cursor linkage scenario, if a second finger is detected to perform a lift operation and remains in the lift state within the first time window, then the cursor linkage scenario is determined to be closed; in other words, the initial state of the cursor linkage scenario is the state of the front and back fingers being pressed down, and when the state of the back finger being lifted is maintained for more than 1 second, the linkage "virtual thumb" is closed.

[0083] And / or, in a cursor linkage scenario, if it is detected that the second finger remains pressed and the touch state of the first finger changes, then the cursor linkage scenario is determined to be closed. In other words, when the second finger is in a lifted state, if it is detected that the touch state of the first finger changes, such as when the first finger changes from "lifted to pressed (01)" or from "pressed to lifted (10)", then the cursor linkage scenario is exited.

[0084] When the back finger remains pressed down, the front finger changes to a lifted state. At this time, it is determined that the front finger continues to perform the cursor-linked interactive function (such as clicking or dragging the virtual thumb) and does not exit the virtual thumb function (cursor-linked scenario).

[0085] In one implementation, the first touch information includes touch state information; The interaction type and gesture recognition parameters are determined based on at least one of the following: two-finger position data, two-finger touch state, and two-finger movement direction, including: If, within the second time window (e.g., 0.2s), a touch state information indicating that the two-finger touch state has switched from the two-finger lifted state to the two-finger pressed state, and remains in the two-finger pressed state for a preset first duration (0.5s), then the interaction operation type is determined to be a long pinch operation. Furthermore, if at least one round of tapping is detected within the third time window, the interaction operation type is determined to be a two-sided alternating tapping operation; wherein, the tapping operation includes: the first finger switching from a raised state to a pressed state on the first screen and holding it for a preset time before switching back to the raised state; the second finger switching from a raised state to a pressed state on the touch module and holding it for a preset time before switching back to the raised state; or, the second finger switching from a raised state to a pressed state on the touch module and holding it for a preset time before switching back to the raised state; and the first finger switching from a raised state to a pressed state on the first screen and holding it for a preset time before switching back to the raised state.

[0086] For example, the two taps of the front and back fingers are not synchronized. While keeping the index finger and thumb at a preset distance (greater than the thickness of the phone), the device is then gently tapped back and forth between the two fingertips like a card.

[0087] Furthermore, if within the first time window (0.2s), the touch state information indicates that the displacement difference vector between the two fingers is greater than a preset first distance threshold (e.g., 3mm), and the two fingers move in opposite directions, then the interaction operation type is confirmed to be a rubbing operation. In other words, if the two fingers move, the distance is greater than 3mm, and the directions are opposite, then a rubbing operation is confirmed to have been performed.

[0088] This application provides an anti-shake mechanism. During multi-screen interactive operation, the forward or backward finger movement distance needs to exceed 3mm (minimum displacement threshold). Only when the movement distance exceeds this minimum displacement threshold are the activation conditions for various operations (rubbing, pushing, pulling, etc.) in the multi-screen interactive mode met. This is to avoid misjudgment of conditions caused by the user's unconscious slight movements during gripping, and to provide a certain buffer area.

[0089] The displacement difference vector between the two fingers, determined by the two-finger position data, includes: For example, the positions of the first finger's touch point on the first screen and the second finger's touch point on the touch module are projected onto the projection coordinate system, respectively. In the projection coordinate system, the first finger moves from the first position to the second position and the second finger moves from the third position to the fourth position, and the displacement difference vector between the first finger and the second finger is calculated.

[0090] For example, let the positions of the leading finger (e.g., the first finger) and the trailing finger (e.g., the second finger) in the projected coordinate system be (x, y) and (a, b), respectively. When the position of the leading finger moves from (x1, y1) to (x2, y2) and the position of the trailing finger moves from (a1, b1) to (a2, b2), the displacement difference vector between the leading and trailing fingers is calculated as (x2, y2) - (a2, b2) - [(x1, y1) - (a1, b1)].

[0091] Understandably, the kneading gesture is a hand gesture performed by the user through the relative movement of their forward and backward fingers. This gesture aims to simulate the kneading action in the real world; in other words, it involves manipulating and adjusting the one-dimensional properties of a controlled object by rotating the index finger and thumb.

[0092] The kneading gesture calculates the distance moved by two fingers to determine vector changes, and then controls the controlled object by varying the direction and length of these vectors. This gesture can also activate different functions and control relationships depending on the position of the fingers when they are placed.

[0093] Furthermore, if within the first time window (e.g., 0.2s), it is detected that the touch state information indicates that two fingers perform a pinch action and move within the pinch state, and the two fingers move in the same direction, then the interaction operation type is confirmed to be a push-pull operation; the pinch action is defined as the distance between the two fingers, determined based on the two-finger position data, decreasing to a preset second distance threshold (e.g., 2mm); the pinch state is defined as the distance between the two fingers, determined based on the two-finger position data, being less than the preset second distance threshold (e.g., 2mm). The first distance threshold is greater than the second distance threshold.

[0094] After a push-pull operation, if only one finger is lifted while the fingers are in the pinched state and the movement is complete (before both fingers are lifted), the multi-screen interaction mode (also known as pinch mode) is disengaged. Pinch mode is activated only when the front and back fingers are pressed simultaneously within a very small time window. Alternatively, after a push-pull operation, lifting only one finger can continue in pinch mode without disengaging, allowing for other operations such as forward / backward anchored swipes. This seamless transition between different pinch gestures also enhances the user experience.

[0095] like Figure 7 As shown in 'a', before performing the push-pull operation, the thumb and forefinger are pinched together and the thumb moves to the right (or along the X-axis); as... Figure 7 As shown in b, the index finger moves upward (or along the Y-axis), eventually as shown in Figure 1. Figure 7 As shown in c in the figure.

[0096] Furthermore, if within the first time window, the touch state information indicates that the two-finger touch state is in the state of single-finger press, and the other finger performs a swipe operation, then the interaction operation type is confirmed to be an anchor point swipe operation, such as... Figure 8As shown in c in the figure.

[0097] Furthermore, when the multi-screen interactive mode is enabled, each specific sub-interaction function (cursor-linked scene, anchor point sliding operation, rubbing, pushing and pulling) in the area pressed by the finger can be distinguished into multiple forms. This allows the same pinch operation logic to be combined with different operation areas (click, move, front touch module) to create different interactive effects.

[0098] Switching between sub-interaction functions mainly refers to the switching between kneading and pushing / pushing. In multi-screen interactive mode, the main operations include the linkage judgment between pinching and sliding. As can be seen from the mapping relationship of the linkage "pinch and slide judgment", the difference in the operation logic of kneading and pinching lies in the directional angle and the distance between the front and back fingers. From the user's intention perspective, once the directional angle changes, it must mean that the user wants to switch to the pinch sub-interaction function. Therefore, the decision to switch sub-functions is made by recognizing changes in the directional angle.

[0099] When the directional angle changes from opposite to the same or vice versa, the system will re-evaluate based on the attributes of the two fingers to switch between sub-interactions. This switching mechanism allows users to hesitate and pause within the corresponding sub-interaction while maintaining the current state. Switching will only occur when the directional angle of the movement changes—that is, when there is a fundamental change in the gesture's movement logic.

[0100] Furthermore, the first touch information includes a first movement direction angle of the first finger; the second touch information includes a second movement direction angle of the second finger.

[0101] As an example, the direction of movement of the two fingers is defined by the angle between the movement directions of the first and second fingers. The specific state of the movement direction of the two fingers is determined by judging the angle formed by the relative movement directions of the first and second fingers. This attribute has three states: same, opposite, and no direction.

[0102] Exemplarily, calculating the bi-finger movement direction between the first finger and the second finger based on the first touch information and the second touch information includes: If the angle between the first and second movement direction angles is greater than a preset angle, then the movement directions of the two fingers are confirmed to be opposite; the preset angle includes, but is not limited to, 90°. Figure 9 a in Figure 9 As shown in 'c', the index finger moves upwards, as... Figure 9 As shown in b, the thumb moves downwards, and thus the two fingers move in opposite directions.

[0103] If the first and second movement direction angles are less than the preset angle, then the movement directions of the two fingers are confirmed to be the same.

[0104] If either the first or the second movement direction angle is zero or empty, then the movement direction of the two fingers is confirmed to be zero.

[0105] The two-finger position data includes the distance between the two fingers and the displacement difference vector between the two fingers.

[0106] As an example, the following method is used to calculate the position data of two fingers: The positions of the first finger's touch point on the first screen and the second finger's touch point on the touch module are projected onto the projection coordinate system, respectively. That is, the positions of the front finger (first finger) and the back finger (second finger)'s touch points on the screen are projected onto the projection coordinate system. Based on the positions of the projected touch points of the first finger and the second finger respectively in the projection coordinate system, the distance between the two fingers is calculated, thus obtaining the distance between the two fingers.

[0107] Preset operation tasks include: split-screen tasks, UI control interaction tasks, and text interaction tasks. UI control interaction tasks include, but are not limited to, list interaction tasks, content editing tasks, and multi-view interaction tasks.

[0108] The interaction control method of this application embodiment will be described in detail below in conjunction with split-screen tasks, UI control interaction tasks, and text interaction tasks.

[0109] First: Multitasking management - split-screen tasks.

[0110] S610, enter split-screen mode. (Example) Figure 10 As shown in a, the first screen of the X-shaped foldable phone includes preset hot zones on both sides.

[0111] (1) When the X-shaped folding screen is not in split-screen mode (e.g. Figure 10 (b) in the example Figure 10 In the context of screen c, using push-pull operations to move a character from a preset hotspot on the left edge of the screen to the right or from a preset hotspot on the right edge of the screen to the left will bring up the split-screen selection bar (e.g., ...). Figure 10 In part d), the split-screen direction is determined based on the starting point of the push / pull, and the final split-screen effect is as follows: Figure 10 The 'e' in the middle.

[0112] (2) After bringing up the split-screen selection bar, you can use the front to swipe up and down to select the APP that needs to be split-screened. After clicking, the split-screen will be successfully completed.

[0113] (3) If you need to divide the screen into 3 APPs, repeat steps (1) and (2) on the basis of 2 APPs.

[0114] S620, split-screen switching.

[0115] (1) In the split-screen state of the X-shaped folding screen, such as Figure 11 a in Figure 11 As shown in b, rubbing operations are performed in different rubbing heat zones to select layers. Rubbing selects the multi-task split-screen layer (where the two-finger distance is mapped to the switching speed). This area will then cycle through the apps to switch between split-screen applications, such as... Figure 11 c in Figure 11 As shown in d, where Figure 11 The content displayed on the screen is to the right of the dashed line, while the operation gestures performed by the user and the schematic diagram of the screen hardware hot zones are to the left of the dashed line.

[0116] (2) The speed of the APP's circular scrolling is determined by the distance between the fingers before and after the rubbing operation. The APP will continuously scroll at a speed. The greater the distance between the two fingers, the faster the scrolling speed, and the smaller the distance between the two fingers, the slower the scrolling speed.

[0117] S630, split-screen adjustment, such as Figure 12 As shown, the operation process includes the following: (1) In the split-screen state of the X-shaped foldable screen, push and pull operations can be used to adjust the split-screen ratio of the APP or to activate the split-screen function (the adjustment boundary is determined by the starting point of the push and pull). For example, the size of the APP split-screen can be adjusted by pushing and pulling from the left / right edge or bottom of the screen to the right / left or top.

[0118] (2) The split-screen size of the APP is discrete, such as Figure 12 In this context, 'a' represents the pushing and pulling motion of the front and rear fingers. Figure 12 In this context, 'b' indicates that top-bottom split screen and left-right split screen correspond to different hot zones. Figure 12 The screen is divided into two parts based on the top and bottom, with each of the two apps occupying half of the screen. Figure 12 The screen is divided into two parts based on the top and bottom, with one app occupying 1 / 3 and the other app occupying 2 / 3. Figure 12 The 'e' in the image is divided into two screens, one for each app, with one app occupying 1 / 3 of the screen and the other for the other 2 / 3. Figure 12 The screen is divided into three sections based on the left and right sides, with each of the three apps occupying one-third of the screen.

[0119] S640, quick return.

[0120] (1) When performing push / pull gestures in the non-edge areas of different apps, the corresponding level of forward / backward movement will be performed according to the area.

[0121] Second: UI control interaction tasks, such as Figure 13 As shown, the operation process includes the following: When you make a rubbing gesture in the non-edge area of ​​different media playback apps, the media content will play at a corresponding speed based on the distance between the two fingers being rubbed.

[0122] Third: Text interaction tasks, such as Figure 14 , Figure 15 As shown, the operation process includes the following: like Figure 14 As shown in a, b, c, and d, the back finger moves while the front finger remains stationary. Moving the back finger controls cursor positioning, while moving the front finger controls the selection of text.

[0123] When making a rubbing gesture while editing text, the system determines the selection of the front or back lever based on the height of the two fingers being rubbed. Thumb on top is for operating the front lever, and thumb on the bottom is for operating the back lever.

[0124] like Figure 15 As shown in a, b, c, and d, when using push-pull operations to select text, the horizontal direction of the displacement of the push-pull operation maps to the selected rows, and the vertical direction of the displacement maps to the selected columns. Similarly, the horizontal direction of the displacement of the kneading operation maps to the selected rows, and the vertical direction of the displacement maps to the selected columns.

[0125] The pinch-to-interaction method proposed in this application utilizes the natural finger-to-finger gestures of users when holding a foldable phone in its unfolded state, constructing a completely new interaction system. Compared with existing technologies, this application has the following advantages: I. Significantly improves the accessibility and comfort of two-handed operation.

[0126] By utilizing the collaborative front touch module of the foldable screen, operations on the far-end areas of the screen that originally required significant hand movement to reach are mapped to natural pinching motions of the front and rear fingers on the front touch module. This allows users to complete various operations without changing their grip or stretching their arms while holding the foldable screen, greatly improving the smoothness of operation when the screen is unfolded.

[0127] II. Optimize and improve the efficiency of two-handed operation.

[0128] This interaction method fully adapts to the user's two-handed operating habits when the foldable screen is unfolded, providing a new dimension of interaction parallel to traditional screen interaction. It highly simulates intuitive actions in the physical world such as rotating knobs, pulling sliders, and operating joysticks, achieving a more natural and immersive human-computer interaction. It allows users to complete complex tasks through simple two-finger coordination while maintaining a stable grip, enabling smoother and more efficient multitasking and content browsing, thus fully leveraging the display advantages of a large screen.

[0129] III. Expanding the interactive boundaries of foldable screen devices.

[0130] Existing foldable screen devices typically rely on traditional single-screen gestures such as taps, swipes, and long presses for interaction. This application combines the front and back screens, giving foldable screens a unique dual-screen collaborative interaction mode. This breaks through the traditional single-screen touch logic of mobile phones, creating a differentiated interactive experience. This not only enhances the interactive value of foldable screens but also lays the foundation for subsequent multi-screen and multimodal human-computer interaction.

[0131] IV. Reduce learning costs and enhance interactive intuitiveness.

[0132] This application employs gestures that closely resemble real-world physical actions, such as rubbing, pushing, pulling, and joystick movements, allowing users to quickly learn and use the interface without complex learning curves. Compared to abstract or symbolic gesture operations, this application is based on "real-world physical metaphors," aligning with user intuition and enhancing the ease of learning and usability of the interaction.

[0133] V. Improve the control precision and continuity of complex tasks.

[0134] Pinch gestures possess high-dimensional input characteristics (direction, distance, force, speed, etc.), enabling the transmission of rich input information within limited gesture movements, thus achieving more precise and continuous control. This is particularly valuable in professional applications (such as music editing, video editing, map manipulation, 3D modeling, etc.), significantly improving operational efficiency and accuracy.

[0135] This application also provides an interactive control device. This interactive control device is applicable to an electronic device including a foldable screen and a touch module; in an unfolded state, the foldable screen is combined to form a first screen, and the first screen and the touch module are opposite each other; exemplaryly, the interactive control device includes: The detection module is used to detect the touch point of the first finger on the first screen to obtain the first touch information; and to detect the touch point of the second finger on the touch module to obtain the second touch information. The interaction mode determination module is used to determine the interaction mode adopted by the user based on the first touch information and the second touch information; The interactive control module is used to continuously detect the first touch information and the second touch information in interactive mode, and control the controlled object displayed on the first screen to perform preset operation tasks based on the first touch information and the second touch information.

[0136] It is understood that the device in this embodiment corresponds to the interactive control method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0137] This application also provides a terminal device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the functions of the various modules in the above-described interactive control method or interactive control device. Exemplarily, the terminal device is the above-described electronic device.

[0138] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0139] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0140] This application also provides a computer-readable storage medium for storing the computer program used in the aforementioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0142] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0143] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An interactive control method, characterized in that, An electronic device is applicable, the electronic device comprising a first screen and a touch module; the first screen and the touch module are set at a preset angle; the method includes: When the interaction mode is a multi-screen linkage interaction mode, the touch point of the first finger on the first screen is detected to obtain the first touch information; the touch point of at least one second finger on the touch module is detected to obtain the second touch information. For each second finger, a specified attribute between the first finger and the second finger is calculated based on the first touch information and the second touch information; the type of the specified attribute includes two-finger position data, two-finger touch state, and two-finger movement direction; The target function type of the preset operation task is determined based on the number of the second fingers; The interaction operation type and gesture recognition parameters are determined based on at least one of the two-finger position data, the two-finger touch state, and the two-finger movement direction. Based on the interaction operation type and gesture recognition parameters, the controlled object displayed on the first screen is controlled to perform a preset operation task corresponding to the target function type.

2. The interactive control method according to claim 1, characterized in that, The step of controlling the controlled object displayed on the first screen to perform a preset operation task corresponding to the target function type according to the interaction operation type and gesture recognition parameters includes at least one of the following: If the interaction operation type is a long pinch operation, then the first screen is controlled to display a preset cursor to enter the cursor linkage scene, the display position of the preset cursor is controlled according to the displacement of the second finger, and the attributes of the controlled object are controlled based on the single-finger interaction operation performed by the first finger on the first screen; the gesture recognition parameters include the displacement of the second finger; If the interaction operation type is a rubbing operation, then the attributes of the controlled object within the operation task are controlled based on the distance index generated by the rubbing operation or the action of the rubbing operation; the gesture recognition parameters include a distance index; the distance index includes a displacement difference vector or a relative distance; If the interaction operation type is a two-sided alternating tapping operation, then the attributes of the controlled object within the operation task are controlled based on the two-sided alternating tapping operation; If the interaction operation type is a push-pull operation, then the attributes of the controlled object within the operation task are controlled based on the displacement vector generated by the push-pull operation or the action of the push-pull operation; the gesture recognition parameters include the displacement vector; If the interaction operation type is an anchor point sliding operation, then the first attribute of the controlled object within the operation task is determined based on the position of the fixed finger, and the second attribute of the controlled object within the operation task is controlled based on the displacement vector of the moving finger; the fixed finger and the moving finger are one of the first finger and the second finger, respectively, and the two are different from each other; the gesture recognition parameters include the position of the fixed finger and the displacement vector of the moving finger.

3. The interactive control method according to claim 1, characterized in that, The first touch information includes touch state information; Determining the interaction operation type and gesture recognition parameters based on at least one of the two-finger position data, the two-finger touch state, and the two-finger movement direction includes at least one of the following: If, within the first time window, the touch state information indicates that the displacement difference vector between the two fingers is greater than a preset first distance threshold, and the two fingers move in opposite directions, then the interaction operation type is confirmed to be a rubbing operation. If, within the second time window, the touch state information indicates that the two-finger touch state has switched from the two-finger lifted state to the two-finger pressed state, and remains in the two-finger pressed state for a preset first duration, then the interaction operation type is determined to be a long pinch operation. If, within the third time window, the touch state information indicates at least one round of tapping, then the interaction operation type is determined to be a two-sided alternating tapping operation; wherein, the tapping operation includes: the first finger switching from a raised state to a pressed state on the first screen and holding it for a preset time before switching back to the raised state; the second finger switching from a raised state to a pressed state on the touch module and holding it for a preset time before switching back to the raised state; or, the second finger switching from a raised state to a pressed state on the touch module and holding it for a preset time before switching back to the raised state; and the first finger switching from a raised state to a pressed state on the first screen and holding it for a preset time before switching back to the raised state. If, within the first time window, the touch state information indicates that two fingers are performing a pinch action and moving in the pinch state, and the two fingers are moving in the same direction, then the interaction operation type is confirmed to be a push-pull operation; the pinch action is determined based on the two-finger position data, and the distance between the two fingers is reduced to a preset second distance threshold; the first distance threshold is greater than the second distance threshold; If, within the first time window, the touch state information indicates that a single finger is pressed down and another finger performs a swipe operation, then the interaction operation type is confirmed to be an anchor point swipe operation. If, within the first time window, both the first and second fingers are detected to switch from the lifted state to the pressed state, then a single pinch operation is confirmed to have been performed.

4. The interactive control method according to claim 2, characterized in that, The method further includes at least one of the following: In the cursor linkage scenario, if it is detected that the first finger performs a lift operation and remains in the lift state, the display position of the preset cursor is adjusted based on the touch position of the first finger; If the second finger is detected to sequentially perform a lift operation, a slide operation, and a press operation within the first time window, the display position of the preset cursor is controlled according to the position change corresponding to the slide operation; In the cursor linkage scenario, if the second finger is detected to be in a raised state and the first finger switches from a raised state to a pressed state, then it is determined that the single-screen interaction mode is entered and the multi-screen linkage interaction mode and the cursor linkage scenario are exited. In a cursor-linked scenario, if the second finger is detected to lift up and remains in the lifted state within the first time window, then the cursor-linked scenario is determined to be closed. In a cursor-linked scenario, if it is detected that the second finger remains pressed while the touch state of the first finger changes, then the cursor-linked scenario is determined to be turned off.

5. The interactive control method according to claim 3, characterized in that, The interaction mode also includes a single-screen interaction mode; the method further includes: The system detects the touch point of a first finger on the first screen to obtain first touch information; it also detects the touch point of a second finger on the touch module to obtain second touch information. The user's interaction mode is determined based on the first touch information and the second touch information.

6. The interactive control method according to claim 5, characterized in that, Determining the user's interaction mode based on the first touch information and the second touch information includes at least one of the following: If it is determined from the first touch information and the second touch information that the user has performed the single pinch operation, then the multi-screen linkage interaction mode is enabled and the single-screen interaction mode is disabled. If the two-finger touch state is in a single-finger press state or a multi-finger press state, then exit the multi-screen linkage interaction mode and enter the single-screen interaction mode.

7. The interactive control method according to claim 1, characterized in that, The first touch information includes a first movement direction angle of a first finger; the second touch information includes a second movement direction angle of a second finger. The step of calculating the bi-finger movement direction between the first finger and the second finger based on the first touch information and the second touch information includes: If the angle between the first movement direction angle and the second movement direction angle is greater than the preset angle, then the movement direction of the two fingers is confirmed to be opposite. If the first movement direction angle and the second movement direction angle are less than the preset angle, then it is confirmed that the two fingers move in the same direction; If either the first or the second movement direction angle is empty, then the movement direction of the two fingers is confirmed to be directionless.

8. The interactive control method according to claim 1, characterized in that, The step of calculating the two-finger touch state between the first finger and the second finger based on the first touch information and the second touch information includes at least one of the following: If the distance the first finger moves on the first screen is less than a first preset threshold, and the distance the second finger moves on the touch module is less than a second preset threshold, then the two-finger touch state is no movement. If the distance the first finger moves on the first screen is greater than a first preset threshold, and the distance the second finger moves on the touch module is less than the second preset threshold, then the two-finger touch state is a single-finger movement. If the distance the first finger moves on the first screen is less than the first preset threshold, and the distance the second finger moves on the touch module is greater than the second preset threshold, then the two-finger touch state is the single-finger movement. If the distance the first finger moves on the first screen is greater than the first preset threshold, and the distance the second finger moves on the touch module is greater than the second preset threshold, then the two-finger touch state is two-finger movement; If both the first and second fingers switch from the lifted state to the pressed state within the first time window, then the two-finger touch state is confirmed to be the two-finger pressed state. If, within the first time window, both the first finger and the second finger are detected to change from a pressed state to a lifted state, then the two-finger touch state is confirmed to be a two-finger lifted state. If, within the first time window, either the first finger or the second finger is detected to change from a raised state to a pressed state, then the two-finger touch state is confirmed to be a single-finger pressed state. If, within the first time window, it is detected that the first finger, the second finger, and the third finger all change from the lifted state to the pressed state, then the two-finger touch state is confirmed to be a multi-finger pressed state. If, within the third time window, the following state changes are detected in the two-finger touch state: two-finger lifted state, two-finger pressed state, two-finger lifted state, two-finger pressed state, and two-finger lifted state, then it is confirmed that the user has performed a double pinch operation.

9. The interactive control method according to claim 1, characterized in that, The two-finger position data includes at least one of the distance between the two fingers and the displacement difference vector between the two fingers, and the calculation of the two-finger position data between the first finger and the second finger based on the first touch information and the second touch information includes at least one of the following: The positions of the first finger's touch point on the first screen and the second finger's touch point on the touch module are projected onto a projection coordinate system. The distance between the two fingers is calculated based on the positions of the projected touch points of the first and second fingers in the projection coordinate system. The positions of the first finger's touch point on the first screen and the second finger's touch point on the touch module are projected onto the projection coordinate system. Based on the first finger moving from the first position to the second position and the second finger moving from the third position to the fourth position in the projection coordinate system, the displacement difference vector between the first finger and the second finger is calculated.

10. An electronic device, characterized in that, The electronic device includes a first screen and a touch module; the first screen and the touch module are set at a preset angle; the electronic device also includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the interactive control method of any one of claims 1-9 and interact with the user.

Images