Touch-control interaction method and apparatus

Abstract

Provided in the present application are a touch-control interaction method and apparatus, which are applied to a terminal device comprising a capacitive touch screen and an ultrasonic sensor. The method comprises: when a terminal device is covered in a liquid or a capacitive touch screen of same fails, acquiring an enable operation for an ultrasonic mode; and in response to the enable operation for the ultrasonic mode, presenting a first interface in an ultrasonic sensing area of the terminal device, wherein the first interface comprises at least one application identifier, and the ultrasonic sensing area is an area of a screen of the terminal device, which area is used for sensing a user operation by means of the ultrasonic sensor. The method provided in the present application can ensure the normal interaction between a user and a terminal device when a capacitive touch screen is not working well or even fails.

Description

Touch interaction methods and devices Technical Field

[0001] This application relates to the field of terminal device technology, specifically to a touch interaction method and apparatus. Background Technology

[0002] With the development of electronic information technology, smartphones, tablets, and other smart terminals have become important devices for users to record their daily lives. Capacitive touchscreens have become the primary input method for people using these devices.

[0003] Capacitive touchscreens have limitations in certain scenarios where they perform poorly. Because they rely on capacitance to detect fingers, they can exhibit erratic touch detection in environments with abundant water droplets or underwater, preventing normal touch operation. Therefore, a new touch interaction method is needed to ensure that users can still interact with the device even when the capacitive touchscreen is malfunctioning or fails.

[0004] Summary of the Invention

[0005] This application provides a touch interaction method and apparatus that can ensure normal interaction between the user and the terminal device when the capacitive touch screen is not working well or even fails.

[0006] In a first aspect, a touch interaction method is provided, applied to a terminal device including a capacitive touchscreen and an ultrasonic sensor. The method includes: acquiring an ultrasonic mode activation operation when the terminal device is covered by liquid or the capacitive touchscreen is malfunctioning; and, in response to the ultrasonic mode activation operation, displaying a first interface in the ultrasonic sensing area of ​​the terminal device, the first interface including at least one application identifier, the ultrasonic sensing area being an area of ​​the terminal device's screen used to sense user operations via the ultrasonic sensor.

[0007] This application provides a touch interaction method that can acquire the start operation of ultrasonic mode when the capacitive touch screen is not working well or even fails, control the terminal device to enter ultrasonic mode, and ensure normal interaction between the user and the terminal device.

[0008] Capacitive touchscreens use the principle of capacitance to sense human touch. A capacitive plate covers the screen of the terminal device. When a finger touches the screen, a capacitor is formed between the human body and the capacitive plate, changing the electric field distribution of the plate. The capacitive touchscreen determines the touch location and gesture by detecting these changes in capacitance.

[0009] Ultrasonic sensors can locate the coordinates of a finger on the screen, recognize a user's fingerprint, and identify touch events. In one possible implementation, the ultrasonic sensor operates at a low frequency in non-ultrasonic mode and at a high frequency in ultrasonic mode.

[0010] In response to the activation of ultrasonic mode, the terminal device can choose to disable the capacitive touchscreen and enter ultrasonic mode. In ultrasonic mode, users can interact with the terminal device by performing gestures within the ultrasonic sensing area.

[0011] The area of ​​the ultrasonic sensing area can be the same size as or smaller than the screen of the terminal device. The area of ​​the first interface can be larger than, smaller than, or the same size as the ultrasonic sensing area.

[0012] In some potential application scenarios, the terminal device may be covered by liquids, including but not limited to any of the following: the terminal device's screen is wetted, the entire terminal device or part of the terminal device is submerged in water, or there is a large amount of oil on the terminal device's screen. This liquid includes, but is not limited to, rainwater, seawater, river water, pool water, tap water, beverages, and oil.

[0013] Capacitive touchscreen malfunctions can be caused by hardware failures, software problems, or external environmental factors. For example, a failure in the chip or module of the capacitive touchscreen can lead to its failure.

[0014] The application identifier can be an application icon or an application list. In some possible implementations, the first screen can also include one or more of the following: application folder, application widgets, wallpaper, search bar, bottom navigation bar, notification bar, theme animation, etc.

[0015] It should be understood that the activation operation of ultrasonic mode can be pre-entered by the user on the terminal device, or it can be configured by the terminal device before leaving the factory.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the operation of activating the ultrasonic mode includes at least one of sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, repeatedly clicking the screen of the terminal device, and long-pressing the screen of the terminal device.

[0017] The preset movement trajectories include, but are not limited to, swipes in different directions, numbers, English characters, and special symbols, such as "W", "M", "S", "C", "||", "|", "V", "<", ">", "&", "8", circles, triangles, hearts, etc. It should be understood that the preset movement trajectories in this application embodiment are not limited to movement trajectories obtained by single-finger operation, but can also be movement trajectories obtained by multi-finger operation.

[0018] Pressing the physical buttons on the terminal device includes, but is not limited to, pressing a single physical button or a combination of pressing multiple physical buttons. For example, the terminal device includes a volume up button, a volume down button, and a home button. Users can enter ultrasonic mode by long-pressing the home button, long-pressing the home button and the volume up button, long-pressing the home button and the volume down button, pressing the home button N1 times consecutively, or pressing the home button N1 times consecutively followed by pressing the volume up button M1 times consecutively, where N1 and M1 are integers greater than 1. It should be understood that the terminal device may also include more physical buttons. With more physical buttons, the terminal device can enter ultrasonic mode through more button combinations. The above examples should not be construed as limiting this application.

[0019] Multiple taps on the terminal device screen include single-finger double tap, single-finger triple tap, multi-finger double tap, multi-finger triple tap, and other operations.

[0020] Long-pressing the screen of a terminal device includes pressing the screen with a single finger for more than a preset time threshold, or pressing the screen with multiple fingers for more than a preset time threshold.

[0021] This application provides a touch interaction method that can acquire the start operation of ultrasonic mode when the capacitive touch screen is not working well or even fails, control the terminal device to enter ultrasonic mode, ensure normal interaction between the user and the terminal device, and the start operation of ultrasonic mode includes a variety of different types, with many choices and user-friendly operation.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring an exit operation of the ultrasonic mode; and, in response to the exit operation of the ultrasonic mode, displaying a second interface on the screen of the terminal device, the second interface being an area of ​​the screen of the terminal device used to sense user operation via the capacitive touchscreen.

[0023] In response to the exit operation of ultrasonic mode, the terminal device can exit ultrasonic mode and select to activate the capacitive touchscreen. Users can then interact with the terminal device using gestures on the capacitive touchscreen.

[0024] The second interface may include any one or more of the following: application icon, application folder, application widget, wallpaper, search bar, bottom navigation bar, notification bar, theme animation, etc.

[0025] The area of ​​the second interface is the same as the screen area of ​​the terminal device, or the area of ​​the second interface can be smaller than the screen area of ​​the terminal device.

[0026] This application provides a touch interaction method that can control the terminal device to exit ultrasonic mode when the capacitive touch screen is working well. Users can interact with the terminal device through the capacitive touch screen, which is highly flexible.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the exit operation of the ultrasonic mode includes at least one of the following: clicking the off icon of the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, and long-pressing the screen of the terminal device.

[0028] The closing indicator for the ultrasonic sensing area can be a closing icon or an indicator to exit ultrasonic mode, such as an "×", "Close", or "Exit" button.

[0029] The preset movement paths include, but are not limited to, sliding in different directions, numbers, English characters and special symbols, such as "W", "M", "S", "C", "||", "|", "V", "<", ">", "&", "8", circles, triangles, hearts, etc.

[0030] Pressing the physical buttons on the terminal device includes, but is not limited to, pressing a single physical button or a combination of pressing multiple physical buttons. For example, the terminal device includes a volume up button, a volume down button, and a home button. Users can exit ultrasonic mode by long-pressing the home button, long-pressing the home button and the volume up button, long-pressing the home button and the volume down button, pressing the home button N2 times consecutively, or pressing the home button N2 times consecutively followed by pressing the volume up button M2 times consecutively, where N2 and M2 are integers greater than 1. It should be understood that the terminal device may also include more physical buttons. With more physical buttons, the terminal device can exit ultrasonic mode through more button combinations. The above examples should not be construed as limiting this application.

[0031] Multiple taps on the terminal device screen include single-finger double tap, single-finger triple tap, multi-finger double tap, multi-finger triple tap, and other operations.

[0032] Long-pressing the screen of a terminal device includes pressing the screen with a single finger for more than a preset time threshold, or pressing the screen with multiple fingers for more than a preset time threshold.

[0033] This application provides a touch interaction method that can control the terminal device to exit ultrasonic mode when the capacitive touch screen is working well. Users can interact with the terminal device through the capacitive touch screen, which is highly flexible.

[0034] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring a user's swiping gesture in the ultrasonic sensing area while the first interface is displayed in the ultrasonic sensing area, the swiping gesture including a left swipe, a right swipe, an up swipe, or a down swipe; and displaying a third interface in the ultrasonic sensing area in response to the swiping gesture, the third interface being different from the first interface.

[0035] For example, a user's left swipe, right swipe, up swipe, or down swipe gesture in the ultrasonic sensing area can be a page-turning operation. The ultrasonic sensing area transitioning from displaying a first interface to displaying a third interface can represent a user turning from the first interface to the third interface.

[0036] The third interface includes any one or more of the following: application icon, application folder, application widget, wallpaper, search bar, bottom navigation bar, notification bar, and theme animation.

[0037] This application provides a touch interaction method that can control the terminal device to enter ultrasonic mode when the capacitive touch screen is not working well or even fails, and switch the interface by sliding in the ultrasonic sensing area, thus ensuring normal interaction between the user and the terminal device.

[0038] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring, in the ultrasonic sensing area, a user's long-press operation on a first application identifier and a trajectory of dragging the first application identifier, wherein the first application identifier is any application identifier in the first interface; in response to the long-press operation on the first application identifier and the trajectory of dragging the first application identifier, presenting a fourth interface in the ultrasonic sensing area, wherein the first application identifier in the fourth interface is located at the end position of the trajectory.

[0039] For example, the first application could be a camera application, a photo album application, a settings application, an email application, a memo application, a calculator application, a calendar application, a fitness and health application, a weather application, a browser application, etc.

[0040] For example, a user can drag the first application icon to move it to a location other than its original position on the first screen, or move it to another screen after swiping through the first screen.

[0041] This application provides a touch interaction method that can control the terminal device to enter ultrasonic mode when the capacitive touch screen is not working well or even fails. The application icon can be dragged in the ultrasonic sensing area to change the position of the application icon, thus ensuring normal interaction between the user and the terminal device.

[0042] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining an operation by a user clicking a second application identifier, wherein the second application identifier is any application identifier in the first interface; and in response to the operation of clicking the second application identifier, presenting a fifth interface in the ultrasonic sensing area, wherein the fifth interface is the interface of the application corresponding to the second application identifier.

[0043] For example, the application corresponding to the second application identifier is the second application, which can be a camera application, photo album application, settings application, email application, memo application, calculator application, calendar application, fitness and health application, weather application, browser application, etc. The second application identifier can be the same as or different from the first application identifier.

[0044] This application provides a touch interaction method that can control the terminal device to enter ultrasonic mode when the capacitive touch screen is not working well or even fails. By clicking the application icon in the ultrasonic sensing area, the required application can be launched, ensuring normal interaction between the user and the terminal device.

[0045] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring a user's operation of rotating the terminal device while the first interface is displayed in the ultrasonic sensing area; and, in response to the operation of rotating the terminal device, displaying a sixth interface in the ultrasonic sensing area, wherein the sixth interface is the first interface after rotation.

[0046] In one possible implementation scenario, the terminal device can be equipped with an accelerometer or gyroscope sensor, which can detect the current tilt angle of the terminal device, thereby controlling the switching between landscape and portrait modes.

[0047] For example, the first interface is a portrait interface and the sixth interface is a landscape interface, or the first interface is a landscape interface and the sixth interface is a portrait interface.

[0048] This application provides a touch interaction method that can control the terminal device to enter ultrasonic mode when the capacitive touch screen is not working well or even fails, and can also control the switching between landscape and portrait modes of the terminal device to ensure normal interaction between the user and the terminal device.

[0049] In conjunction with the first aspect, in some implementations of the first aspect, before the first interface is presented in the ultrasonic sensing area of ​​the terminal device, the method further includes: acquiring the fingerprint or password input by the user.

[0050] When the terminal device is locked, the user's fingerprint or password can be obtained for verification first, and the first interface can be presented only if the verification is successful.

[0051] For example, fingerprints can be entered by placing a finger on a physical button, placing a finger on the screen, or swiping a finger across the screen. In one possible implementation, the terminal device's screen integrates a fingerprint sensor. When a user's finger is placed on the fingerprint sensing area on the screen, or when the finger's movement passes through the fingerprint sensing area, the terminal device can acquire the user's fingerprint. The fingerprint sensing area is the area on the terminal device's screen used to sense the user's fingerprint. The fingerprint sensor can be an ultrasonic sensor, and the fingerprint sensing area can be the ultrasonic sensing area.

[0052] For example, the password can be entered by the user via a physical key or by the user entering it in the ultrasonic sensing area.

[0053] This application provides a touch interaction method that can verify the user's identity before entering ultrasonic mode, thereby improving security and protecting user privacy.

[0054] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: presenting any one or more of the following buttons in the ultrasonic sensing area: settings, troubleshooting, information saving, and emergency assistance.

[0055] This application provides a touch interaction method that can control the terminal device to enter ultrasonic mode when the capacitive touch screen is not working well or even fails. In ultrasonic mode, functions such as settings, troubleshooting, information saving, and emergency assistance are provided to meet the needs of users in emergency situations.

[0056] In a second aspect, a computer device is provided for use in a terminal device including a capacitive touchscreen and an ultrasonic sensor. The device includes: an acquisition module for acquiring an ultrasonic mode activation operation when the terminal device is covered by liquid or the capacitive touchscreen is malfunctioning; and a processing module for displaying a first interface in the ultrasonic sensing area of ​​the terminal device in response to the ultrasonic mode activation operation. The first interface includes at least one application identifier, and the ultrasonic sensing area is an area of ​​the terminal device's screen used to sense user operations via the ultrasonic sensor.

[0057] In conjunction with the second aspect, in some implementations of the second aspect, the operation of activating the ultrasonic mode includes at least one of sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, repeatedly clicking the screen of the terminal device, and long-pressing the screen of the terminal device.

[0058] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire the exit operation of the ultrasonic mode; the processing module is further configured to, in response to the exit operation of the ultrasonic mode, display a second interface on the screen of the terminal device, the second interface being an area of ​​the screen of the terminal device used to sense user operation via the capacitive touchscreen.

[0059] In conjunction with the second aspect, in some implementations of the second aspect, the exit operation of the ultrasonic mode includes at least one of the following: clicking the off icon of the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, and long-pressing the screen of the terminal device.

[0060] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire a user's swiping gesture in the ultrasonic sensing area while the first interface is displayed in the ultrasonic sensing area, the swiping gesture including a left swipe gesture, a right swipe gesture, an up swipe gesture, or a down swipe gesture; the processing module is further configured to display a third interface in the ultrasonic sensing area in response to the swiping gesture, the third interface being different from the first interface.

[0061] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire the user's long-press operation on the first application identifier and the trajectory of dragging the first application identifier in the ultrasonic sensing area, wherein the first application identifier is any application identifier in the first interface; the processing module is further configured to, in response to the long-press operation on the first application identifier and the trajectory of dragging the first application identifier, present a fourth interface in the ultrasonic sensing area, wherein the first application identifier in the fourth interface is located at the end position of the trajectory.

[0062] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire the user's operation of clicking the second application identifier, wherein the second application identifier is any application identifier in the first interface; the processing module is further configured to, in response to the operation of clicking the second application identifier, present a fifth interface in the ultrasonic sensing area, wherein the fifth interface is the interface of the application corresponding to the second application identifier.

[0063] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire the user's operation of rotating the terminal device while the first interface is displayed in the ultrasonic sensing area; the processing module is further configured to display a sixth interface in the ultrasonic sensing area in response to the operation of rotating the terminal device, the sixth interface being the first interface after rotation.

[0064] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition module is further configured to acquire the fingerprint or password input by the user.

[0065] In conjunction with the second aspect, in some implementations of the second aspect, the processing module is also configured to display any one or more of the following buttons in the ultrasonic sensing area: settings, troubleshooting, information saving, and emergency assistance.

[0066] The beneficial effects of the second aspect and any possible implementation of the second aspect correspond to the beneficial effects of the first aspect and any possible implementation of the first aspect, which will not be elaborated further.

[0067] Thirdly, a touch interaction method is provided, applied to a terminal device including a capacitive touchscreen and an ultrasonic sensor. The method includes: acquiring an ultrasonic mode startup operation; responding to the ultrasonic mode startup operation, displaying a first interface in the ultrasonic sensing area of ​​the terminal device, the first interface including at least one application identifier, the ultrasonic sensing area being an area of ​​the terminal device's screen used to sense user operation through the ultrasonic sensor; and detecting that the terminal device is covered by liquid.

[0068] This application provides a touch interaction method that can acquire the start operation of ultrasonic mode before the terminal device is covered by liquid, control the terminal device to enter ultrasonic mode, and ensure that the user can still interact normally with the terminal device when it is covered by liquid.

[0069] Capacitive touchscreens use the principle of capacitance to sense human touch. A capacitive plate covers the screen of the terminal device. When a finger touches the screen, a capacitor is formed between the human body and the capacitive plate, changing the electric field distribution of the plate. The capacitive touchscreen determines the touch location and gesture by detecting these changes in capacitance.

[0070] Ultrasonic sensors can locate the coordinates of a finger on the screen, recognize a user's fingerprint, and identify touch events. In one possible implementation, the ultrasonic sensor operates at a low frequency in non-ultrasonic mode and at a high frequency in ultrasonic mode.

[0071] In response to the activation of ultrasonic mode, the terminal device can choose to disable the capacitive touchscreen and enter ultrasonic mode. In ultrasonic mode, users can interact with the terminal device by performing gestures within the ultrasonic sensing area.

[0072] The area of ​​the ultrasonic sensing area can be the same size as or smaller than the screen of the terminal device. The area of ​​the first interface can be larger than, smaller than, or the same size as the ultrasonic sensing area.

[0073] In some potential application scenarios, the state in which a terminal device is covered by liquid includes, but is not limited to, any of the following: the terminal device's screen is wetted, the entire terminal device or part of the terminal device is submerged in water, or there is a large amount of oil on the terminal device's screen. This liquid includes, but is not limited to, rainwater, seawater, river water, pool water, tap water, beverages, and oil.

[0074] The application identifier can be an application icon or an application list. In some possible implementations, the first screen can also include one or more of the following: application folder, application widgets, wallpaper, search bar, bottom navigation bar, notification bar, theme animation, etc.

[0075] In conjunction with the third aspect, in some implementations of the third aspect, the operation of activating the ultrasonic mode includes at least one of sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, repeatedly clicking the screen of the terminal device, and long-pressing the screen of the terminal device.

[0076] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: acquiring an exit operation of the ultrasonic mode; and, in response to the exit operation of the ultrasonic mode, displaying a second interface on the screen of the terminal device, the second interface being an area of ​​the terminal device's screen used to sense user operation via the capacitive touchscreen.

[0077] In conjunction with the third aspect, in some implementations of the third aspect, the exit operation of the ultrasonic mode includes at least one of the following: clicking the off icon of the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, and long-pressing the screen of the terminal device.

[0078] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: acquiring a user's swiping gesture in the ultrasonic sensing area while the first interface is displayed in the ultrasonic sensing area, the swiping gesture including a left swipe, a right swipe, an up swipe, or a down swipe; and displaying a third interface in the ultrasonic sensing area in response to the swiping gesture, the third interface being different from the first interface.

[0079] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: acquiring, in the ultrasonic sensing area, a user's long-press operation on a first application identifier and a trajectory of dragging the first application identifier, wherein the first application identifier is any application identifier in the first interface; in response to the long-press operation on the first application identifier and the trajectory of dragging the first application identifier, presenting a fourth interface in the ultrasonic sensing area, wherein the first application identifier in the fourth interface is located at the end position of the trajectory.

[0080] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: obtaining an operation by a user clicking a second application identifier, wherein the second application identifier is any application identifier in the first interface; and responding to the operation of clicking the second application identifier, presenting a fifth interface in the ultrasonic sensing area, wherein the fifth interface is the interface of the application corresponding to the second application identifier.

[0081] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: acquiring a user's operation of rotating the terminal device while the first interface is displayed in the ultrasonic sensing area; and, in response to the operation of rotating the terminal device, displaying a sixth interface in the ultrasonic sensing area, the sixth interface being the first interface after rotation.

[0082] In conjunction with the third aspect, in some implementations of the third aspect, before the first interface is presented in the ultrasonic sensing area of ​​the terminal device, the method further includes: acquiring the fingerprint or password input by the user.

[0083] In conjunction with the third aspect, in some implementations of the third aspect, the method further includes: presenting any one or more of the following buttons in the ultrasonic sensing area: settings, troubleshooting, information saving, and emergency assistance.

[0084] The beneficial effects of the third aspect and any possible implementation of the third aspect correspond to the beneficial effects of the first aspect and any possible implementation of the first aspect, which will not be elaborated further.

[0085] Fourthly, a computer apparatus is provided, comprising several functional modules for implementing any of the methods of the third aspect.

[0086] For example, the computer device may include an acquisition module and a processing module.

[0087] Fifthly, embodiments of this application provide a terminal device, the terminal device including a processor, the processor being coupled to a memory, reading and executing instructions and / or program code in the memory to execute the first aspect or any possible implementation of the first aspect, or to execute the third aspect or any possible implementation of the third aspect.

[0088] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing program code that, when the computer storage medium is run on a computer, causes the computer to perform the first aspect or any possible implementation thereof, or causes the computer to perform the third aspect or any possible implementation thereof.

[0089] In a seventh aspect, embodiments of this application provide a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform as in the first aspect or any possible implementation thereof, or causes the computer to perform as in the third aspect or any possible implementation thereof.

[0090] Eighthly, a chip is provided, including a processor for reading instructions stored in a memory, wherein when the processor executes the instructions, the chip implements the first aspect or any possible implementation of the first aspect, or implements the third aspect or any possible implementation of the third aspect. Attached Figure Description

[0091] Figure 1 is a schematic diagram of the hardware architecture of a terminal device applicable to an embodiment of this application.

[0092] Figure 2 is a schematic diagram of the software architecture of a terminal device applicable to an embodiment of this application.

[0093] Figure 3 is an exemplary flowchart of a touch interaction method provided in an embodiment of this application.

[0094] Figure 4 is a schematic diagram of a capacitive touch screen and an ultrasonic sensing area provided in an embodiment of this application.

[0095] Figure 5 is a schematic diagram of the start-up operation of the ultrasonic mode provided in the embodiment of this application.

[0096] Figure 6 is a schematic diagram of the first interface provided in an embodiment of this application.

[0097] Figure 7 is a schematic diagram of the first interface presented in response to an ultrasonic mode startup operation according to an embodiment of this application.

[0098] Figure 8 is a schematic diagram of the first interface presented during the startup operation of another response ultrasonic mode provided in an embodiment of this application.

[0099] Figure 9 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0100] Figure 10 is a schematic diagram of the second interface presented during the exit operation of the response ultrasonic mode according to an embodiment of this application.

[0101] Figure 11 is a schematic diagram of a third interface presented in response to a user's swipe gesture, according to an embodiment of this application.

[0102] Figure 12 is a schematic diagram of the fourth interface presented in response to user long-press and drag operations according to an embodiment of this application.

[0103] Figure 13 is a schematic diagram of the fifth interface presented in response to user click operations according to an embodiment of this application.

[0104] Figure 14 is a schematic diagram of the sixth interface presented in response to the user's rotation operation according to an embodiment of this application.

[0105] Figure 15 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0106] Figure 16 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0107] Figure 17 is a structural example diagram of a computer device provided in an embodiment of this application.

[0108] Figure 18 is a schematic diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0109] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort should fall within the scope of protection of this application.

[0110] In the embodiments of this application, the words "exemplary," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner.

[0111] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0112] The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0113] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0114] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0115] With the development of electronic information technology, smartphones, tablets, and other smart terminals have become important devices for users to record their daily lives. Capacitive touchscreens have become the primary input method for people using these devices.

[0116] Capacitive touchscreens have some limitations in certain scenarios where they perform poorly. Because they rely on capacitance to detect fingers, they can misreport touch points in environments with abundant water or underwater, preventing normal touch operation. Currently, users often wrap their devices in waterproof bags to protect them from water damage, but this complete enclosure makes operation inconvenient.

[0117] When a capacitive touchscreen malfunctions or fails, the user loses the ability to interact with the terminal device. This application constructs a touch interaction method that allows the user to interact normally with the terminal device even when the capacitive touchscreen is malfunctioning or fails.

[0118] The methods provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of terminal device.

[0119] For example, Figure 1 shows a schematic diagram of the structure of terminal device 100. Terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, an ultrasonic sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0120] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0121] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0122] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0123] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0124] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0125] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0126] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0127] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0128] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0129] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to enable the display function of the terminal device 100.

[0130] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0131] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, or USB Type-C port. USB port 130 can be used to connect a charger to charge terminal device 100, and can also be used for data transfer between terminal device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0132] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0133] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also supply power to the terminal device via the power management module 141.

[0134] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0135] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0136] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0137] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0138] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0139] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0140] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0141] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0142] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0143] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0144] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0145] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0146] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0147] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0148] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0149] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0150] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0151] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0152] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0153] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or make hands-free calls through the speaker 170A.

[0154] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 100 answers a phone call or voice message, the receiver 170B can be brought close to the listener's ear to hear the voice.

[0155] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 100 may be equipped with at least one microphone 170C. In some embodiments, terminal device 100 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 100 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0156] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0157] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, terminal device 100 detects the intensity of the touch operation based on pressure sensor 180A. Terminal device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0158] The gyroscope sensor 180B can be used to determine the motion attitude of the terminal device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0159] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0160] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0161] The 180E accelerometer can detect the magnitude of acceleration of the terminal device 100 in various directions (typically three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the terminal device, and can be applied to applications such as landscape / portrait switching and pedometers.

[0162] A distance sensor 180F is used to measure distance. The terminal device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, the terminal device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0163] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The terminal device 100 emits infrared light outward through the LED. The terminal device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 may use the proximity sensor 180G to detect when a user holds the terminal device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and screen locking.

[0164] The ambient light sensor 180L is used to sense the ambient light intensity. The terminal device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touches.

[0165] The ultrasonic sensor 180H is used to locate the coordinates of a finger on the screen and can also recognize the user's fingerprint and touch events. The terminal device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, and answering calls with fingerprints.

[0166] Temperature sensor 180J is used to detect temperature. In some embodiments, terminal device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, terminal device 100 reduces the performance of the processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, terminal device 100 heats battery 142 to prevent abnormal shutdown of terminal device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, terminal device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0167] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "capacitive touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of terminal device 100, in a different position than display screen 194.

[0168] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0169] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal device 100 can receive button input and generate key signal inputs related to user settings and function control of terminal device 100.

[0170] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0171] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0172] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 100. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the terminal device 100 uses an embedded SIM (eSIM) card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0173] It should be understood that the phone cards in the embodiments of this application include, but are not limited to, SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated circuit cards (UICC), etc.

[0174] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of terminal device 100.

[0175] Figure 2 is a software structure block diagram of the terminal device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0176] As shown in Figure 2, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0177] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0178] As shown in Figure 2, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0179] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0180] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0181] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0182] The phone manager is used to provide communication functions for terminal device 100. For example, it manages call status (including connection, hang-up, etc.).

[0183] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0184] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the device, and flashing indicator lights.

[0185] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0186] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0187] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0188] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0189] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0190] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0191] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0192] A 2D graphics engine is a graphics engine for 2D drawing.

[0193] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0194] It should be understood that the technical solutions in the embodiments of this application can be used in systems such as Android, iOS, and HarmonyOS.

[0195] Figure 3 is an exemplary flowchart of a touch interaction method provided in an embodiment of this application. This method is applied to a terminal device including a capacitive touchscreen and an ultrasonic sensor.

[0196] 310, Obtain the start operation of ultrasonic mode.

[0197] The ultrasonic mode is activated when the terminal device is covered by liquid or the capacitive touchscreen is malfunctioning.

[0198] Capacitive touchscreens use the principle of capacitance to sense human touch. A capacitive plate covers the screen of the terminal device. When a finger touches the screen, a capacitor is formed between the human body and the capacitive plate, changing the electric field distribution of the plate. The capacitive touchscreen determines the touch location and gesture by detecting these changes in capacitance.

[0199] Ultrasonic sensors can locate the coordinates of a finger on the screen, recognize a user's fingerprint, and identify touch events. In one possible implementation, the ultrasonic sensor operates at a low frequency in non-ultrasonic mode and at a high frequency in ultrasonic mode.

[0200] In some potential application scenarios, the terminal device may be covered by liquids, including but not limited to any of the following: the terminal device's screen is wetted, the entire terminal device or part of the terminal device is submerged in water, or there is a large amount of oil on the terminal device's screen. This liquid includes, but is not limited to, rainwater, seawater, river water, pool water, tap water, beverages, and oil.

[0201] Capacitive touchscreen malfunctions can be caused by hardware failures, software problems, or external environmental factors. For example, a failure in the chip or module of the capacitive touchscreen can lead to its failure.

[0202] Users can activate the ultrasonic mode on or off the terminal device by performing the ultrasonic mode activation operation. The device can be locked or unlocked. This application does not restrict the device state during the ultrasonic mode activation operation, as long as the ultrasonic sensor is functioning properly.

[0203] 320, the first interface is presented in the ultrasonic sensing area.

[0204] In response to the activation of the ultrasonic mode, a first interface is displayed in the ultrasonic sensing area of ​​the terminal device. The first interface includes at least one application identifier. The ultrasonic sensing area is the area of ​​the terminal device's screen used to sense user operations via an ultrasonic sensor.

[0205] In response to the activation of ultrasonic mode, the terminal device can choose to disable the capacitive touchscreen and enter ultrasonic mode. In ultrasonic mode, users can interact with the terminal device by performing gestures within the ultrasonic sensing area.

[0206] The application identifier can be an application icon or an application list. In some possible implementations, the first screen can also include one or more of the following: application folder, application widgets, wallpaper, search bar, bottom navigation bar, notification bar, theme animation, etc.

[0207] Figure 4 is a schematic diagram of a capacitive touch screen and an ultrasonic sensing area provided in an embodiment of this application.

[0208] Typically, the area of ​​a capacitive touchscreen is the same size as the screen of the terminal device. The area of ​​an ultrasonic sensing area can be equal to or smaller than the screen of the terminal device.

[0209] In Figure 4(a), the area of ​​the capacitive touchscreen is equal to the area of ​​the terminal device's screen, the area of ​​the ultrasonic sensing area is smaller than the area of ​​the capacitive touchscreen, and the area of ​​the ultrasonic sensing area is less than half the area of ​​the terminal device's screen; in Figure 4(b), the area of ​​the capacitive touchscreen and the area of ​​the ultrasonic sensing area are equal to the area of ​​the terminal device's screen; in Figure 4(c), the area of ​​the capacitive touchscreen is equal to the area of ​​the terminal device's screen, the area of ​​the ultrasonic sensing area is smaller than the area of ​​the capacitive touchscreen, and the area of ​​the ultrasonic sensing area is greater than half the area of ​​the terminal device's screen.

[0210] The specific shape of the ultrasonic sensing area is not limited in the embodiments of this application. For example, in Figure 4(a), the ultrasonic sensing area is a rectangle, and in Figure 4(b) and Figure 4(c), the ultrasonic sensing area is a rounded rectangle.

[0211] In this embodiment, the border of the ultrasonic sensing area may be displayed on the interface to remind the user to perform gesture operations in the ultrasonic sensing area, or it may not be displayed on the interface. This application does not limit this.

[0212] Figure 5 is a schematic diagram of the start-up operation of the ultrasonic mode provided in the embodiment of this application.

[0213] The activation of ultrasonic mode includes at least one of the following: the user slides a preset movement trajectory on the screen of the terminal device, presses a physical button on the terminal device, taps the screen of the terminal device multiple times, or presses and holds the screen of the terminal device.

[0214] It should be understood that the activation operation of ultrasonic mode can be pre-entered by the user on the terminal device, or it can be configured by the terminal device before leaving the factory.

[0215] The preset movement trajectories include, but are not limited to, swipes in different directions, numbers, English characters, and special symbols, such as "W", "M", "S", "ZZ", "C", "||", "|", "V", "<", ">", "&", "8", circles, triangles, hearts, etc. It should be understood that the preset movement trajectories in this application embodiment are not limited to movement trajectories obtained by single-finger operation, but can also be movement trajectories obtained by multi-finger operation.

[0216] The starting position of the movement trajectory can be within the ultrasonic sensing area or within a non-ultrasonic sensing area on the screen. The ending position of the movement trajectory can also be within the ultrasonic sensing area or within a non-ultrasonic sensing area on the screen. The non-ultrasonic sensing area refers to the area on the terminal device's screen other than the ultrasonic sensing area. This application does not restrict the specific starting and ending positions of the movement trajectory, as long as at least part of the movement trajectory falls within the ultrasonic sensing area.

[0217] Pressing the physical buttons on the terminal device includes, but is not limited to, pressing a single physical button or a combination of pressing multiple physical buttons. For example, the terminal device includes a volume up button, a volume down button, and a home button. Users can enter ultrasonic mode by long-pressing the home button, long-pressing the home button and the volume up button, long-pressing the home button and the volume down button, pressing the home button N1 times consecutively, or pressing the home button N1 times consecutively followed by pressing the volume up button M1 times consecutively, where N1 and M1 are integers greater than 1. It should be understood that the terminal device may also include more physical buttons. With more physical buttons, the terminal device can enter ultrasonic mode through more button combinations. The above examples should not be construed as limiting this application.

[0218] Multiple taps on the terminal device screen include single-finger double tap, single-finger triple tap, multi-finger double tap, multi-finger triple tap, and other operations.

[0219] Long-pressing the screen of a terminal device includes pressing the screen with a single finger for more than a preset time threshold, or pressing the screen with multiple fingers for more than a preset time threshold.

[0220] In Figure 5(a), the ultrasonic mode is activated by the user sliding a preset movement trajectory "W" within the ultrasonic sensing area using a single finger. The entire movement trajectory "W" falls within the ultrasonic sensing area. In Figure 5(b), the ultrasonic mode is activated by the user sliding a preset movement trajectory "V" on the terminal device screen using a single finger. The starting and ending positions of the movement trajectory "V" are in the non-ultrasonic sensing area. In Figure 5(c), the ultrasonic mode is activated by the user sliding a preset heart-shaped movement trajectory within the ultrasonic sensing area using multiple fingers. The entire heart-shaped movement trajectory falls within the ultrasonic sensing area. In Figure 5(d), the ultrasonic mode is activated by the user sliding a preset movement trajectory "|||" on the terminal device screen using multiple fingers. The starting position of the movement trajectory "|||" is in the non-ultrasonic sensing area, and the ending position is in the ultrasonic sensing area. In Figure 5(e), the ultrasonic mode is activated by the user pressing a combination of the volume up button and the home button, for example, simultaneously pressing and holding the home button and the volume up button, or continuously pressing the home button. After pressing the volume up button N1 times, press the volume up button M1 times consecutively; In Figure 5(f), the ultrasonic mode is activated by the user pressing the volume up button and the volume down button in combination, for example, pressing and holding the volume up button and the volume down button simultaneously, or pressing the volume up button and the volume down button multiple times in a short period of time; In Figure 5(g), the ultrasonic mode is activated by the user tapping or holding the ultrasonic sensing area multiple times using multi-finger operation, for example, the user holding the ultrasonic sensing area with two fingers, or the user double-tapping the ultrasonic sensing area with three fingers; In Figure 5(h), the ultrasonic mode is activated by the user tapping or holding the ultrasonic sensing area multiple times using single-finger operation, for example, the user holding the ultrasonic sensing area with one finger, or the user double-tapping the ultrasonic sensing area with one finger; In Figure 5(i), the ultrasonic mode is activated by the user pressing the volume down button while simultaneously sliding a preset movement trajectory “||” on the screen of the terminal device using multi-finger operation, with the starting position of the movement trajectory “||” in the non-ultrasonic sensing area and the ending position in the ultrasonic sensing area.

[0221] Figure 6 is a schematic diagram of the first interface provided in an embodiment of this application.

[0222] The area of ​​the first interface can be larger than, smaller than, or the same size as the ultrasonic sensing area. Users can interact with the terminal device by performing gesture operations within the ultrasonic sensing area.

[0223] In Figure 6(a), the area of ​​the first interface is equal to the area of ​​the ultrasonic sensing area and smaller than the area of ​​the terminal device screen. The first interface includes an application list of multiple applications and a button to exit ultrasonic mode. In Figure 6(b), the area of ​​the first interface is larger than the area of ​​the ultrasonic sensing area and equal to the area of ​​the terminal device screen. The first interface located in the ultrasonic sensing area includes a close icon indicating exiting ultrasonic mode and multiple application icons. The first interface located outside the ultrasonic sensing area includes application widgets. In Figure 6(c), the area of ​​the first interface is smaller than the area of ​​the ultrasonic sensing area, and the area of ​​the ultrasonic sensing area is equal to the area of ​​the terminal device screen. The first interface includes a search bar, a bottom navigation bar, and multiple application icons. In Figure 6(d), the area of ​​the first interface is equal to the area of ​​the ultrasonic sensing area and equal to the area of ​​the terminal device screen. The first interface includes application widgets, a bottom navigation bar, and multiple application icons.

[0224] Figure 7 is a schematic diagram of the first interface presented in response to an ultrasonic mode startup operation according to an embodiment of this application.

[0225] In one possible implementation, the terminal device is in a screen-off but unlocked state. The user uses a single finger to slide a preset movement trajectory "W" within the ultrasonic sensing area, ensuring the entire trajectory "W" falls within this area. The terminal device responds to this ultrasonic mode activation, entering ultrasonic mode and displaying the first interface on the screen. In ultrasonic mode, the user can interact with the terminal device using gestures within the ultrasonic sensing area.

[0226] In one possible implementation, the terminal device is in a screen-off and locked state. The user uses a single finger to slide a preset movement trajectory "W" within the ultrasonic sensing area, with the entire trajectory "W" falling within this area. The terminal device collects the user's fingerprint using the ultrasonic sensor and matches it with a preset fingerprint. If the similarity between the acquired fingerprint and the preset fingerprint is greater than a preset matching threshold, the fingerprint match is successful. The terminal device then responds to the ultrasonic mode activation operation, enters ultrasonic mode, unlocks the system, and displays the first interface on the screen. It should be understood that even when multiple fingerprints are collected (e.g., through multi-finger operation), fingerprint matching is successful as long as at least one fingerprint has a similarity greater than the preset matching threshold; otherwise, fingerprint matching fails. In ultrasonic mode, the user can interact with the terminal device using gestures within the ultrasonic sensing area.

[0227] Figure 8 is a schematic diagram of the first interface presented during the startup operation of another response ultrasonic mode provided in an embodiment of this application.

[0228] In one possible implementation, the terminal device is in a screen-on, unlocked state, and the user simultaneously presses and holds the volume up and volume down buttons. The terminal device responds to this ultrasonic mode activation, enters ultrasonic mode, and displays the first interface on the screen. In ultrasonic mode, the user can interact with the terminal device using gestures within the ultrasonic sensing area.

[0229] In one possible implementation, the terminal device is in a screen-on and locked state. The user first unlocks the system by entering a fingerprint or password, and then simultaneously presses and holds the volume up and volume down buttons. The terminal device responds to the ultrasonic mode activation operation, enters ultrasonic mode, and displays the first interface on the screen. In ultrasonic mode, the user can interact with the terminal device by performing gestures within the ultrasonic sensing area.

[0230] Figure 9 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0231] 910, Obtain the exit operation for ultrasonic mode.

[0232] The exit operation of ultrasonic mode includes at least one of the following: clicking the off icon in the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, or long-pressing the screen of the terminal device.

[0233] It should be understood that the exit operation and the start operation of ultrasonic mode can be the same or different. For example, a user can double-click the screen to enter ultrasonic mode, and then double-click the screen again to exit ultrasonic mode. For the exit operation of ultrasonic mode, please refer to the description of the start operation of ultrasonic mode in Figure 5; this application will not repeat it further.

[0234] 920, the second interface is displayed on the screen.

[0235] In response to the exit operation of ultrasonic mode, a second interface is displayed on the terminal device's screen. The second interface is the area of ​​the terminal device's screen used to sense user operations via a capacitive touchscreen.

[0236] The second interface may include any one or more of the following: application icon, application folder, application widget, wallpaper, search bar, bottom navigation bar, notification bar, theme animation, etc.

[0237] The area of ​​the second interface is the same as the screen area of ​​the terminal device, or the area of ​​the second interface can be smaller than the screen area of ​​the terminal device.

[0238] Figure 10 is a schematic diagram of the second interface presented during the exit operation of the response ultrasonic mode according to an embodiment of this application.

[0239] It should be understood that the exit operation of ultrasonic mode can be pre-entered by the user on the terminal device, or it can be configured by the terminal device before leaving the factory.

[0240] Users can exit ultrasonic mode on the terminal device when the screen is on, when the screen is off, when the device is locked, or when the device is unlocked. This application does not restrict the device state when performing the ultrasonic mode exit operation.

[0241] The closing indicator for the ultrasonic sensing area can be a closing icon or an indicator to exit ultrasonic mode, such as an "×", "Close", or "Exit" button.

[0242] The preset movement trajectories include, but are not limited to, swipes in different directions, numbers, English characters, and special symbols, such as "W", "M", "S", "ZZ", "C", "||", "|", "V", "<", ">", "&", "8", circles, triangles, hearts, etc. It should be understood that the preset movement trajectories in this application embodiment are not limited to movement trajectories obtained by single-finger operation, but can also be movement trajectories obtained by multi-finger operation.

[0243] The starting position of the movement trajectory can be within the ultrasonic sensing area or within a non-ultrasonic sensing area on the screen. The ending position of the movement trajectory can also be within the ultrasonic sensing area or within a non-ultrasonic sensing area on the screen. The non-ultrasonic sensing area refers to the area on the terminal device's screen other than the ultrasonic sensing area. This application does not restrict the specific starting and ending positions of the movement trajectory, as long as at least part of the movement trajectory falls within the ultrasonic sensing area.

[0244] Pressing the physical buttons on the terminal device includes, but is not limited to, pressing a single physical button or a combination of pressing multiple physical buttons. For example, the terminal device includes a volume up button, a volume down button, and a home button. Users can enter ultrasonic mode by long-pressing the home button, long-pressing the home button and the volume up button, long-pressing the home button and the volume down button, pressing the home button N2 times consecutively, or pressing the home button N2 times consecutively followed by pressing the volume up button M2 times consecutively, where N2 and M2 are integers greater than 1. It should be understood that the terminal device may also include more physical buttons. With more physical buttons, the terminal device can enter ultrasonic mode through more button combinations. The above examples should not be construed as limiting this application.

[0245] Multiple taps on the terminal device screen include single-finger double tap, single-finger triple tap, multi-finger double tap, multi-finger triple tap, and other operations.

[0246] Long-pressing the screen of a terminal device includes pressing the screen with a single finger for more than a preset time threshold, or pressing the screen with multiple fingers for more than a preset time threshold.

[0247] In Figure 10(a), the exit operation of the ultrasonic mode is for the user to click the "Exit" button on the first interface within the ultrasonic sensing area; in Figure 10(b), the exit operation of the ultrasonic mode is for the user to click the "×" mark on the first interface within the ultrasonic sensing area; in Figure 10(c), the exit operation of the ultrasonic mode is for the user to use multi-finger operation to slide a preset heart-shaped movement trajectory within the ultrasonic sensing area, with the entire heart-shaped movement trajectory falling within the ultrasonic sensing area; in Figure 10(d), the exit operation of the ultrasonic mode is for the user to press and hold the ultrasonic sensing area with one finger, or for the user to click the ultrasonic sensing area multiple times with one finger; in Figure 10(e), the exit operation of the ultrasonic mode is for the user to press the combination of the volume up button and the home button, for example, to press and hold the home button and the volume up button simultaneously, or to press the home button N2 times and then press the volume up button M2 times.

[0248] In response to the exit operation of any of the ultrasonic modes in Figures 10(a) to 10(e), the screen of the terminal device displays any of the second interfaces in Figures 10(f) to 10(j).

[0249] In Figure 10(f), the second interface includes multiple applications and a bottom navigation bar; in Figure 10(g), the second interface includes a search bar and multiple application widgets; in Figure 10(h), the second interface includes a search bar, multiple application widgets, and multiple applications; in Figure 10(i), the second interface includes multiple applications, application widgets, and a bottom navigation bar; in Figure 10(j), the second interface includes multiple applications, a search bar, and a bottom navigation bar.

[0250] In one possible implementation, if the ultrasonic sensing area and the screen area of ​​the terminal device are the same size, the interface displayed on the terminal device's screen can remain unchanged after the user exits the ultrasonic mode. The only difference is that the terminal device responds to the user's operation through the ultrasonic sensor in ultrasonic mode, and responds to the user's operation through the capacitive touchscreen after the terminal device exits the ultrasonic mode.

[0251] Figure 11 is a schematic diagram of a third interface presented in response to a user's swipe gesture, according to an embodiment of this application.

[0252] While the ultrasonic sensing area displays the first interface, the user's swipe gesture within the ultrasonic sensing area is acquired. The swipe gesture includes a left swipe, a right swipe, an up swipe, or a down swipe. In response to this swipe gesture, the terminal device displays a third interface within the ultrasonic sensing area, which differs from the first interface.

[0253] Users can use left, right, up, or down swipe gestures within the ultrasonic sensing area to perform page turning operations. The ultrasonic sensing area can also display a page transitioning from the first to the third screen.

[0254] The third interface includes any one or more of the following: application icon, application folder, application widget, wallpaper, search bar, bottom navigation bar, notification bar, multitasking status bar, and theme animation.

[0255] Figures 11(a) to 11(d) are schematic diagrams of the ultrasonic sensing area presenting the first interface, and Figures 11(e) to 11(h) are schematic diagrams of the ultrasonic sensing area presenting the third interface.

[0256] In Figure 11(a), the terminal device acquires the user's left swipe gesture in the ultrasonic sensing area, and the terminal device displays the third interface shown in Figure 11(e). The application list included in the third interface is different from the application list included in the first interface shown in Figure 11(a). In Figure 11(b), the terminal device acquires the user's right swipe gesture in the ultrasonic sensing area, and the terminal device displays the third interface shown in Figure 11(f). The application icons included in the third interface are different from the application icons included in the first interface shown in Figure 11(b). In Figure 11(c), the terminal device acquires the user's up swipe gesture in the ultrasonic sensing area, and the terminal device displays the third interface shown in Figure 11(g). The third interface includes a multi-tasking status bar, which is different from the first interface shown in Figure 11(c). In Figure 11(d), the terminal device acquires the user's down swipe gesture in the ultrasonic sensing area, and the terminal device displays the third interface shown in Figure 11(h). The third interface includes a notification bar, multiple application icons, and a bottom navigation bar, which is different from the first interface shown in Figure 11(d).

[0257] It should be understood that when a user swipes left or right on the first screen, the third screen displayed on the terminal device may also include application widgets; similarly, when a user swipes up or down on the first screen, the third screen displayed on the terminal device may include more application icons. The above examples should not be construed as limiting this application.

[0258] Figure 12 is a schematic diagram of the fourth interface presented in response to user long-press and drag operations according to an embodiment of this application.

[0259] The terminal device acquires the user's long-press operation and the trajectory of dragging the first application icon within the ultrasonic sensing area. The first application icon can be any application icon in the first interface. In response to the long-press operation and the dragging trajectory, the terminal device displays a fourth interface within the ultrasonic sensing area. The first application icon in the fourth interface is located at the endpoint of the trajectory. The endpoint of the trajectory is located within the ultrasonic sensing area.

[0260] For example, a user can drag the first application icon to move it to a location other than its original position on the first screen, or move it to another screen after swiping through the first screen.

[0261] In Figure 12(a), the user long-presses the camera icon in the application list and drags it to the position of the clock icon. In response to this long-press and drag operation, the terminal device displays the fourth interface shown in Figure 12(c) in the ultrasonic sensing area. The camera icon in the fourth interface is located at the end of the dragging trajectory, which is the original position of the clock icon. In Figure 12(b), the user long-presses the camera icon and drags it to the position of the contacts icon. In response to this long-press and drag operation, the terminal device displays the fourth interface shown in Figure 12(d) in the ultrasonic sensing area. The camera icon in the fourth interface is located at the end of the dragging trajectory, which is the original position of the contacts icon.

[0262] Figure 13 is a schematic diagram of the fifth interface presented in response to user click operations according to an embodiment of this application.

[0263] The terminal device receives a user's click on a second application identifier, which can be any application identifier in the first interface. In response to this click, the terminal device launches the application corresponding to the second application identifier and displays a fifth interface in the ultrasonic sensing area. The fifth interface is the interface of the application corresponding to the second application identifier.

[0264] After launching the corresponding application, the application interface can be displayed in full screen or only within the ultrasonic sensing area.

[0265] In Figure 13(a), when the user clicks the camera icon in the application list, the terminal device responds to the click and displays the fifth interface shown in Figure 13(d) in the ultrasonic sensing area. The fifth interface is the user interface of the camera application, where the user can click the photo, video, and other buttons in the ultrasonic sensing area to use different camera functions. In Figure 13(b), when the user clicks the calculator icon, the terminal device responds to the click and displays the fifth interface shown in Figure 13(e) in the ultrasonic sensing area. The fifth interface is the user interface of the calculator application, where the user can click the number, operator, and other buttons in the ultrasonic sensing area to use the calculation function. In Figure 13(c), when the user clicks the message icon, the terminal device responds to the click and displays the fifth interface shown in Figure 13(f) in the ultrasonic sensing area. The fifth interface is the user interface of the messaging application, where the user can reply to messages in the message input box.

[0266] Figure 14 is a schematic diagram of the sixth interface presented in response to the user's rotation operation according to an embodiment of this application.

[0267] While the first interface is displayed in the ultrasonic sensing area, the user's rotation of the terminal device is detected. In response to this rotation, the terminal device displays a sixth interface in the ultrasonic sensing area, which is the rotated first interface.

[0268] In one possible implementation scenario, the terminal device can be equipped with an accelerometer or gyroscope sensor, which can detect the current tilt angle of the terminal device, thereby controlling the switching between landscape and portrait modes.

[0269] For example, the first interface is a portrait interface and the sixth interface is a landscape interface, or the first interface is a landscape interface and the sixth interface is a portrait interface.

[0270] In Figure 14(a), the first interface is a portrait screen. When the user rotates the terminal device 90 degrees clockwise, the terminal device responds to the rotation operation and displays the sixth interface shown in Figure 14(b) in the ultrasonic sensing area. The sixth interface is a landscape screen. In Figure 14(c), the first interface is a landscape screen. When the user rotates the terminal device 90 degrees counterclockwise, the terminal device responds to the rotation operation and displays the sixth interface shown in Figure 14(d) in the ultrasonic sensing area. The sixth interface is a portrait screen.

[0271] The ultrasonic sensing area in this embodiment may display one or more of the following buttons: settings, troubleshooting, information saving, and emergency assistance. For example, any one of the first, second, third, fourth, fifth, and sixth interfaces of the ultrasonic sensing area may include buttons such as settings, troubleshooting, information saving, and emergency assistance.

[0272] It should be understood that the page turning operation, dragging application icon operation, launching application operation, and rotating operation in the above embodiments are only examples. Users can also perform operations such as two-finger zooming in and out of images, dragging multiple application icons to the icon folder, adding or deleting application widgets, and adding or deleting application icons in the ultrasonic sensing area. The above examples should not be construed as limiting this application.

[0273] Figure 15 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0274] 1510, the terminal device enters ultrasonic mode.

[0275] An application system refers to the overall system consisting of the operating system and application software on a terminal device.

[0276] Capacitive touchscreens use the principle of capacitance to sense human touch. A capacitive plate covers the screen of the terminal device. When a finger touches the screen, a capacitor is formed between the human body and the capacitive plate, changing the electric field distribution of the plate. The capacitive touchscreen determines the touch location and gesture by detecting these changes in capacitance.

[0277] The terminal device in this embodiment integrates an ultrasonic sensor in its display screen. This ultrasonic sensor can locate the coordinates of a finger on the screen, recognize the user's fingerprint, and identify touch events. The ultrasonic sensor emits ultrasonic pulses through the screen. These pulses are reflected on the finger and received. By measuring the reflection time between the finger and the screen, the coordinates of the finger are determined, thus achieving precise touch positioning. The ultrasonic sensor can also be used for under-display fingerprint recognition. Ultrasonic pulses are reflected on the fingerprint and received. The sensor converts the received ultrasonic data into a three-dimensional fingerprint image to recognize the user's fingerprint, enabling secure unlocking and authentication. The ultrasonic sensor can recognize touch events, including single-point touch, multi-point touch, and gesture operations, thereby enabling user interaction with the mobile phone screen.

[0278] In this embodiment, the user can initiate an ultrasonic mode activation operation to control the terminal device to enter ultrasonic mode. The ultrasonic mode activation operation can be pre-programmed by the user on the terminal device or pre-configured at the factory. This allows the terminal device to recognize the ultrasonic mode activation operation and enter ultrasonic mode when the capacitive touchscreen is malfunctioning or even fails.

[0279] For instructions on starting the ultrasonic mode, please refer to the descriptions in Figures 3 and 5, which will not be repeated here.

[0280] In one possible implementation, if the terminal device is in a locked state, unlocking is also required, such as fingerprint authentication or password authentication. This unlocking operation and the ultrasonic mode activation operation can be performed in parallel or sequentially; this application does not impose any restrictions on this.

[0281] Optionally, a detection step can be added to first detect whether the terminal device is covered by liquid or whether the capacitive touchscreen is malfunctioning. Only when the terminal device is confirmed to be covered by liquid or the capacitive touchscreen is malfunctioning should it enter ultrasonic mode. For example, after receiving the ultrasonic mode activation operation, the terminal device first determines that the capacitive touchscreen data is abnormal before entering ultrasonic mode.

[0282] When the capacitive touchscreen has poor touch operation or the system cannot respond to user commands due to capacitive touchscreen failure, ultrasonic touch operation can be performed using the method provided in this application embodiment, which makes up for the shortcomings of capacitive touchscreen and improves the user experience.

[0283] 1520, Display the application identifier in the ultrasonic sensing area.

[0284] The application system displays an application identifier within the effective range of the ultrasonic sensor. This application identifier can be an application icon or an application list.

[0285] 1530, Disable capacitive touchscreen.

[0286] In ultrasonic mode, touch reporting data of capacitive touch screen is disabled, and users interact with the terminal device based on ultrasonic sensor data in the ultrasonic sensing area of ​​the terminal device.

[0287] It should be understood that disabling the capacitive touchscreen is an optional step, and the application system may also choose not to use the touch point reporting data of the capacitive touchscreen and not disable the capacitive touchscreen.

[0288] It should be understood that steps 1520 and 1530 can be executed in parallel or in an interchangeable order; the step numbers are for illustrative purposes only.

[0289] 1540, The ultrasonic sensor reports touch coordinate information and touch events.

[0290] Optionally, the touch coordinate information can be directly integrated into the ultrasonic sensor for calculation, or the ultrasonic sensor can upload ultrasonic data to the application system for calculation.

[0291] Touch coordinate information can be the position of the user's finger on the screen, or the coordinates of the finger on the ultrasonic sensing area. Touch events include different events such as finger pressing, finger holding, finger movement, and finger lifting.

[0292] 1550, Application Interaction.

[0293] Application icons are displayed within the ultrasonic sensing area. Users can directly click on the application icon to launch the corresponding application. Please refer to the description in Figure 13 for details.

[0294] In one possible application scenario, within the ultrasonic sensing area, left / right / up / down swiping is supported to place more applications or display other content. Please refer to the description in Figure 11 for details.

[0295] In one possible application scenario, within the ultrasonic sensing area, it is possible to drag the application icon by long-pressing. Please refer to the description in Figure 12 for details.

[0296] In one possible application scenario, an off button can be added within the ultrasonic sensing area, allowing the user to exit the ultrasonic mode by clicking the off button. Please refer to the description in Figure 10 for details.

[0297] Figure 16 is an exemplary flowchart of another touch interaction method provided in an embodiment of this application.

[0298] 1610, Obtain the start operation of ultrasonic mode.

[0299] For instructions on starting the ultrasonic mode, please refer to the descriptions in Figures 3 and 5, which will not be repeated here.

[0300] In this step, the terminal device can initiate the ultrasonic mode activation operation even when it is not covered by liquid. This means the user can activate the ultrasonic mode before the capacitive touchscreen malfunctions or becomes unresponsive. For example, the user can activate the ultrasonic mode before immersing the terminal device in water.

[0301] 1620, the first interface is presented in the ultrasonic sensing area.

[0302] The description of the first interface can be found in Figures 3 and 6 to 8, and will not be repeated here.

[0303] 1630, the terminal device was detected to be covered by liquid.

[0304] Once the terminal device enters ultrasonic mode, it is covered by liquid, allowing users to interact with the terminal device through gestures within the ultrasonic sensing area.

[0305] The terminal device being covered by liquid includes, but is not limited to, any of the following scenarios: the terminal device's screen is wetted, the entire terminal device or part of the terminal device is submerged in water, or there is a large amount of oil on the terminal device's screen. This liquid includes, but is not limited to, rainwater, seawater, river water, pool water, tap water, beverages, and oil.

[0306] The above describes a touch interaction method according to embodiments of this application. The apparatus and device according to embodiments of this application are described below with reference to Figures 17 and 18. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to the foregoing method embodiments.

[0307] Figure 17 is a structural example diagram of a computer device 1700 provided in an embodiment of this application. This computer device 1700 may possess the functions of the terminal device in the above method embodiments and can be used to execute the steps performed by the functions of the terminal device in the above method embodiments. This function can be implemented by hardware, or by software or hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0308] In one possible implementation, the computer device 1700 includes an acquisition module 1710 and a processing module 1720. The acquisition module 1710 and the processing module 1720 are coupled to each other.

[0309] The acquisition module 1710 is used to acquire the start operation of the ultrasonic mode when the terminal device is covered by liquid or the capacitive touch screen is malfunctioning, so as to execute some or all of the steps in the method in Figure 3, the method in Figure 9, and the methods in Figures 7, 8, 10 to 16.

[0310] The processing module 1720 is used to respond to the start operation of the ultrasonic mode, present a first interface in the ultrasonic sensing area, and execute some or all of the steps in the methods of Figures 3, 7 to 16.

[0311] Figure 18 illustrates a terminal device 1800 provided in an embodiment of this application. As shown, the terminal device 1800 includes at least one processor 1810 and a transceiver 1820. The processor 1810 is coupled to a memory and is used to execute instructions stored in the memory to control the transceiver 1820 to transmit and / or receive signals.

[0312] Optionally, the terminal device 1800 also includes a memory 1830 for storing instructions.

[0313] In some embodiments, the processor 1810 and the memory 1830 can be combined into a single processing device, with the processor 1810 executing program code stored in the memory 1830 to achieve the aforementioned functions. Specifically, the memory 1830 can be integrated into the processor 1810 or independent of it.

[0314] In some embodiments, transceiver 1820 may include a receiver (or receiver unit) and a transmitter (or transmitter unit).

[0315] The transceiver 1820 may further include an antenna, and the number of antennas may be one or more. The transceiver 1820 may be a communication interface or an interface circuit.

[0316] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a terminal device, the terminal device performs the aforementioned method steps to implement the touch interaction method in the above embodiment.

[0317] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the touch interaction method in the above embodiment.

[0318] Furthermore, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. When the apparatus is a chip, the chip may include a connected processor and a memory. The memory stores computer execution instructions, and when the chip is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to perform the touch interaction methods in the above-described method embodiments.

[0319] In one possible implementation, the chip implementing the touch interaction method provided in this application includes a transceiver module and a processing module. The transceiver module can be an input / output circuit or a communication interface; the processing module can be a processor, microprocessor, or integrated circuit integrated on the chip.

[0320] In this embodiment, the terminal device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0321] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0322] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0323] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or split into more modules, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0324] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0325] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0326] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0327] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

A touch interaction method, characterized in that, The method, applied to a terminal device including a capacitive touchscreen and an ultrasonic sensor, comprises: When the terminal device is covered by liquid or the capacitive touch screen is malfunctioning, the ultrasonic mode activation operation is obtained. In response to the activation of the ultrasonic mode, a first interface is displayed in the ultrasonic sensing area of ​​the terminal device. The first interface includes at least one application identifier. The ultrasonic sensing area is the area of ​​the terminal device's screen used to sense user operations via the ultrasonic sensor. The method according to claim 1, characterized in that, The operation to activate the ultrasonic mode includes at least one of the following: sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, tapping the screen of the terminal device multiple times, or long-pressing the screen of the terminal device. The method according to claim 1 or 2, characterized in that, The method further includes: Get the exit operation for ultrasonic mode; In response to the exit operation of the ultrasonic mode, a second interface is displayed on the screen of the terminal device. The second interface is the area of ​​the terminal device screen used to sense user operation through the capacitive touch screen. The method according to claim 3, characterized in that, The operation to exit the ultrasonic mode includes at least one of the following: clicking the off icon in the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, or long-pressing the screen of the terminal device. The method according to any one of claims 1 to 4, characterized in that, The method further includes: When the first interface is displayed in the ultrasonic sensing area, the user's swiping gesture in the ultrasonic sensing area is acquired. The swiping gesture includes a left swipe gesture, a right swipe gesture, an up swipe gesture, or a down swipe gesture. In response to the swipe gesture, a third interface is presented in the ultrasonic sensing area, which is different from the first interface. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The operation of long-pressing the first application icon and the trajectory of dragging the first application icon are acquired in the ultrasonic sensing area. The first application icon is any application icon in the first interface. In response to the operation of long-pressing the first application icon and the trajectory of dragging the first application icon, a fourth interface is presented in the ultrasonic sensing area, and the first application icon in the fourth interface is located at the end position of the trajectory. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The user clicks on a second application identifier, where the second application identifier is any application identifier on the first interface; In response to the click on the second application identifier, a fifth interface is displayed in the ultrasonic sensing area. The fifth interface is the interface of the application corresponding to the second application identifier. The method according to any one of claims 1 to 7, characterized in that, The method further includes: While the first interface is displayed in the ultrasonic sensing area, the user's operation of rotating the terminal device is acquired. In response to the operation of rotating the terminal device, a sixth interface is presented in the ultrasonic sensing area, the sixth interface being the first interface after rotation. The method according to any one of claims 1 to 8, characterized in that, Before the first interface is presented in the ultrasonic sensing area of ​​the terminal device, the method further includes: Obtain the user's fingerprint or password. The method according to any one of claims 1 to 9, characterized in that, The method further includes: The ultrasonic sensing area displays one or more of the following buttons: Settings, Troubleshooting, Information Saving, and Emergency Assistance. A touch interaction method, characterized in that, The method, applied to a terminal device including a capacitive touchscreen and an ultrasonic sensor, comprises: Obtain the ultrasonic mode start operation; In response to the ultrasonic mode activation operation, a first interface is displayed in the ultrasonic sensing area of ​​the terminal device. The first interface includes at least one application identifier. The ultrasonic sensing area is the area of ​​the terminal device's screen used to sense user operations through the ultrasonic sensor. The terminal device was detected to be covered by liquid. A computer device, characterized in that, An apparatus for use in terminal devices including capacitive touchscreens and ultrasonic sensors, the apparatus comprising: The acquisition module is used to acquire the start operation of the ultrasonic mode when the terminal device is covered by liquid or the capacitive touch screen is malfunctioning. A processing module is configured to, in response to the activation operation of the ultrasonic mode, present a first interface in the ultrasonic sensing area of ​​the terminal device. The first interface includes at least one application identifier. The ultrasonic sensing area is the area of ​​the terminal device's screen used to sense user operations via the ultrasonic sensor. The apparatus according to claim 12 is characterized in that, The operation to activate the ultrasonic mode includes at least one of the following: sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, tapping the screen of the terminal device multiple times, or long-pressing the screen of the terminal device. The apparatus according to claim 12 or 13 is characterized in that, The acquisition module is also used to acquire the exit operation of the ultrasonic mode; The processing module is also configured to display a second interface on the screen of the terminal device in response to the exit operation of the ultrasonic mode. The second interface is an area of ​​the terminal device screen used to sense user operation through the capacitive touch screen. The apparatus according to claim 14 is characterized in that, The operation to exit the ultrasonic mode includes at least one of the following: clicking the off icon in the ultrasonic sensing area, sliding a preset movement trajectory on the screen of the terminal device, pressing a physical button on the terminal device, clicking the screen of the terminal device multiple times, or long-pressing the screen of the terminal device. The apparatus according to any one of claims 12 to 15 is characterized in that, The acquisition module is further configured to acquire the user's swiping gesture in the ultrasonic sensing area when the first interface is displayed in the ultrasonic sensing area, and the swiping gesture includes a left swipe gesture, a right swipe gesture, an up swipe gesture, or a down swipe gesture. The processing module is also configured to, in response to the swipe gesture, present a third interface in the ultrasonic sensing area, the third interface being different from the first interface. The apparatus according to any one of claims 12 to 16, characterized in that, The acquisition module is also used to acquire the user's operation of long-pressing the first application icon and the trajectory of dragging the first application icon in the ultrasonic sensing area, wherein the first application icon is any application icon in the first interface; The processing module is further configured to, in response to the operation of long-pressing the first application icon and the trajectory of dragging the first application icon, present a fourth interface in the ultrasonic sensing area, wherein the first application icon in the fourth interface is located at the end position of the trajectory. The apparatus according to any one of claims 12 to 17, characterized in that, The acquisition module is further configured to acquire the user's action of clicking on the second application identifier, wherein the second application identifier is any application identifier in the first interface; The processing module is further configured to, in response to the operation of clicking the second application identifier, present a fifth interface in the ultrasonic sensing area, the fifth interface being the interface of the application corresponding to the second application identifier. The apparatus according to any one of claims 12 to 18, characterized in that, The acquisition module is also used to acquire the user's operation of rotating the terminal device when the first interface is displayed in the ultrasonic sensing area; The processing module is further configured to, in response to the operation of rotating the terminal device, present a sixth interface in the ultrasonic sensing area, the sixth interface being the first interface after rotation. The apparatus according to any one of claims 12 to 19, characterized in that, The acquisition module is also used to acquire the fingerprint or password entered by the user. The apparatus according to any one of claims 12 to 20 is characterized in that, The processing module is also used to display one or more of the following buttons in the ultrasonic sensing area: settings, troubleshooting, information saving, and emergency help. A terminal device, characterized in that, include: A processor configured to be coupled to memory, read and execute instructions and / or program code in the memory to perform the method as described in any one of claims 1 to 11. A computer-readable medium, characterized in that, The computer-readable medium stores computer program code that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 11. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 11. A chip characterized in that, The device includes a processor and a memory, wherein the processor is configured to read instructions stored in the memory, and when the processor executes the instructions, causes the chip to implement the method of any one of claims 1 to 11.

Images