Interaction method, terminal equipment and computer readable storage medium

Abstract

The invention relates to the technical field of computers, in particular to an interaction method, terminal equipment and a computer readable storage medium. When a user performs touch operation on terminal equipment, a touch point coordinate is not determined in a touch area containing a plurality of touch points based on capacitance difference value data, but a touch image corresponding to the whole touch area is generated based on the capacitance difference value data. Furthermore, according to the interaction method provided by the invention, the terminal equipment can display the touch image at the corresponding position touched by the user, so that personalized and interesting touch interaction is provided for the user based on the touch images corresponding to various touch operations of the user.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to an interaction method, a terminal device, and a computer-readable storage medium. Background Art

[0002] With the development of electronic technology, various touch-interactive terminal devices have been widely used. For example, terminal devices such as mobile phones, tablet computers, smart watches, and laptop computers can all use touch interaction as the main human-computer interaction method.

[0003] Taking a touch screen (touch panel, TP) as an example, referring to Figure 1a the schematic diagram of the touch interaction scenario shown, when the tablet computer 10 displays the desktop 100, the user single-finger clicks on the touch screen of the tablet computer 10.

[0004] In a solution providing touch interaction, the tablet computer 10 can collect the original capacitance data of the touch screen in real time based on a touch panel sensor (TP sensor), and determine capacitance difference data representing capacitance changes according to the original capacitance data. Furthermore, according to the capacitance difference data, the contact coordinates corresponding to the user's single-finger click operation are determined, so as to provide corresponding touch interaction based on the contact coordinates.

[0005] For example, Figure 1b shows the capacitance difference data 101 of the touch screen when the user single-finger clicks on the touch screen of the tablet computer 10. Referring to Figure 1b , the capacitance difference data 101 includes capacitance differences of M×N touch points. Among them, the area 101a where the capacitance difference is greater than the preset threshold may be the user's touch area. Furthermore, the tablet computer 10 takes the coordinates of the touch point corresponding to the center of gravity in the area 101a as the contact coordinates corresponding to the user's touch operation. For example, the tablet computer 10 can determine that the coordinates of the contact point 101b are (15, 38).

[0006] Furthermore, the tablet computer 10 can determine that the foregoing contact point 101b is located in the display area of the text message icon 100a. Therefore, the tablet computer 10 can open the information interface 102 of the text message application in response to the user's single-finger click operation.

[0007] It can be understood that the terminal device can only provide corresponding touch interaction for preset touch operations. For example, the preset touch operations that the terminal device can respond to usually include single-finger click, single-finger swipe, two-finger pinch, two-finger joint tap, etc. However, for other non-preset touch operations, the terminal device usually determines them as accidental touches. For example, referring to Figure 1c, when the user presses the palm on the touch screen of the tablet computer 10, the tablet computer 10 determines that the touch operation of the user at this time is a mis-touch. At this time, the tablet computer 10 will not respond to operations such as the user's palm pressing. In this way, the above touch interaction solution cannot respond to some non-preset touch operations, resulting in a relatively single interaction method provided by the terminal device and lacking interest. Summary of the Invention

[0008] The present application provides an interaction method, which can provide personalized and interesting touch interaction to the user based on the touch images corresponding to various touch operations of the user.

[0009] In a first aspect, the present application provides an interaction method, which is applied to a first terminal device. The method includes:

[0010] Detecting a first touch operation of the user on the screen of the first terminal device, where the contact area of the user's first touch operation with the touch screen is a first contact area; displaying the first contact area on the screen; detecting an interaction instruction corresponding to the displayed first contact area, and performing an operation corresponding to the interaction instruction.

[0011] Herein, the foregoing first terminal device may be a terminal device capable of generating corresponding touch images based on various touch operations of the user and providing personalized and interesting touch interaction to the user based on the touch images, such as the following tablet computer 10 and the like. The screen of the first terminal device may be the following touch screen. The first touch operation may be an operation in which the user presses a touch part such as a finger or a palm on the touch screen or moves on the touch screen. For example Figure 2a , Figure 2f the single-finger sliding operation shown; Figure 2c , Figure 2e , Figure 4b , Figure 4d the palm pressing operation shown; Figure 4c the three-finger sliding operation shown, Figure 6d the palm sliding operation shown, etc., direct contact operations, as well as the indirect touch operations described below. Correspondingly, the foregoing first contact area may be a touch image corresponding to the user's first touch operation, which will not be listed one by one here.

[0012] In a possible implementation of the above first aspect, the first touch operation includes one or more second touch operations, and the contact area of the second touch operation with the touch screen is a second contact area; wherein, the first contact area corresponds to one or more second contact areas, and displaying the first contact area on the screen includes: sequentially displaying one or more second contact areas on the screen, where the second contact area has a corresponding set display duration.

[0013] Here, the foregoing first touch operation may be composed of one or more second touch operations, which is a touch operation that the user continuously performs on the touch screen for a certain period of time. The second touch operation is the touch operation corresponding to a certain moment within the duration of the continuous touch. Correspondingly, the second contact area may be a touch image corresponding to the user's second touch operation on the touch screen at a certain moment, such as several frames of touch images in the following text.

[0014] It can be understood that one or more second contact areas within the duration of the continuous touch may constitute the first contact area. For example, Figure 2a the shown touch trace 200b (as an example of the first contact area) may be composed of multiple touch images 200c; Figure 4e the shown touch animation 405 may be composed of multiple touch images including touch image 405a, touch image 405b, and touch image 405c; Figure 4f the shown touch image 406 may be composed of several frames of touch images. Here, the second contact areas constituting each first contact area in the following text will not be listed one by one.

[0015] Here, the foregoing sequentially displaying one or more second contact areas may be to display each frame of touch image on the touch screen in the generation time sequence of each frame of touch image. It can be understood that the generation time sequence of each frame of touch image is the time sequence of each second touch operation of the user. And different display effects can be created based on the display duration of the second contact area. For the specific display duration and display effects, please refer to the specific description in step 306 below, which will not be elaborated here.

[0016] In a possible implementation of the foregoing first aspect, the screen displays a first interface of a first application. Among them, detecting a first touch operation of the user on the screen of the first terminal device includes:

[0017] detecting a first touch operation of the user on the first interface; and, displaying a first contact area on the screen includes: displaying the first contact area in the first interface.

[0018] Here, the first application may be the following touch recording application, and the first interface of the first application may be the recording interface of the touch recording application.

[0019] In a possible implementation of the foregoing first aspect, the first interface includes a first control, and, detecting an interaction instruction corresponding to the displayed first contact area and performing an operation corresponding to the interaction instruction includes: detecting a first user operation of the user on the first control in the first interface and saving the displayed first contact area as a first image.

[0020] Here, the first control in the first interface can be the modification control 402a and the save control 402b described below. The first user operation can include a single-finger click operation on the modification control 402a and / or the save control 402b, etc. The first image can be the user image described below.

[0021] The process of saving the displayed first contact area as the first image can be to superimpose several second contact areas to generate the corresponding first contact area in the form of a static image after the user completes the first touch operation, and save the first contact area as the first image. For a specific example of saving the displayed first contact area as the first image, refer to the relevant description in step 306 below, and details will not be elaborated here.

[0022] In a possible implementation of the above first aspect, in a possible implementation of the above first aspect, the first interface includes a first control, and when an interaction instruction corresponding to the displayed first contact area is detected, the operation of executing the corresponding interaction instruction includes: detecting a first user operation on the first control in the first interface, and saving the process of sequentially displaying a plurality of second contact areas on the first interface as a first animated image.

[0023] Here, the first animated image can be the user animated image described below. The process of saving the process of sequentially displaying a plurality of second contact areas on the first interface as the first animated image, for example, can be to display the corresponding frames of the touch images corresponding to the trajectory 401a in real time at the corresponding positions on the touch screen to generate a touch animated image 405 in the form of an animated image, and save the touch animated image as the user animated image.

[0024] In a possible implementation of the above first aspect, after saving the displayed first contact area as the first image, it further includes: setting the first image as at least any one of the desktop wallpaper, lock screen wallpaper, application background image, avatar, emoji of the first terminal device, and sending the first image to the second terminal device.

[0025] In a possible implementation of the above first aspect, after saving the process of sequentially displaying a plurality of second contact areas on the first interface as the first animated image, it further includes: setting the first animated image as at least any one of the desktop wallpaper, lock screen wallpaper, application background image, avatar, emoji of the first terminal device, and sending the first animated image to the second terminal device.

[0026] Here, after generating the first image and / or the first animated image, the user can set the first image and / or the first animated image as the desktop wallpaper, the lock screen wallpaper, the application background image, the avatar, the emoji, or can also share the first image and / or the first animated image with other users. Here, the other users can be the users using the aforementioned second terminal device. Here, no restrictive description is made for the second terminal device, and any terminal device that can receive the first image and / or the first animated image can be the second terminal device.

[0027] Based on the above implementation, the user can press touch parts such as fingers and palms on the touch screen or move them on the touch screen to generate and display the first image and / or the first animated image representing touch traces such as fingerprints and palm prints. Furthermore, the user can set the first image and / or the first animated image as an image, a wallpaper, an avatar, an emoji, etc.

[0028] In a possible implementation of the above first aspect, the screen displays a second interface. Among them, detecting a first touch operation of the user on the screen of the first terminal device includes: detecting a first touch operation of the user on the second interface, and detecting an interaction instruction corresponding to the displayed first contact area. Performing the operation of the corresponding interaction instruction includes: detecting a first touch operation of the user on the second interface and displaying a first touch area in the second interface.

[0029] In a possible implementation of the above first aspect, displaying the first touch area in the second interface includes: sequentially displaying a plurality of second contact areas on the second interface.

[0030] In a possible implementation of the above first aspect, the second interface includes at least any one of the following: the desktop of the first terminal device, the lock screen interface, the application program interface.

[0031] Here, the second interface can be the display of the graphical user interface 202, the desktop 600, the text message interface 601, etc. It can be understood that when the second interface includes M interfaces, the first terminal device is in a split-screen display state. At this time, a plurality of second contact areas can be sequentially displayed based on N interfaces among the M interfaces. For example, when the second interface includes the desktop and the application program interface, a plurality of second contact areas only located on the desktop or the application program interface can be determined, and the aforementioned plurality of second contact areas are only displayed on the desktop or the application program interface.

[0032] Here, the process of sequentially displaying a plurality of second contact areas on the second interface can be seen in the specific description in step 505 below and will not be elaborated here.

[0033] Based on the above implementation, when the user presses and / or moves touch parts such as fingers and palms in the second interface displayed by the terminal device, the second interface can display a touch image in real time to visualize the user's touch traces.

[0034] In a possible implementation of the above first aspect, the first terminal device establishes a remote communication connection with the third terminal device based on the second application. Wherein, the third interface of the second application is displayed on the screen, and a first touch operation of the user on the screen of the first terminal device is detected, including: detecting a first touch operation of the user on the third interface; and displaying a first contact area on the screen, including: displaying the first contact area on the third interface.

[0035] Here, the third terminal device may be a terminal device such as the following tablet computer 20. The second application may be the following instant messaging application, phone application, etc. The third interface of the second application may be the following video interface 800, voice interface 802, etc.

[0036] In a possible implementation of the above first aspect, detecting an interaction instruction corresponding to the displayed first contact area includes: detecting the establishment of a remote communication connection between the first terminal device and the third terminal device, and / or detecting a second user operation of the user on a second control in the third interface; and performing an operation corresponding to the interaction instruction includes: sending the displayed first contact area to the third terminal device.

[0037] Here, the second control may be the following interaction control, for example, and the second user operation on the second control may be a click operation on the interaction control. For a specific example of sending the displayed first contact area to the third terminal device, refer to the specific description in step 705 below, and details are not elaborated here.

[0038] In a possible implementation of the above first aspect, displaying the first contact area on the third interface includes: sequentially displaying a plurality of second contact areas on the third interface; and sending the displayed first contact area to the third terminal device includes: sequentially sending the plurality of second contact areas sequentially displayed on the third interface to the third terminal device.

[0039] Here, the first terminal device may send touch images at each moment (as an example of each second contact area) frame by frame to the third terminal device. For example, the tablet computer 10 may sequentially send each frame of touch image constituting the touch trace 802a to the tablet computer 20 frame by frame.

[0040] In a possible implementation of the above first aspect, after sending the displayed first contact area to the third terminal device, the method further includes: receiving an instruction sent by the third terminal device; providing an output effect to the user based on the instruction, and / or displaying a dynamic effect on the third interface.

[0041] Here, the instruction sent by the third terminal device may be the instruction sent by the tablet computer 20 to the tablet computer 10 based on the remote communication connection in step 706 below.

[0042] In a possible implementation of the first aspect above, the output effects include at least any one of the following: sound, vibration, light, and backlight flashing.

[0043] In a possible implementation of the first aspect above, the dynamic effects include at least any one of the following: changing one or more combinations of the color, transparency, and line shape of the first contact area displayed in the third interface, displaying a preset pattern effect around the first contact area, and displaying a preset animation in the third interface.

[0044] In a possible implementation of the first aspect above, the first terminal device includes a plurality of touch sensors, where each touch sensor is used to collect the capacitance difference of one or more touch points, and the capacitance differences collected by the plurality of touch sensors form capacitance difference data; and the second contact area is determined in the following manner: determining a first capacitance matrix in the capacitance difference data based on the second touch operation and a preset threshold, where the first capacitance matrix includes several capacitance differences greater than the preset threshold; and converting the first capacitance matrix into a second touch area according to a preset image processing algorithm.

[0045] Here, the touch sensor can be, for example, the capacitance sensor in the following text. The capacitance difference data can be the capacitance difference data 100 and the capacitance difference data 403 in the following text. The preset threshold can be the touch threshold in the following text. The first capacitance matrix can be the area 101a and the capacitance matrix 403a in the following text.

[0046] In a possible implementation of the first aspect above, the first terminal device includes a plurality of capacitance difference intervals with corresponding color parameters, and converting the first capacitance matrix into a second touch area according to a preset image processing algorithm further includes: determining the capacitance difference interval to which each capacitance difference in the first capacitance matrix belongs; and determining the color of each pixel point in the second touch area according to the color parameter corresponding to the capacitance difference interval to which each capacitance difference belongs, where each pixel point in the second touch area corresponds to each capacitance difference in the first capacitance matrix.

[0047] Here, through the above implementation, the second touch area can present a color depth effect related to the distance between the user's touch part and the touch screen, making the displayed second touch area more realistic.

[0048] In a second aspect, the present application provides an interaction method applied to a third terminal device. The third terminal device establishes a remote communication connection with the first terminal device based on a second application. The screen of the third terminal device displays a fourth interface of the second application, and the method includes: receiving, based on the second application, a first contact area sent by the first terminal device; and displaying the first contact area in the fourth interface.

[0049] Here, the fourth interface of the second application may be the following video interface 801 and voice interface 803.

[0050] In a possible implementation of the second aspect above, receiving the first contact area sent by the first terminal device based on the second application includes: sequentially receiving a plurality of second touch areas based on the second application; wherein, displaying the first contact area in the fourth interface includes: sequentially displaying the plurality of second touch areas in the fourth interface.

[0051] In a possible implementation of the second aspect above, after displaying the first contact area in the fourth interface, the method further includes: detecting a third touch operation of the user on the fourth interface, wherein the contact area of the third touch operation with the touch screen is the third contact area; detecting that the third contact area coincides or partially coincides with the first contact area displayed in the fourth interface; providing an output effect to the user, and / or displaying a dynamic effect in the fourth interface; sending an instruction to the first terminal device.

[0052] Here, the third touch operation of the user of the third terminal device on the fourth interface is substantially the same as the first touch operation of the user of the first terminal device on the third interface, which will not be elaborated here. For the example of detecting that the third contact area coincides or partially coincides with the first contact area displayed in the fourth interface, specific details can be found in the following description of step 706, which will not be elaborated here.

[0053] It can be understood that the third touch operation may include one or more fourth touch operations, and the contact area of the fourth touch operation with the touch screen is the fourth contact area; wherein, the third contact area corresponds to one or more fourth contact areas. The third terminal device can also display the third contact area on the screen. Here, displaying the third contact area may include: sequentially displaying one or more fourth contact areas on the screen, wherein the fourth contact area has a corresponding set display duration.

[0054] Here, the foregoing third touch operation may be composed of one or more fourth touch operations, which is a touch operation of the user on the touch screen for a certain duration. The fourth touch operation is the touch operation corresponding to a certain moment within the duration of the continuous touch. Correspondingly, the fourth contact area may be the touch image corresponding to the user's fourth touch operation on the touch screen at a certain moment, for example, it may be several frames of touch images corresponding to one third touch operation.

[0055] Furthermore, detecting that the third contact area coincides or partially coincides with the first contact area displayed in the fourth interface may be detecting that at least one fourth contact area coincides or partially coincides with at least one second contact area.

[0056] In a third aspect, the present application provides a first terminal device, including: one or more processors; one or more memories; one or more programs are stored in the one or more memories, and when the one or more programs are executed by the one or more processors, the first terminal device is caused to execute the interaction methods provided by the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect.

[0057] In a fourth aspect, the present application provides a third terminal device, characterized by including: one or more processors; one or more memories; one or more programs are stored in the one or more memories, and when the one or more programs are executed by the one or more processors, the third terminal device is caused to execute the interaction methods provided by the above-mentioned second aspect and various possible implementations of the above-mentioned second aspect.

[0058] In a fifth aspect, the present application provides a computer-readable medium, characterized in that instructions are stored on the readable medium, and when the instructions are executed on a computer, the computer is caused to execute the interaction methods provided by the above-mentioned first aspect, second aspect, and various possible implementations of the above-mentioned first aspect and second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1a Shown is a schematic diagram of a touch interaction scenario provided by the present application;

[0060] Figure 1b Shown is a schematic diagram of capacitance difference data of a terminal device provided by the present application;

[0061] Figure 1c Shown is a schematic diagram of a touch operation provided by the present application;

[0062] Figure 2a Shown is a schematic diagram of the effect of generating a touch trace provided by an embodiment of the present application;

[0063] Figure 2b Shown is a schematic diagram of the effect of generating a touch trace provided by the present application;

[0064] Figure 2c Shown is a schematic diagram of the interaction effect in a touch interaction scenario provided by an embodiment of the present application;

[0065] Figure 2d Shown is a schematic diagram of the application scenario of a palm print picture provided by an embodiment of the present application;

[0066] Figure 2e Shown is a schematic diagram of the interaction effect in another touch interaction scenario provided by an embodiment of the present application;

[0067] Figure 2f Shown is a schematic diagram of the interaction effect in yet another touch interaction scenario provided by an embodiment of the present application;

[0068] Figure 3 The figure shows a schematic flowchart of generating a user picture based on a touch operation provided by an embodiment of the present application;

[0069] Figure 4a The figure shows a schematic interface diagram of a recording interface provided by an embodiment of the present application;

[0070] Figure 4b The figure shows another schematic diagram of a touch operation provided by an embodiment of the present application;

[0071] Figure 4c The figure shows still another schematic diagram of a touch operation provided by an embodiment of the present application;

[0072] Figure 4d The figure shows a schematic process diagram of generating a touch image based on capacitance difference data provided by an embodiment of the present application;

[0073] Figure 4e The figure shows a schematic display diagram of a touch image provided by an embodiment of the present application;

[0074] Figure 4f The figure shows another schematic display diagram of a touch image provided by an embodiment of the present application;

[0075] Figure 5 The figure shows a schematic flowchart of generating a touch trace based on a touch operation provided by an embodiment of the present application;

[0076] Figure 6a The figure shows a schematic interface diagram of a desktop provided by an embodiment of the present application;

[0077] Figure 6b The figure shows a schematic diagram of the effect of displaying a touch image in a split - screen mode provided by an embodiment of the present application;

[0078] Figure 6c The figure shows another schematic diagram of the effect of displaying a touch image in a split - screen mode provided by an embodiment of the present application;

[0079] Figure 6d The figure shows a schematic diagram of the effect of displaying a touch trace provided by an embodiment of the present application;

[0080] Figure 7 The figure shows a schematic flowchart of real - time sharing of a touch image based on remote communication provided by an embodiment of the present application;

[0081] Figure 8a The figure shows a schematic diagram of interaction for a video call provided by an embodiment of the present application;

[0082] Figure 8bThe figure shows an interaction schematic diagram for making a voice call provided by an embodiment of the present application;

[0083] Figure 8c The figure shows a touch operation schematic diagram during a video call provided by an embodiment of the present application;

[0084] Figure 8d The figure shows a touch operation schematic diagram during a voice call provided by an embodiment of the present application;

[0085] Figure 8e The figure shows a touch image schematic diagram generated during a video call provided by an embodiment of the present application;

[0086] Figure 8f The figure shows a touch image schematic diagram generated during a voice call provided by an embodiment of the present application;

[0087] Figure 9a The figure shows an effect schematic diagram of real-time sharing of touch traces during a video call provided by an embodiment of the present application;

[0088] Figure 9b The figure shows another effect schematic diagram of real-time sharing of touch traces during a video call provided by an embodiment of the present application;

[0089] Figure 9c The figure shows yet another effect schematic diagram of real-time sharing of touch traces during a video call provided by an embodiment of the present application;

[0090] Figure 9d The figure shows an effect schematic diagram of real-time sharing of touch traces during a voice call provided by an embodiment of the present application;

[0091] Figure 9e The figure shows another effect schematic diagram of real-time sharing of touch traces during a voice call provided by an embodiment of the present application;

[0092] Figure 10a The figure shows a system architecture schematic diagram related to implementing the interaction method provided by the present application in a terminal device;

[0093] Figure 10b The figure shows a process schematic diagram for generating a user picture based on capacitance difference data provided by an embodiment of the present application;

[0094] Figure 10c The figure shows a process schematic diagram for generating an implemented touch trace based on capacitance difference data provided by an embodiment of the present application;

[0095] Figure 10dThe figure shows a schematic flowchart of a process for real-time sharing of user touch operations based on capacitance difference data and remote communication provided by an embodiment of the present application;

[0096] Figure 10e The figure shows a schematic diagram of a system architecture related to implementing the interaction method provided by the present application in another terminal device provided by the present application;

[0097] Figure 11 The figure shows a schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Detailed implementation manners

[0098] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification and specific implementation manners.

[0099] It can be understood that the terminal device in the embodiments of the present application may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, smart TV, wearable device, tablet computer (Pad), computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0100] Based on the foregoing content, it can be known that when a user wants the terminal device to also provide touch interaction for non-preset touch operations, the foregoing solutions cannot meet the user's needs. For example, the user may want to display the trace of the touch operation on the display screen, set the touch image corresponding to the touch operation as an avatar or wallpaper, or want to share the touch image corresponding to the touch operation with other users in real time based on remote communication to achieve interesting interactions such as high-fiving through the air.

[0101] Therefore, to solve the problem that the above touch control solution cannot provide touch interaction for non - preset touch operations, the present application provides an interaction method. When a user performs a touch operation on a terminal device, instead of determining a contact coordinate in a touch area containing multiple touch points based on the aforementioned capacitance difference data, a touch image corresponding to the entire touch area is generated based on the capacitance difference data. Furthermore, the terminal device can display the touch image at the corresponding position where the user touches, so as to provide personalized and interesting touch interaction to the user based on the touch images corresponding to various touch operations of the user. It can be understood that various touch operations of the user can include the aforementioned preset touch operations such as single - finger click, single - finger swipe, two - finger pinch, two - finger joint tap, etc., and also include non - preset touch operations such as palm press.

[0102] For example, in a personalized and interesting touch interaction scenario, the user can press a touch part such as a finger or a palm on the touch screen or move it on the touch screen to generate and display touch traces such as fingerprints and palm prints. The user can save the touch trace and set it as an image, wallpaper, etc.

[0103] It can be understood that when the interaction method provided by the embodiments of the present application is applied to a terminal device such as a tablet computer, the terminal device can implement the above - mentioned interaction method through a pre - installed application program. The application program can be a touch recording application or other applications with similar functions of generating touch images, which is not limited herein. In some other embodiments, the system of the terminal device can also provide a service or a callable function module with a function of generating touch images similar to the above - mentioned touch recording application to support other applications running on the terminal device to call the service or the function module to implement the interaction method provided by the present application. The above - mentioned other applications can be, for example, Changlian TM WeChat TM and other instant messaging applications, which is not limited herein.

[0104] Specifically, taking the example of a user moving a single finger on the touch screen, Figure 2a According to the embodiments of the present application, a schematic diagram of the display effect of generating a touch trace in this touch interaction scenario is shown.

[0105] See Figure 2a, taking the tablet computer 10 running the foregoing touch recording application to apply the interaction method provided by the present application as an example, the tablet computer 10 displays the acquisition interface 200a of the touch recording application. The user can record the touch traces corresponding to the single-finger sliding operation in the acquisition interface 200a. Specifically, during the process of the user's single-finger sliding, the tablet computer 10 can generate a touch image corresponding to the entire touch area based on the capacitance difference data corresponding to each moment, referring to the touch trace 200b. Among them, the touch trace 200b can be formed by the tablet computer 10 continuously displaying the touch images corresponding to each moment in the acquisition interface 200a. Furthermore, the tablet computer 10 can save the touch trace 200b for the user to use.

[0106] It can be understood that the capacitance difference data can accurately reflect the touch position of the user on the touch screen and the size of the touch area. Therefore, the display position of the touch image generated based on the capacitance difference data can accurately reflect the position of the contact area between the user and the touch screen, the size of the touch image can accurately reflect the area of the contact area between the user and the touch screen, and the shape of the touch image can accurately reflect the shape of the contact area between the user and the touch screen.

[0107] In addition, the magnitude of the capacitance value of the capacitance difference data can characterize the distance between different touch parts of the user and the touch screen. Here, the touch part refers to the part where the distance between the user and the touch screen is relatively close so that the touch sensor of the touch screen can detect the capacitance change, including the touch part actually touching the touch screen, and also including the part that does not touch the touch screen but is relatively close to the touch screen, so that the touch sensor can detect the capacitance change. Taking the user's finger touching the touch screen as an example, the finger area of the finger touching the touch screen and the finger edge area that does not touch the touch screen but is relatively close to the touch screen may both cause the touch sensor to detect the capacitance change. Here, the capacitance difference data of the finger area touching the touch screen is generally greater than the capacitance difference data of the finger edge area not touching the touch screen. It can be understood that generally, the greater the pressing force of the user on the touch screen, the larger the area of the touch part actually touching the touch screen. Therefore, the terminal device can also set different color parameters based on different capacitance value intervals to present a visual effect with different shades in the touch image, so as to more vividly reflect the touch operation of the user on the touch screen.

[0108] The above touch images constituting the touch trace 200b can be referred to Figure 2a the shown touch image 200c. The touch image 200c can be a fingerprint image generated corresponding to the user's contact with the touch screen at a certain moment. Continuing to refer to Figure 2a, the size of the touch image 200c can characterize the area size of the contact area between the user and the touch screen, the shape of the touch image 200c can characterize the fingerprint shape when the user touches the touch screen, and the color depth of the touch image 200c can characterize the distance between different touch parts of the user's finger and the touch screen.

[0109] For comparison, Figure 2b shows a schematic diagram of a touch trace generated by a solution that provides touch interaction based on contact coordinates. Still taking the example of a user moving a single finger on the touch screen, refer to Figure 2b , during the process of the user sliding a single finger, the tablet computer 10 can determine corresponding contact coordinates in the touch area based on the capacitance difference data corresponding to each moment. Furthermore, the tablet computer 10 can continuously display the contact coordinates corresponding to each moment in the acquisition interface 200a to form Figure 2b the touch trace 200d shown. The touch images that make up the touch trace 200d can refer to Figure 2b the touch image 200e shown. The shape of this image 200e is generally a preset shape such as a circle, and it cannot reflect the contact area shape when the user touches the touch screen. Also, the size of this image 200e is generally determined by a preset line thickness parameter and cannot accurately reflect the size of the contact area between the user and the touch screen. Additionally, the color of this image 200e is generally determined by a preset color and cannot reflect the distance between different touch parts of the user's finger and the touch screen.

[0110] It can be understood that this application does not make a restrictive description of the type of touch operation of the user in the acquisition interface 200a. For example, refer to Figure 2c another schematic diagram of generating a touch trace shown, the touch operation of the user in the acquisition interface 200a can also be a palm pressing operation. Furthermore, the tablet computer 10 can display the palm print picture 200f in the acquisition interface 200a, and the user can save this palm print picture 200f. In one example scenario, refer to Figure 2d , after the user saves the palm print picture 200f, the palm print picture 200f can be set as the avatar of the application A to be displayed in the chat interface 201 of the application A. Here, the application A can be WeChat TM , QQ TM and other instant messaging applications, or can also be the SMS application of the tablet computer 10, etc. This application does not make a restrictive description of the application A.

[0111] It can be understood that after the user saves the palm print picture 200f, the palm print picture 200f can also be set as the desktop wallpaper, lock screen wallpaper, interface wallpaper, made into a meme, etc. Here, no restrictive description is made of the specific scenario where the user applies the palm print picture 200f.

[0112] For example, in another scenario of personalized and interesting touch interaction, the user can press and / or move touch parts such as fingers and palms on the interface displayed by the terminal device, and the interface displays a touch image in real time to visualize the user's touch traces.

[0113] Specifically, the application program and / or system process of the terminal device can collect the aforementioned capacitance difference data at a preset frequency to generate a real-time touch image. Moreover, the application interface of the application program and / or the graphical user interface of the system can display the touch image in real time to achieve an effect of displaying the user's touch traces.

[0114] As an example, Figure 2e According to the embodiments of the present application, a schematic diagram of the effect of this interaction solution is shown.

[0115] See Figure 2e , the system process A of the tablet computer 10 is used to display the graphical user interface 202. When the user presses the palm on the touch screen that displays the graphical user interface 202, the system process A can generate a palm print image 202a corresponding to the touch operation and display it at the corresponding position of the touch operation in the graphical user interface 202. It can be understood that when the user's palm moves, the display position of the palm print image 202a in the graphical user interface 202 can also change correspondingly. Furthermore, based on the interaction solution provided by the present application, the tablet computer 10 can visualize various touch operations of the user and can set the image generated corresponding to the user's touch operation as the interface background image, etc., improving the interest of touch interaction.

[0116] It can be understood that the real-time touch image can also be displayed in the interface of the application program. The above solution of displaying the real-time touch image in the graphical user interface is only an example.

[0117] For another example, in another scenario of personalized and interesting touch interaction, the user can press and / or move touch objects such as fingers and palms on the interface of the instant messaging application. The instant messaging application running on the terminal device can generate corresponding touch traces based on the interaction method provided by the present application and can share the generated touch traces to other terminal devices for remote communication, so that users in remote communication can see each other's touch traces.

[0118] As an example, Figure 2f According to the embodiments of the present application, a schematic diagram of the effect of this interaction solution is shown.

[0119] See Figure 2f, taking the tablet computers 10 and 20 used for remote communication as examples of terminal devices. During the process of remote communication between the user of the tablet computer 10 and the user of the tablet computer 20, the tablet computer 10 can send the touch image of the user to the tablet computer 20 based on this remote communication. The tablet computer 20 can display this touch image in the display interface of this remote communication, so that the user of the tablet computer 20 can see the touch traces of the user of the tablet computer 10. For example, patterns or handwriting drawn by the user of the tablet computer 10 can be seen. Herein, the aforementioned remote communication includes video calls, voice calls, etc.

[0120] Continue to refer to Figure 2f , taking the aforementioned remote communication as a video call as an example. The tablet computer 10 and the tablet computer 20 conduct a video call based on the application C. The tablet computer 10 displays the video interface 203, and the tablet computer 20 displays the video interface 204.

[0121] When the user of the tablet computer 10 touches the touch screen on which the video interface 203 is displayed, the application C can generate the touch image at this time and send the touch image to the tablet computer 20 through the application server. For example, when the user of the tablet computer 10 makes a single-finger swipe on the touch screen on which the video interface 203 is displayed to generate a heart-shaped touch trace, correspondingly, the application C generates multiple touch images corresponding to this heart-shaped touch trace. Furthermore, the application C of the tablet computer 20 can display the heart-shaped pattern 204a in the video interface 204 based on the received multiple touch images. In this way, the interactive experience of the user during the remote communication process can be enriched and the interestingness can be improved.

[0122] It can be understood that the touch operation of the user includes a direct contact operation in which the user directly physically touches the touch screen, such as directly touching the touch screen of the terminal device with body parts such as fingers and palms. It also includes an indirect touch operation of the user on the touch screen through a touch object. Herein, the aforementioned touch object includes touch tools such as a stylus and touchscreen gloves, as well as any other conductive medium. In addition, if the touch screen can detect the user's air operation based on a self-capacitance sensor, the touch operation of the user can also be the aforementioned air operation of hovering above the touch screen or moving while hovering above the touch screen. It can be understood that the interactive method provided by the embodiments of the present application can generate a touch image corresponding to the touch operation in any way of touch operation. Furthermore, the personalized touch interaction function as described above can be provided for the user based on this touch image.

[0123] Next, the implementation process of the interactive method provided by the present application will be described in detail in combination with specific embodiments.

[0124] Specifically, Example 1 below provides a specific implementation process of generating a user picture based on a user's touch operation with a tablet computer as the execution entity through the interaction method provided in this application.

[0125] Example 1

[0126] This embodiment will specifically illustrate the specific process of implementing an interaction effect of generating a user picture based on a user's touch operation in conjunction with the accompanying drawings.

[0127] Figure 3 According to the embodiment of this application, a schematic flowchart of a terminal device generating a user picture based on a user's touch operation is shown. It can be understood that Figure 3 The execution entity of each step of the shown process can be the aforementioned tablet computer 10. For the sake of convenience of description, the execution entity of each step will not be described in detail hereinafter when introducing each step.

[0128] It should also be stated that the numbering of the steps in the method and process in the embodiment of this application is for the convenience of reference, rather than limiting the sequence. If there is a sequence between the steps, it shall be subject to the written description.

[0129] Specifically, the interaction method provided in the embodiment of this application that can generate a user picture may include the following steps:

[0130] 301: Run the touch recording application.

[0131] Exemplarily, the tablet computer 10 can run the touch recording application (as an example of the first application) in response to a user operation, so that the user can input images such as palm prints and finger prints corresponding to the touch operation. Here, the touch recording application can be a system application installed in the tablet computer 10 or a third-party application.

[0132] 302: Display the recording interface.

[0133] Exemplarily, the tablet computer 10 can display the recording interface of the touch recording application, and the user can input images such as palm prints and finger prints corresponding to the touch operation in this recording interface.

[0134] Figure 4a According to the embodiment of this application, a schematic diagram of a recording interface is shown.

[0135] See Figure 4a, the tablet computer 10 can display a recording interface 400 (as an example of the first interface). The recording interface 400 may include a recording area 401. The user can press a finger, palm or other part on the touch screen area where the recording area 401 is displayed (as an example of the first touch operation) to record the touch image corresponding to the touch operation. Or, move a finger, palm or other part within the touch screen area (as an example of the first touch operation) to record the touch animation corresponding to the touch trace. In addition, when the user finishes the touch operation on the touch screen, the generated touch image or touch animation (as an example of the first contact area displayed) can be displayed in the recording area 401.

[0136] The recording interface 400 may further include an operation area 402. The user can modify and / or save the generated touch image or touch animation based on the modification control 402a and / or the save control 402b provided in the operation area 402 (as an example of the first control). For example, the user can single - finger click the save control 402b (as an example of the first user operation) to save the displayed touch image or touch animation as a user picture or user animation in the gallery (as an example of the first image). It can be understood that the above - mentioned user operation of single - finger clicking the save control 402b is only an example, and this application does not make restrictive descriptions on the user operations for saving touch images or touch animations.

[0137] 303: Collect the original capacitance data generated by the user's touch operation.

[0138] Exemplarily, the tablet computer 10 can collect the original capacitance data of the touch screen based on a touch sensor (or capacitance sensor) to detect the user's touch operation. Here, for the sake of narrative coherence, the process of the tablet computer 10 collecting the original capacitance data will be specifically described in the following description of Figure 10a and will not be elaborated here.

[0139] As an example, Figure 4b According to the embodiments of the present application, a schematic diagram of a user's touch operation is shown.

[0140] See Figure 4b , the user can press the palm on the recording area 401. The tablet computer 10 collects the original capacitance data corresponding to the pressing action and can generate a palm print image corresponding to the pressing action based on the following steps 304 to 306.

[0141] Figure 4c According to the embodiments of the present application, another schematic diagram of a user's touch operation is shown.

[0142] See Figure 4c, the user can touch the touch screen showing the recording area 401 with three fingers and slide the three fingers in the recording area 401 to form a circular trajectory 401a (as an example of the first touch operation). Specifically, the tablet computer 10 can collect a continuous number of frames of original capacitance data during the process of the user moving the fingers to form the trajectory 401a, and generate a touch image or a touch animated image corresponding to the trajectory 401a (as an example of the first contact area for display) based on the following steps 304 to 306 according to the original capacitance data of several frames.

[0143] Here, it can be understood that the user's touch operation (for example, the first touch operation) includes a direct contact operation in which the user directly physically touches the touch screen, such as directly touching the touch screen of the terminal device with a human body part such as a finger or a palm. It also includes an indirect touch operation of the touch screen by the user through a touch object. Here, the aforementioned touch object includes a touch pen, a touch screen glove and other touch screen tools, as well as any other conductive medium. In addition, if the touch screen can detect the user's air operation based on a self-capacitance sensor, the user's touch operation can also be a hovering operation above the touch screen or a moving air operation floating above the touch screen. It can be understood that the interaction method provided by the embodiments of the present application can generate a touch image corresponding to any touch operation. Furthermore, the personalized touch interaction function as described above can be provided to the user based on the touch image. Here, no restrictive description is made on the user's touch operation method.

[0144] 304: Generate capacitance difference data based on the original capacitance data.

[0145] Exemplarily, after the tablet computer 10 collects the original capacitance data, it can generate capacitance difference data based on the reference capacitance data when there is no touch operation on the touch screen. Here, for the sake of narrative coherence, the process of the tablet computer 10 generating capacitance difference data will be specifically described in the following description of Figure 10a how to generate capacitance difference data of the tablet computer 10.

[0146] For the touch operation of the user pressing the recording area 401 with the palm (as an example of the first touch operation), the tablet computer 10 can generate capacitance difference data corresponding to the palm pressing operation based on the original capacitance data when the user presses the recording area 401 with the palm.

[0147] For the touch operation of the user forming the trajectory 401a based on the three-finger operation, the tablet computer 10 can generate several frames of capacitance difference data corresponding to the trajectory 401a based on the continuous several frames of original capacitance data collected during the process of the user moving the fingers.

[0148] 305: Generate a touch image based on the capacitance difference data.

[0149] Exemplarily, the touch recording application of the tablet computer 10 can continuously obtain the foregoing several frames of capacitance difference data during the process of the user moving a finger. Further, after obtaining each frame of capacitance difference data, each frame of capacitance difference data can be converted into each frame of touch image based on an image processing algorithm.

[0150] Specifically, Figure 4d According to an embodiment of the present application, a schematic diagram of generating a touch image based on capacitance difference data is shown.

[0151] See Figure 4d , the capacitance difference data is generated based on a touch operation in which a user presses a palm on a touch screen. It can be understood that the capacitance difference data is a capacitance matrix composed of capacitance differences of several touch points. One touch sensor can correspond to one or more touch points, and the capacitance difference of each touch point can be determined based on the original capacitance value collected by its corresponding touch sensor. If the capacitance difference is greater than a preset touch threshold, it indicates that the touch position of the user includes the sensing position of the touch point corresponding to the capacitance difference. Figure 4d The capacitance values of each touch point involved in the capacitance difference data 403 are not shown. It can be understood that the matrix structure of the capacitance difference data 403 is substantially the same as the matrix structure of the capacitance difference data 100 shown above, and details are not described herein again. Figure 1a Shown above

[0152] Here, after obtaining the capacitance difference data, the touch recording application can first preprocess the capacitance difference data. For example, if the tablet computer 10 is in a floating ground state without electrical connection to the ground, it may cause the capacitance differences corresponding to some capacitance sensors to be negative. At this time, if a touch image is directly generated based on the capacitance difference data, the generated touch image may not be able to fully reflect the user's touch operation. Therefore, the negative capacitance differences are compensated to be positive based on the preprocessing of the capacitance difference data to improve the integrity of the touch image.

[0153] Continue to refer to Figure 4d, the tablet computer 10 can generate corresponding capacitance difference data 403 according to the pressing operation of the user's palm. Specifically, the preset touch threshold of the tablet computer 10 can be 1000. Therefore, when the user's palm presses on the touch screen, all the touch points in the capacitance difference data 403 with capacitance differences greater than 1000 can form a palm-shaped capacitance matrix 403a. Furthermore, the touch recording application of the tablet computer 10 can process the capacitance matrix 403a into a touch image 403b (as an example of the first contact area for display) through an image processing algorithm. The aforementioned image processing algorithm includes, but is not limited to, one or more of an interpolation algorithm, a boundary calibration algorithm, a blur algorithm, a coloring rendering algorithm, etc. Among them, the interpolation algorithm can be used to make the color transition of the generated touch image more natural; the boundary calibration algorithm can accurately calculate the image edge position in the touch image; the blur algorithm can make the image edge position determined by the boundary calibration algorithm smoother; the coloring rendering algorithm can dye the touch image. For example, by using a non-linear coloring curve for coloring rendering calculation, the generated touch image can be made more hierarchical.

[0154] It can be understood that the larger the capacitance value may represent that the distance between the touch part of the user on the touch screen and the touch screen is closer. Therefore, the touch recording application can also divide the capacitance values into several capacitance difference intervals, and each capacitance difference interval can correspond to different color parameters. For example, different capacitance difference intervals can correspond to color parameters with different transparencies. Furthermore, the touch image can present a color depth effect related to the distance between different touch parts of the user and the touch screen. Through the above method, the touch image can be made more realistic. For example, referring to Figure 4d the touch image 403b shown, during the process of the user's touch operation, when the capacitance value of the touch screen is greater than 1500, the touch image at the corresponding moment can enhance the color depth, so that the touch image 403b presents different color change effects of light and shade related to the distance between different touch parts of the user and the touch screen. When the user finishes the touch operation on the touch screen, the touch recording application can display the touch image 403b in the recording area 401 of the recording interface 400. It can be understood that different capacitance difference intervals can also correspond to color parameters of different colors. Furthermore, the touch image can present different color effects related to the distance between different touch parts of the user and the touch screen. For example, the image area corresponding to the touch part closer to the touch screen has a warmer tone, and the image area corresponding to the touch part farther from the touch screen has a colder tone. Here, the present application does not make restrictive explanations on the color parameter differences corresponding to each capacitance difference interval. The differences can also be grayscale, saturation, color temperature, etc. In addition, each capacitance difference interval can also correspond to different display parameters. The aforementioned display parameters can be filling patterns, line shapes, sharpness, brightness, contrast, etc. Here, no restrictive explanations are made on the specific types of display parameters.

[0155] The recording interface 400 may further include a selection area 404. The selection area 404 may include operation options such as "fill" and "line". The user may adjust effects such as the color of the touch image based on these operation options. The operation options may further include rotation, zoom, blur, sharpen, contrast adjustment, saturation adjustment, etc. Here, no restrictive description is made on the adjustment content of the touch image that the touch recording application can provide.

[0156] 306: Save the touch image as a user picture.

[0157] Exemplarily, after generating a number of frame touch images during the user's real-time touch operation based on the above steps 301 to 305, the touch recording application may generate a touch image or a touch animation corresponding to the touch operation based on the number of frame touch images. After generating a touch image or a touch animation corresponding to the user's touch operation (as an example of the displayed first contact area), the touch image or the touch animation may be saved as a user picture or a user animation in the picture library (as an example of the first image) so that the user can apply the user picture or the user animation to multiple usage scenarios. For example, as shown in the foregoing Figure 2b shown, the user may set the user picture representing the palm print as the personal account avatar for logging in to the instant messaging application.

[0158] Exemplarily, the touch recording application may generate a touch image or a touch animation corresponding to the trajectory 401a based on a number of frame touch images during the process of the user's three-finger swipe to form the trajectory 401a. It can be understood that the user's three-finger swipe operation can be decomposed into touch operations at several moments (as an example of the second touch operation), and each touch operation at each moment has corresponding frame touch images (as an example of the second contact area).

[0159] Specifically, Figure 4e According to an embodiment of the present application, a display schematic diagram of a touch animation is shown.

[0160] Here, after generating a number of frame touch images corresponding to the trajectory 401a based on the above steps 301 to 305, the touch recording application may display each frame touch image in real time at the corresponding position on the touch screen to generate a touch animation 405 in the form of a dynamic image (as an example of the displayed first contact area). It can be understood that the touch recording application may also set the display duration of each frame touch image to create a dynamic effect of the touch animation 405 gradually disappearing. It is also possible to continuously display each frame touch image to create a dynamic effect of continuously displaying the touch animation 405. Here, the present application does not make a restrictive description on the specific display effect of the touch animation 405.

[0161] It can be understood that the aforementioned display duration of each frame of the touch image determines the display effect of the touch animated image 405. For example, when the display duration is less than the first time threshold, a display effect of real-time displaying the current touch image can be created. The first time threshold can be, for example, 0.5 seconds, 0.2 seconds, etc., and no restrictive description is made for the first time threshold here.

[0162] When the display duration is between the second time threshold and the third time threshold, a dynamic effect of the touch animated image 405 gradually disappearing can be created. The interval from the second time threshold to the third time threshold can be, for example, 0.5 seconds to 2 seconds, 0.2 seconds to 1 second, etc., and no restrictive description is made for the second time threshold and the third time threshold here.

[0163] When the display duration is greater than the fourth time threshold, a dynamic effect of continuously displaying the touch animated image 405 can be created. The fourth time threshold can be, for example, 10 minutes, 1 hour, etc., and no restrictive description is made for the fourth time threshold here.

[0164] It can be understood that the touch images at each moment that make up the touch animated image 405 can be the three-finger fingerprint images of the user at that moment. Figure 4e In the touch animated image 405, only the touch images 405a, touch image 405b, and touch image 405c corresponding to moment A, moment B, and moment C are taken as examples, and the touch images at all moments that make up the touch animated image 405 are not shown.

[0165] Furthermore, the tablet computer 10 can save the touch animated image 405 as a user animated image in the picture gallery for the user to set the user animated image as a dynamic wallpaper, etc.

[0166] Specifically, Figure 4f According to the embodiments of the present application, a display schematic diagram of a touch image is shown.

[0167] Here, after generating a plurality of frames of touch images corresponding to the trajectory 401a based on the above steps 301 to 305, the touch recording application can superimpose the aforementioned plurality of frames of touch images to generate a touch image 406 in the form of a static image corresponding to the trajectory 401a (as an example of the first contact area for display) after the user completes the touch operation.

[0168] Similarly, the tablet computer 10 can save the touch image 406 as a user picture in the picture gallery for the user to set the user picture as a wallpaper, etc.

[0169] In other usage scenarios, based on the aforementioned user pictures or user GIFs, the tablet computer 10 can provide personalized touch interaction functions. For example, it can also enable the user to set the user picture or user GIF corresponding to the touch operation as the desktop wallpaper, lock screen wallpaper, application background image, emoji, or share the user picture or user GIF to the terminal devices of other users. Here, no restrictive description is made on the specific application methods of the user pictures and user GIFs.

[0170] Based on the implementation process described in the above steps 301 to 306, the interaction method provided in the embodiment of the present application can generate a user picture corresponding to the touch operation. Furthermore, based on the user picture, the personalized touch interaction functions as described above can be provided for the user.

[0171] Next, in combination with another Embodiment 2, the specific implementation process of the tablet computer 10 generating real-time touch traces based on the interaction method provided in the embodiment of the present application will be continued to be introduced.

[0172] Embodiment 2

[0173] This embodiment will specifically illustrate the specific process of realizing an interaction effect of generating real-time touch traces based on the user's touch operation in combination with the accompanying drawings.

[0174] Figure 5 According to the embodiment of the present application, a schematic flowchart of the terminal device generating real-time touch traces based on the user's touch operation is shown. It can be understood that Figure 5 The execution subject of each step of the shown process can be the aforementioned tablet computer 10. For the sake of convenience of description, the execution subject of each step will not be repeated hereinafter when introducing each step.

[0175] Specifically, the interaction method provided in the embodiment of the present application that can generate real-time touch traces may include the following steps:

[0176] 501: Display the desktop or lock screen interface.

[0177] Exemplarily, Figure 6a According to the embodiment of the present application, a schematic diagram of a displayed desktop is shown.

[0178] See Figure 6a , the tablet computer 10 can display the desktop 600 based on the window manager. Here, the style of the desktop 600 is only an example, and the present application does not make restrictive descriptions on the graphical user interface design of the desktop 600 of the tablet computer 10.

[0179] 502: Collect the original capacitance data generated corresponding to the user's touch operation.

[0180] Exemplarily, for the specific content of the original capacitance data collected by the tablet computer 10 corresponding to the user's touch operation, reference may be made to the specific description in the foregoing step 303, which will not be elaborated herein.

[0181] 503: Generate capacitance difference data based on the original capacitance data.

[0182] Exemplarily, for the specific content of the tablet computer 10 generating capacitance difference data based on the original capacitance data, reference may be made to the specific description in the foregoing step 304, which will not be elaborated herein.

[0183] 504: Generate a touch image based on the capacitance difference data.

[0184] Exemplarily, the desktop service or the lock screen service of the tablet computer 10 may generate a touch image based on the capacitance difference data. Herein, the desktop service and the lock screen service may be system services for displaying the desktop and the lock screen. For example, they may be built-in programs in the read only memory (ROM) of the tablet computer 10, or non-ROM built-in application programs. Herein, for the sake of narrative coherence, the process of the desktop service or the lock screen service of the tablet computer 10 obtaining the capacitance difference data will be specifically described in the following description of Figure 10a Herein, the specific content of the desktop service or the lock screen service generating a touch image based on the capacitance difference data is substantially the same as the process of the touch recording application generating a touch image in the foregoing step 305. The specific operations may refer to the relevant description in the foregoing step 305, which will not be elaborated herein.

[0185] 505: Map the touch image on the desktop or the lock screen interface.

[0186] Exemplarily, the desktop service or the lock screen service of the tablet computer 10 may adjust the resolution of the touch image according to the interpolation algorithm. For example, the resolution of the touch image may be adjusted to be the same as the resolution of the display screen of the tablet computer 10. Further, the tablet computer 10 may display the touch image with the adjusted resolution on the desktop or the lock screen interface (as an example of the second interface), that is, the display position of the touch image may be the same as the user's touch position.

[0187] Figure 6b According to the embodiment of the present application, a schematic diagram of the effect of displaying a touch image in a split screen mode is shown.

[0188] See Figure 6b , it can be understood that when the tablet computer 10 displays the desktop 600 and the short message interface 601 in a split screen manner, the window manager of the tablet computer 10 may crop the touch image with the adjusted resolution based on the display window size of the desktop 600. Further, only the touch image 600a is displayed in the display window of the desktop 600.

[0189] Figure 6c Another schematic diagram showing the effect of displaying a touch image in a split-screen mode is illustrated according to an embodiment of the present application.

[0190] Referring to Figure 6c , it can be understood that the tablet computer 10 can also completely display the touch image 600b with the adjusted resolution on the display screen of the tablet computer 10.

[0191] Based on the above, it can be understood that when the user's finger or palm slides or stays on the touch screen, the tablet computer 10 can update the touch image displayed on the desktop or the lock screen interface in real time according to the user's touch operations at different times, so as to visualize the user's various touch operations and increase the fun of touch interaction. It can be understood that the touch image that is displayed on the desktop or the lock screen interface in real time based on the user's touch operation is the touch trace corresponding to the touch operation at the current moment.

[0192] For example, Figure 6d A schematic diagram showing the effect of displaying a touch trace is illustrated.

[0193] Referring to Figure 6d , the user's touch operation can be that the palm slides from the lower left corner to the upper right corner of the tablet computer 10. Figure 6d Taking the start time and the end time of the touch operation as an example, the touch images corresponding to the start time and the end time are illustrated. Specifically, corresponding to the start time of the touch operation, the desktop service of the tablet computer 10 can generate the touch image corresponding to this time and map it in the desktop 600 to form the touch image 600c. Corresponding to the end time of the touch operation, the desktop service of the tablet computer 10 can generate the touch image corresponding to this time and map it in the desktop 600 to form the touch image 600d. It can be understood that the desktop service of the tablet computer 10 refreshes the touch image displayed in the desktop 600 at a certain display frequency, and thus can display the touch trace that moves accordingly as the user's palm moves.

[0194] It can be understood that the attributes such as the line color, linearity, filling color, and transparency of the touch image displayed in the desktop 600 can be set by the desktop service or can be customized and adjusted by the user. Here, no specific display effect is limited.

[0195] In addition, based on the content that is substantially the same as the foregoing steps 501 to 505, each application program or other system services can also display the user's touch trace on other interfaces other than the desktop and the lock screen interface. Here, no specific interface for displaying the user trace is limited.

[0196] Based on the implementation process described in the above steps 501 to 505, the interaction method provided in the embodiments of the present application can visualize various touch operations of the user, increasing the fun of touch interaction.

[0197] Next, in combination with another Embodiment 3, the specific implementation process of the tablet computer 10 sharing the user's touch operation in real time according to the interaction method provided in the embodiments of the present application will be continued.

[0198] Embodiment 3

[0199] This embodiment will specifically illustrate the specific process of implementing the interaction effect of sharing the touch image corresponding to the user's touch operation in real time based on remote communication in combination with the accompanying drawings.

[0200] Figure 7 According to the embodiments of the present application, a schematic diagram of the interaction of the terminal device sharing the touch image corresponding to the user's touch operation in real time based on remote communication is shown. It can be understood that Figure 7 The shown process can be implemented by the interaction of two terminal devices that have established remote communication. Here, taking the tablet computer 10 and the tablet computer 20 as examples, the process of the tablet computer 10 sharing the touch image with the tablet computer 20 based on remote communication will be specifically described.

[0201] Specifically, the interaction that can share the touch image in real time based on remote communication provided in the embodiments of the present application may include the following steps:

[0202] 701: The tablet computer 10 establishes a remote communication connection with the tablet computer 20.

[0203] In one example, the tablet computer 10 and the tablet computer 20 can run an instant messaging application (as an example of a second application) and establish a remote communication connection based on the instant messaging application. Here, the instant messaging application may include, but is not limited to, Changlian TM WeChat TM QQ TM DingTalk TM and other application programs that realize online chatting through instant messaging technology, as well as application programs with instant messaging functions such as conference applications, live broadcast applications, teaching applications, game applications, and video applications.

[0204] Exemplarily, taking the instant messaging application as the WeChat TM application as an example, both the tablet computer 10 and the tablet computer 20 are installed with the WeChat TM application, and the user can establish a remote communication connection through the WeChat TM application running on the tablet computer 10 and the tablet computer 20. For example, the way for the tablet computer 10 and the tablet computer 20 to establish a remote communication connection can be to start a video call. For example, the WeChat TMThe application initiates a video call request to the tablet computer 20, and the WeChat running on the tablet computer 20 TM After the application responds to the request, the tablet computer 10 and the tablet computer 20 establish a remote communication connection, and can transmit video call data based on the application server, etc. The video call data can be the captured images and / or voice data collected by the tablet computer 10 and / or the tablet computer 20. Furthermore, the tablet computer 10 and the tablet computer 20 can implement the following steps 705 to 706 based on the remote communication connection.

[0205] For another example, the tablet computer 10 and the tablet computer 20 may establish a remote communication connection by starting a voice call. TM The application initiates a voice call request to the tablet computer 20, and the WeChat running on the tablet computer 20 TM After the application responds to the request, the tablet computer 10 establishes a remote communication connection with the tablet computer 20, and voice call data can be transmitted based on the application server and the like.

[0206] It is understood that the above application server can be the WeChat running on the tablet computer 10 or the tablet computer 20. TM In other embodiments, remote communication and transmission of video call data between the tablet computer 10 and the tablet computer 20 may also be achieved through other servers, such as general servers, database servers or file servers, etc., which are not limited here.

[0207] In another example, the way in which the tablet computer 10 and the tablet computer 20 establish a remote communication connection, for example, can also be to establish a new air interface voice bearer (voice over newradio, VoNR) call or a long-term evolution voice bearer (voice over long term evolution, VoLTE) call through a telephone application (as an example of a second application). Here, the aforementioned telephone application can be a system application or a third-party application for users to initiate call service operations. Among them, the remote communication connection established between the tablet computer 10 and the tablet computer 20 based on the VoNR call or the VoLTE call can be a voice call or a video call, which is not limited here.

[0208] For example, Figure 8a According to an embodiment of the present application, a schematic diagram of an interface for a video call between a tablet computer 10 and a tablet computer 20 is shown.

[0209] See also Figure 8a , based on WeChat on Tablet 10 and Tablet 20 TMTaking the application for video calls as an example, the tablet computer 10 can display the video images captured by the tablet computer 10 and the tablet computer 20 in the video interface 800 (as an example of the third interface). The tablet computer 20 can display the video images captured by the tablet computer 10 and the tablet computer 20 in the video interface 801. It can be understood that the tablet computer 10 and / or the tablet computer 20 can also only display the video images captured by the tablet computer 10 or the tablet computer 20. Here, no restrictive description is made on the display content of the video interface.

[0210] It can be understood that the tablet computer 10 and the tablet computer 20 can also conduct video calls based on VoNR calls or VoLTE calls. Here, no restrictive description is made on the specific manner of conducting video calls between the tablet computer 10 and the tablet computer 20.

[0211] For example, Figure 8b According to an embodiment of the present application, a schematic diagram of an interface for a voice call between a tablet computer 10 and a tablet computer 20 is shown.

[0212] See Figure 8b , taking the voice call between the tablet computer 10 and the tablet computer 20 based on the WeChat TM application as an example, the WeChat TM application of the tablet computer 10 can display a voice interface 802 (as an example of the third interface). The WeChat TM application of the tablet computer 20 can display a voice interface 803. It can be understood that the voice interface 802 and the voice interface 803 are only examples, and the present application does not make a restrictive description on the specific content of the voice interface.

[0213] It can be understood that the tablet computer 10 and the tablet computer 20 can also conduct voice calls based on VoNR calls or VoLTE calls. Here, no restrictive description is made on the specific manner of conducting voice calls between the tablet computer 10 and the tablet computer 20.

[0214] 702: The tablet computer 10 collects the original capacitance data generated corresponding to the user's touch operation.

[0215] Exemplarily, the user of the tablet computer 10 can perform a touch operation on the touch screen during the process of remote communication.

[0216] For example, Figure 8c shows a schematic diagram of a touch operation performed by the user of the tablet computer 10.

[0217] See Figure 8c , the user of the tablet computer 10 can single-finger long-press the display area 800a of the video interface 800.

[0218] For another example, Figure 8dShows a schematic diagram of another touch operation implemented by a user of the tablet computer 10.

[0219] See Figure 8d , the user of the tablet computer 10 can draw a heart-shaped touch trace 802a in the voice interface 802.

[0220] Furthermore, the tablet computer 10 can collect the original capacitance data corresponding to the user's touch operation. Specifically, for the specific content of the original capacitance data collected by the tablet computer 10 corresponding to the user's touch operation, reference can be made to the specific description in the foregoing step 303, and details are not described herein again.

[0221] 703: The tablet computer 10 generates capacitance difference data based on the original capacitance data.

[0222] Exemplarily, for the specific content of the capacitance difference data generated by the tablet computer 10 based on the original capacitance data, reference can be made to the specific description in the foregoing step 304, and details are not described herein again.

[0223] 704: The tablet computer 10 generates a touch image based on the capacitance difference data.

[0224] Exemplarily, the WeChat TM application running on the tablet computer 10 can obtain the capacitance difference data generated corresponding to the user's touch operation, and generate a touch image based on the capacitance difference data. Here, for the sake of narrative coherence, the process of the WeChat Figure 10a application running on the tablet computer 10 obtaining the capacitance difference data will be specifically described in the following description of TM .

[0225] For example, Figure 8e According to an embodiment of the present application, a schematic diagram of a touch image corresponding to a touch operation is shown.

[0226] See Figure 8e , corresponding to the touch operation of the user's single-finger long pressing on the display area 800a of the video interface 800, the WeChat TM application running on the tablet computer 10 can generate a touch image 800b representing a single-finger fingerprint and display the touch image 800b in the video interface 800.

[0227] For example, Figure 8f According to an embodiment of the present application, a schematic diagram of a touch image corresponding to a touch operation is shown.

[0228] See Figure 8f , corresponding to the touch operation of the user drawing a heart-shaped touch trace 802a in the voice interface 802, the WeChat TM application running on the tablet computer 10 can generate multiple frames of touch images corresponding to different moments, and each frame of touch image can represent the single-finger fingerprint of the user at a certain moment.Figure 8f Taking moments A, B, and C during the process of drawing the touch trace 802a as examples, the touch image 802b corresponding to moment A, the touch image 802c corresponding to moment B, and the touch image 802d corresponding to moment C are shown.

[0229] Here, the specific content of generating a touch image by the desktop service or the lock screen service based on the capacitance difference data is essentially the same as the process of generating a touch image by the touch recording application in the foregoing step 305. The specific operations can be referred to the relevant descriptions in the foregoing step 305 and will not be elaborated here.

[0230] 705: The tablet computer 10 sends the touch image to the tablet computer 20 based on the remote communication connection.

[0231] Exemplarily, the WeChat TM application running on the tablet computer 10 can, after generating the touch image corresponding to the user's touch operation, send the touch image to the WeChat TM application running on the tablet computer 20 in real time based on the application server.

[0232] It can be understood that the tablet computer 10 sending the touch image to the tablet computer 20 in real time can also be through other servers, such as a general server, a database server, or a file server, etc., which is not limited here.

[0233] It can be understood that in one example manner, after establishing a remote communication connection with the tablet computer 20, the tablet computer 10 can automatically execute the interaction operation of sending the touch image to the tablet computer 20. Correspondingly, after establishing a remote communication connection with the tablet computer 10, the tablet computer 20 can automatically receive the touch image sent by the tablet computer 10 to display the touch image in the remote communication interface of the tablet computer 20 based on the following step 706.

[0234] In another example manner, the remote communication interface (as an example of the third interface) displayed on the tablet computer 10 may include an interaction control (as an example of the second control). The user can click the interaction control (as an example of the second user operation, not shown in the figure), so that the tablet computer 10 executes the interaction operation of sending the touch image to the tablet computer 20. Correspondingly, the remote communication interface of the tablet computer 20 may also include an interaction control. Furthermore, after establishing a remote communication connection with the tablet computer 10, based on the click operation of the user of the tablet computer 20 on the interaction control, the tablet computer 20 can receive the touch image sent by the tablet computer 10 to display the touch image in the remote communication interface (as an example of the fourth interface) of the tablet computer 20 based on the following step 706.

[0235] 706: The tablet computer 20 displays the touch image in the remote communication interface.

[0236] Exemplarily, the WeChat application running on the tablet computer 20 TM After receiving the real-time touch image, the application can display the touch image in the remote communication interface. The remote communication interface can be the aforementioned video interface or the aforementioned voice interface.

[0237] Specifically, the WeChat application running on the tablet computer 20 TM After receiving the real-time touch image, the application can first adjust the size of the touch image. For example, the display screen sizes of the tablet computer 10 and the tablet computer 20 may be different. Therefore, the WeChat TM application running on the tablet computer 20 can adjust the size of the touch image so that the screen pixel density (pixels per inch, PPI) of the touch image displayed on the display screen of the tablet computer 20 is the same as that of the touch image displayed on the display screen of the tablet computer 10. That is, the size of the touch image displayed in the tablet computer 20 is the same as that of the touch image displayed in the tablet computer 10.

[0238] It can be understood that the touch image usually can represent biometric features such as fingerprints and palm prints of the user of the tablet computer 10. Therefore, when the display screen sizes of the terminal devices vary greatly, for example, when the terminal devices for remote communication are a mobile phone and a laptop computer, if the touch image is directly mapped according to the screen size, it may cause the touch images such as fingerprints and palm prints displayed on the laptop computer to be too large; or, the touch images such as fingerprints and palm prints displayed on the mobile phone are too small and lack a sense of reality.

[0239] Therefore, the above operation of unifying the PPI can ensure that the sizes of the touch images such as fingerprints and palm prints displayed on different terminal devices are not affected by the difference in the display screen size. Furthermore, problems such as lack of a sense of reality caused by the inconsistent display screen sizes of the terminal devices can be avoided.

[0240] Furthermore, when the WeChat application running on the tablet computer 20 TM has adjusted the size of the touch image, it can display the touch image at the corresponding position in the remote communication interface.

[0241] For example Figure 9a According to the embodiments of the present application, a schematic diagram showing the effect of real-time sharing of touch traces in a video call scenario is shown.

[0242] See Figure 9a , when the user of the tablet computer 10 single-finger long-presses the display area 900a of the video interface 900, the WeChat application running on the tablet computer 10 TM can send the touch image 900b corresponding to the touch operation to the WeChat application running on the tablet computer 20 TM application. Furthermore, the WeChat application running on the tablet computer 20TM The application can display the touch image 901a at the corresponding position of the video interface 901. Here, the touch image 901a can be the touch image 900b after being unified by PPI.

[0243] In addition, the WeChat TM application running on the tablet computer 20 can also display the touch image of the user of the tablet computer 20 based on the foregoing steps 701 to 706. And, the touch image of the user of the tablet computer 20 can be sent to the WeChat TM application running on the tablet computer 10 in real time, so that the touch image of the user of the tablet computer 20 is displayed in real time in the video interface 900 of the tablet computer 10.

[0244] For example, Figure 9b According to the embodiments of the present application, a schematic diagram of the effect of real-time sharing of touch traces in another video call scenario is shown.

[0245] See Figure 9b , the WeChat TM application running on the tablet computer 10 shares the touch image 900b of its user to the video interface 901 of the tablet computer 20, and is displayed as the touch image 901a. The WeChat TM application running on the tablet computer 20 shares the touch image 901b of its user to the video interface 900 of the tablet computer 10, and is displayed as the touch image 900c.

[0246] When the touch image of the user of the tablet computer 10 coincides or partially coincides with the touch image of the user of the tablet computer 20, for example, in the video interface 900 of the tablet computer 10, the touch image 900b coincides or partially coincides with the touch image 900c, and in the video interface 901 of the tablet computer 20, the touch image 901a coincides or partially coincides with the touch image 901b, the WeChat TM applications running on the tablet computer 10 and the tablet computer 20 can provide output effects for the user and / or can display dynamic effects in the video interface 900 and the video interface 901. Here, the foregoing output effects include the effects realized by the output components of the terminal device, including one or more combinations of sound, vibration, light, backlight flashing, etc. The foregoing dynamic effects can be one or more combinations of the following ways: one or more combinations of changing features such as the color, transparency, and linearity of the touch image; displaying a pattern effect around the touch image, and the pattern can be various shape effects such as a heart shape, a star shape, a circular shape, a scattered halo effect, etc.; displaying an animation in the video interface, for example, presenting visual effects such as falling hearts and stars in the video interface, displaying a preset animation in the video interface, etc.

[0247] For example, when the touch images of the users of tablet computer 10 overlap or partially overlap with those of the users of tablet computer 20, tablet computer 20 can provide an output effect for the users of tablet computer 20 and / or display a dynamic effect. Meanwhile, tablet computer 20 can send an instruction to tablet computer 10 based on a remote communication connection, so that after receiving the instruction, tablet computer 10 can provide an output effect for the users of tablet computer 10 and / or display a dynamic effect.

[0248] The foregoing Figure 9a and Figure 9b In the aforementioned effect diagrams, the touch images are all displayed in the video picture. It can be understood that the video interface can also provide two display areas. For example, the video picture can be displayed based on one display area, and the touch image can be displayed based on another display area. Or, the video picture can be displayed based on one window, and the touch image can be displayed based on another window.

[0249] For example, Figure 9c According to an embodiment of the present application, an effect diagram of displaying touch traces is shown.

[0250] See Figure 9c , the video interface 900 of tablet computer 10 may include a display area 900d for displaying a captured picture and a display area 900e for displaying a touch image. Correspondingly, the video interface 901 of tablet computer 20 may include a display area 901c for displaying a captured picture and a display area 901d for displaying a touch image.

[0251] When a user of tablet computer 10 performs a touch operation in the display area 900e, the WeChat TM application running on tablet computer 10 can display its corresponding touch image 900b in this display area 900e and share it with the WeChat TM application running on tablet computer 20, so as to display a touch image 901a corresponding to the touch image 900b in the display area 901d.

[0252] It can be understood that tablet computer 10 and / or tablet computer 20 can display the touch image in the video picture, or can also display the touch image in a separate display area or window. The display mode of the touch image can be determined by the WeChat TM application or can also be selected by the user independently.

[0253] Figure 9d According to an embodiment of the present application, a schematic diagram of the effect of real-time sharing of touch traces in a voice call scenario is shown.

[0254] See Figure 9d , the user of tablet computer 10 can use a single finger to slide in the voice interface 902 to draw a touch trace in the shape of a heart. Furthermore, the WeChat running on tablet computer 10TM The application can display the touch images at each moment in real time on the voice interface 902, and send the touch images at each moment to WeChat running on the tablet computer 20 TM The application sends the touch images at each moment in real time.

[0255] Herein, Figure 9d Only the touch images 902a, touch image 902b, and touch image 902c corresponding to moment A, moment B, and moment C are shown in the voice interface 902 of the tablet computer 10. The touch images 903a, touch image 903b, and touch image 903c displayed by the tablet computer 20 at moment A, moment B, and moment C are shown in the voice interface 903 of the tablet computer 20. It can be understood that the user of the tablet computer 20 can see the dynamic heart-shaped trace composed of the touch images at each moment in the voice interface 903. In this way, multiple users in remote communication can share the touch trace, providing a richer interaction experience.

[0256] In addition, the tablet computer 10 and / or the tablet computer 20 can continuously display the touch images corresponding to each moment of a touch operation.

[0257] For example, Figure 9e According to an embodiment of the present application, a schematic diagram showing another effect of real-time sharing of touch traces in a voice call scenario is shown.

[0258] See Figure 9e , WeChat running on the tablet computer 10 TM The application can continuously display the touch images corresponding to each moment during the process of the user using a single finger to slide to draw a heart shape in the voice interface 902. Furthermore, after the user finishes drawing, WeChat TM The application running on the tablet computer 10 can display the complete heart-shaped image 902d in the voice interface 902. Correspondingly, WeChat TM The application running on the tablet computer 20 can also continuously display the received touch images in the voice interface 903. Furthermore, the complete heart-shaped image 903d can be displayed in the voice interface 903. Based on this method, users can draw and share patterns and / or texts in real time during remote communication, enhancing the fun of touch interaction.

[0259] It can be understood that the voice interface can also display touch images based on different display areas or windows. For specific content, reference can be made to the relevant description of Figure 9c above, which will not be elaborated here.

[0260] It can be understood that WeChat TMThe application can use the above method of sharing touch traces in real time as a touch interaction sharing function that users can choose to enable or not. Furthermore, when the user needs to share touch traces in real time, the touch interaction sharing function can be enabled; conversely, when there is no need to share touch traces in real time, the touch interaction sharing function can be disabled.

[0261] Next, the process of generating a touch image by the foregoing tablet computer 10 based on the interaction method provided in this application will be specifically described with reference to the accompanying drawings. It can be understood that the following process of generating a touch image is also applicable to the foregoing tablet computer 20 and other terminal devices.

[0262] Figure 10a The schematic diagram of the system architecture related to implementing the interaction method provided in this application in the tablet computer 10 is shown according to an embodiment of this application.

[0263] See Figure 10a , the modules related to implementing the above interaction method in the tablet computer 10 are distributed in the hardware layer 510, the kernel layer 520, the system library 530, the application framework layer 540, and the application layer 550.

[0264] Among them, the hardware layer 510 may include a touch chip 511 and a touch screen 512. Here, the touch chip 511 is used to drive the touch screen 512, so that the touch screen 512 collects the foregoing raw capacitance data according to a certain sampling frequency based on a plurality of touch sensors. The touch screen 512 sends the collected raw capacitance data to the touch chip 511, so that the touch chip 511 can capture the touch operation of the user on the touch screen 512 in real time. The touch chip 511 is also used to send a notification to the touch driver 521 after receiving the raw capacitance data, so that the touch driver 521 can call the raw capacitance data.

[0265] Here, the raw capacitance data may be a capacitance matrix composed of several raw capacitance values. One touch sensor may correspond to one or more touch points, and each touch point may correspond to a raw capacitance value. The raw capacitance value may represent the capacitance value of its corresponding touch point at each acquisition moment.

[0266] Among them, the kernel layer 520 may include a touch driver 521. The touch driver 521 is used to call the raw capacitance data after receiving the notification sent by the touch chip 511 and forward the raw capacitance data to the touch algorithm service 531.

[0267] Among them, the system library 530 may include a touch algorithm service 531. The touch algorithm service 531 may include reference capacitance data when there is no touch operation on the touch screen. It can be understood that the reference capacitance data may be a capacitance matrix composed of several reference capacitance values. One touch sensor may correspond to one or more touch points, and each touch point may correspond to a reference capacitance value. The reference capacitance value may represent the capacitance value of its corresponding touch point when not being touched. Furthermore, the touch algorithm service 531 may use the difference between the original capacitance data and the reference capacitance data as the aforementioned capacitance difference data.

[0268] The application framework layer 540 may include a window manager 541 and an input system 542. The window manager 541 may display the touch images generated by each application in the system process and / or the application layer 550 on the system interface and / or the application interface. Here, the system interface may include a desktop, a lock screen interface, etc. The application interface may include the interfaces of each application program.

[0269] The application layer 550 may include system applications such as a touch recording application 551, a gallery application 552, an instant messaging application 553, a phone application 554, and / or third-party applications. Each application may call the capacitance difference data through an interface to generate a touch image based on the capacitance difference data, and further provide the touch interaction solutions of the aforementioned Embodiments 1 to 3 for the user. The aforementioned interface may be, for example, a hardware interface definition language (HIDL) interface under the Android TM operating system architecture. This application does not make restrictive descriptions on the interface for obtaining the capacitance difference data.

[0270] For example, Figure 10b According to an embodiment of the present application, a schematic flowchart of generating a user picture based on capacitance difference data is shown.

[0271] See Figure 10b , the tablet computer 10 may run the touch recording application 551, and the touch algorithm service 531 generates capacitance difference data according to the originally collected capacitance data in real time. Furthermore, the touch recording application 551 may read the capacitance difference data through the HIDL interface and generate a corresponding touch image according to the capacitance difference data. The touch recording application 551 may save the touch image as a user picture. For example, the user may store the touch image as a user picture in the gallery application 552. Therefore, the user may set the user picture as an avatar, a desktop wallpaper, a lock screen wallpaper, etc. according to actual needs.

[0272] For another example, Figure 10c According to an embodiment of the present application, a schematic flowchart of generating real-time touch traces based on capacitance difference data is shown.

[0273] See Figure 10c , the tablet computer 10 can display the desktop or the lock screen interface based on the aforementioned desktop service or the aforementioned lock screen service. The touch algorithm service 531 generates capacitance difference data according to the originally collected capacitance data in real time. The desktop service or the lock screen service can read the capacitance difference data through the HIDL interface, and generate corresponding touch images according to the capacitance difference data. Furthermore, the window manager 541 can display the touch images on the desktop or the lock screen interface of the tablet computer 10 to create a touch interaction effect of real-time displaying touch traces.

[0274] For another example, Figure 10d The flowchart showing the real-time sharing of user touch operations based on capacitance difference data and remote communication according to the embodiments of the present application is illustrated.

[0275] See Figure 10d , the tablet computer 10 can perform remote communication with the tablet computer 20 based on the instant messaging application 553 or the phone application 554. The touch algorithm service 531 of the tablet computer 10 generates capacitance difference data according to the originally collected capacitance data in real time. The instant messaging application 553 or the phone application 554 of the tablet computer 10 reads the capacitance difference data through the HIDL interface, and generates corresponding touch images according to the capacitance difference data. The instant messaging application 553 or the phone application 554 of the tablet computer 10 sends the touch images to the instant messaging application 553 or the phone application 554 running on the tablet computer 20 through the application server to share the touch traces with the tablet computer 20 in real time. Taking the remote communication between the tablet computer 10 and the tablet computer 20 based on the TM WeChat TM application as an example, the WeChat TM applications running on both the tablet computer 10 and the tablet computer 20 can enable the aforementioned touch interaction sharing function to share the touch traces in real time. For example, the WeChat TM application running on the tablet computer 10 can read the capacitance difference data through the HIDL interface, and generate corresponding touch images according to the capacitance difference data. Furthermore, the WeChat TM application running on the tablet computer 10 can send the touch images to the WeChat

[0276] It can be understood that while providing the above touch interaction for the user through the system architecture of the tablet computer 10, the tablet computer 10 can also determine the contact coordinates of the user's touch operation, and execute corresponding functions based on the contact coordinates.

[0277] For example, after the touch algorithm service 531 generates capacitance difference data, it can continue to generate contact coordinates based on the capacitance difference data and send the contact coordinates to the touch driver 521. Further, the touch driver 521 can distribute the contact coordinates to each application in the system process or the application layer 550 through the input system 542 in the application framework layer 540, so that the system process or each application provides touch interaction for the user according to the contact coordinates. The touch interaction can be to determine the button corresponding to the contact coordinates in the system interface or the application interface and provide the corresponding function for the user based on the button.

[0278] Figure 10e The system architecture diagram related to implementing another interaction method is shown according to an embodiment of the present application. This interaction method can only implement touch interaction based on contact coordinates.

[0279] Herein, for the content of implementing touch interaction based on contact coordinates in this interaction method, reference can be made to Figure 10d the relevant content therein, which will not be elaborated herein.

[0280] In comparison Figure 10e with the foregoing Figure 10d it can be seen that the interaction method provided by the embodiment of the present application utilizes the capacitance difference data required for generating contact coordinates, generates a touch image based on the capacitance difference data, and provides multiple touch interaction solutions for the user according to the touch image. For example, the touch image is displayed on the display screen of the terminal device, the touch image is set as an avatar or a wallpaper, and the touch image is shared in real time with other terminal devices based on remote communication. In this way, a touch interaction method that can cover all touch operations is provided for the user, enriching the interest of touch interaction.

[0281] Figure 11 The structural schematic diagram of the terminal device 100 according to the embodiment is shown according to an embodiment of the present application. It can be understood that the terminal device 100 may include the foregoing tablet computer 10 or tablet computer 20.

[0282] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and an ambient light sensor 180L.

[0283] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figures, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0284] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0285] The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0286] A memory can also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the above-mentioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0287] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, an HIDL interface, etc.

[0288] The application program in the terminal device 100 can obtain the capacitance difference data generated by the touch algorithm service based on the HIDL interface.

[0289] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only for illustrative purposes and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0290] The wireless communication function of the terminal device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc. This wireless communication function can be, for example, a video call, a voice call, etc. based on an instant messaging application or a telephone application.

[0291] Antenna 1 and Antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, Antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0292] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through Antenna 1, filter, amplify, and process the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through Antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.

[0293] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor can be an independent device. In some other embodiments, the modulation and demodulation processor can be independent of the processor 110 and be disposed in the same device as the mobile communication module 150 or other functional modules.

[0294] The wireless communication module 160 may provide solutions for wireless communications applied to the terminal device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and so on. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0295] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technologies.

[0296] The terminal device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0297] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0298] The terminal device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0299] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0300] The camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0301] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121, and / or the instructions stored in the memory provided in the processor.

[0302] The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates having conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations with the same touch position but different touch operation intensities can correspond to different operation instructions.

[0303] The touch sensor 180K, also called a "touch control device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect a touch operation acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the terminal device 100, at a different position from that of the display screen 194. In the interaction method provided in this application, the touch sensor 180K can transmit the original capacitance data to the touch control chip.

[0304] The embodiments of this application also provide a computer program product for implementing the interaction methods provided in the above embodiments.

[0305] Embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer program modules or module code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device.

[0306] The computer program modules or module code can be applied to input instructions to perform the various functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0307] The module code can be implemented in a high-level modular language or an object-oriented programming language to communicate with the processing system. When needed, the module code can also be implemented in assembly language or machine language. In fact, the mechanisms described in this application are not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0308] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more transient or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Accordingly, machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to, floppy disks, optical disks, optical discs, magneto-optical discs, read only memory (ROM), random access memory (RAM), erasable programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) in electrical, optical, acoustic, or other forms via the Internet. Thus, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0309] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed in embodiments of the present application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0310] The disclosure of embodiments of the present application also relates to apparatus for performing the operations in the text. The apparatus may be specially constructed for the required purposes or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application specific integrated circuit (ASIC) or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. In addition, the computers referred to in the specification may include a single processor or may be architectures involving multiple processors for increased computing power.

[0311] In addition, the language used in this specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the disclosed subject matter. Accordingly, the disclosure of embodiments of this application is intended to be illustrative rather than limiting of the scope of the concepts discussed herein.

Claims

1. An interaction method, applied to a first terminal device, characterized in that, The method includes: Detecting a first touch operation of a user on the screen of the first terminal device, wherein a contact area of the first touch operation of the user with the touch screen is a first contact area; Displaying the first contact area on the screen; Detecting an interaction instruction corresponding to the displayed first contact area and performing an operation corresponding to the interaction instruction.

2. The method according to claim 1, characterized in that, The first touch operation includes one or more second touch operations, and a contact area of the second touch operation with the touch screen is a second contact area; wherein, The first contact area corresponds to one or more of the second contact areas, and, The displaying the first contact area on the screen includes: Sequentially displaying the one or more second contact areas on the screen, wherein the second contact area has a corresponding set display duration.

3. The method according to claim 2, wherein The screen displays a first interface of a first application, wherein the detecting a first touch operation of a user on the screen of the first terminal device includes: Detecting a first touch operation of the user on the first interface; and, The displaying the first contact area on the screen includes: Displaying the first contact area in the first interface.

4. The method according to claim 3, wherein The first interface includes a first control, and, The detecting an interaction instruction corresponding to the displayed first contact area and performing an operation corresponding to the interaction instruction includes: Detecting a first user operation of the user on the first control in the first interface and saving the displayed first contact area as a first image.

5. The method according to claim 3, wherein The first interface includes a first control, and, The detecting an interaction instruction corresponding to the displayed first contact area and performing an operation corresponding to the interaction instruction includes: Detecting a first user operation of the user on the first control in the first interface and saving the process of sequentially displaying a plurality of the second contact areas on the first interface as a first animated image.

6. The method according to claim 4, characterized in that After saving the displayed first contact area as a first image, it further includes: Setting the first image as at least any one of a desktop wallpaper, a lock screen wallpaper, an application program background image, an avatar, an emoji of the first terminal device, and sending the first image to a second terminal device.

7. The method according to claim 5, characterized in that, After saving the process of sequentially displaying a plurality of the second contact areas on the first interface as a first animated image, it further includes: Setting the first animated image as at least any one of a desktop wallpaper, a lock screen wallpaper, an application program background image, an avatar, an emoji of the first terminal device, and sending the first animated image to a second terminal device.

8. The method according to claim 2, characterized in that The screen displays a second interface, wherein the detecting a first touch operation of a user on the screen of the first terminal device includes: Detecting a first touch operation of the user on the second interface, and, The detecting an interaction instruction corresponding to the displayed first contact area and performing an operation corresponding to the interaction instruction includes: Detecting a first touch operation of the user on the second interface and displaying the first touch area in the second interface.

9. The method according to claim 8, characterized in that The displaying the first touch area in the second interface includes: The multiple second contact areas are sequentially displayed on the second interface.

10. The method according to claim 8, wherein The second interface includes at least any one of the following: The desktop of the first terminal device, the lock screen interface, and the application program interface.

11. The method according to claim 2, wherein The first terminal device establishes a remote communication connection with the third terminal device based on a second application. Among them, the screen displays a third interface of the second application, and The detection of the first touch operation of the user on the screen of the first terminal device includes: Detecting the first touch operation of the user on the third interface; and The display of the first contact area on the screen includes: Displaying the first contact area on the third interface.

12. The method according to claim 11, wherein The detection of the interaction instruction corresponding to the displayed first contact area includes: Detecting the establishment of a remote communication connection between the first terminal device and the third terminal device, and / or detecting a second user operation of the user on a second control in the third interface; and The execution of the operation corresponding to the interaction instruction includes: Sending the displayed first contact area to the third terminal device.

13. The method according to claim 12, characterized in that, The display of the first contact area on the third interface includes: Sequentially displaying the multiple second contact areas on the third interface; and The sending of the displayed first contact area to the third terminal device includes: Sequentially sending the multiple second contact areas sequentially displayed on the third interface to the third terminal device.

14. The method according to claim 13, wherein After sending the displayed first contact area to the third terminal device, the method further includes: Receiving an instruction sent by the third terminal device; Providing an output effect to the user based on the instruction, and / or displaying a dynamic effect on the third interface.

15. The method according to claim 14, wherein The output effect includes at least any one of the following: Sound, vibration, light, backlight flashing.

16. The method according to claim 14, wherein The dynamic effect includes at least any one of the following: Changing one or more combinations of the color, transparency, and line shape of the first contact area displayed in the third interface, displaying a preset pattern effect around the first contact area, and displaying a preset animation on the third interface.

17. The method according to any one of claims 2 to 16, characterized in that, The first terminal device includes multiple touch sensors. Among them, each touch sensor is used to collect the capacitance difference of one or more of the touch points. The capacitance differences collected by the multiple touch sensors constitute capacitance difference data; And, the second contact area is determined in the following manner: Determining a first capacitance matrix in the capacitance difference data based on the second touch operation and a preset threshold. The first capacitance matrix includes several capacitance differences greater than the preset threshold; Converting the first capacitance matrix into the second touch area according to a preset image processing algorithm.

18. The method according to claim 17, wherein The first terminal device includes multiple capacitance difference intervals with corresponding color parameters. Among them, the conversion of the first capacitance matrix into the second touch area according to the preset image processing algorithm further includes: Judging the capacitance difference interval to which each capacitance difference in the first capacitance matrix belongs; Determine the colors of the respective pixel points in the second touch area according to the color parameters corresponding to the capacitance difference intervals to which the respective capacitance differences belong, wherein the respective pixel points in the second touch area correspond one-to-one with the respective capacitance differences in the first capacitance matrix.

19. An interaction method, applied to a third terminal device, characterized in that, The third terminal device establishes a remote communication connection with the first terminal device based on a second application, wherein the screen of the third terminal device displays a fourth interface of the second application, and the method includes: Receiving, based on the second application, a first contact area sent by the first terminal device; Displaying the first contact area in the fourth interface.

20. The method according to claim 19, wherein, The receiving, based on the second application, the first contact area sent by the first terminal device includes: Sequentially receiving, based on the second application, a plurality of second touch areas; Wherein, the displaying the first contact area in the fourth interface includes: Sequentially displaying the plurality of second touch areas in the fourth interface.

21. The method according to claim 20, wherein After the displaying the first contact area in the fourth interface, the method further includes: Detecting a third touch operation on the fourth interface by the user, wherein the contact area of the third touch operation with the touch screen is a third contact area; Detecting that the third contact area coincides or partially coincides with the first contact area displayed in the fourth interface; Providing an output effect to the user, and / or displaying a dynamic effect in the fourth interface; Sending an instruction to the first terminal device.

22. A first terminal device, characterized in that, Including: One or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the first terminal device is caused to execute the interaction method according to any one of claims 1 to 18.

23. A third terminal device, characterized in that, Including: One or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the third terminal device is caused to execute the interaction method according to any one of claims 19 to 21.

24. A computer-readable medium, characterized in that, Instructions are stored on the readable medium, and when the instructions are executed on a computer, the computer is caused to execute the interaction method according to any one of claims 1 to 21.

Images