An interaction method, device, electronic device and computer readable storage medium

Abstract

The application provides an interaction method and device, electronic equipment and computer readable storage medium, and relates to the technical field of electronic equipment. The method comprises the following steps: in the case that a first operation acting on a first interface triggered by an input device is detected, determining the device identifier of the input device; determining the target function configured by the input device based on the device identifier; determining the target object from the to-be-started objects based on the target function; wherein the to-be-started objects comprise applications and / or function modules; and starting the target object. The application can realize the quick starting of the applications and function modules in the electronic equipment, facilitate the user to use the applications and function modules quickly, and help to improve the interaction experience between the user and the electronic equipment.

Description

Technical Field

[0001] This application relates to the field of electronic device technology, and more specifically, to an interaction method, apparatus, electronic device, and computer-readable storage medium in the field of electronic device technology. Background Technology

[0002] Currently, when using smart devices, the process of opening specific functional controls is often cumbersome, requiring multiple steps. For example, in desktop software, users often need to first open the main menu and then navigate through layers of menus to reach the desired functional control. This not only increases operational complexity but also significantly reduces work efficiency. Summary of the Invention

[0003] This application provides an interaction method, apparatus, electronic device, and computer-readable storage medium. This application enables the rapid launch of applications and functional modules in electronic devices, facilitating quick and easy use of applications and functional modules by users, and improving the user's interactive experience with electronic devices.

[0004] In a first aspect, an interaction method is provided, comprising: upon detecting that an input device triggers a first operation on a first interface, determining a device identifier of the input device; determining a target function configured on the input device based on the device identifier; determining a target object from objects to be launched based on the target function; wherein the objects to be launched include applications and / or functional modules; and launching the target object.

[0005] Based on the above technical solution, the interaction method provided in this application adopts the following approach: upon detecting that an input device triggers a first operation on the first interface, the device identifier of the input device is determined, then the target function configured on the input device is determined based on the device identifier, and finally, the target object is determined from the objects to be launched based on the target function. The objects to be launched include applications and / or functional modules, thereby launching the target object. This technical solution enables the rapid launch of applications and / or functional modules associated with the input device in the electronic device when the user interacts with the electronic device using the input device. It simplifies the process of the user manually launching applications and / or functional modules, facilitates the user's quick use of applications and functional modules, and helps improve the user's interactive experience with the electronic device.

[0006] In one possible implementation, determining the target object from the objects to be launched based on the target function includes: obtaining multiple objects associated with the target function from the objects to be launched, and obtaining multiple candidate objects; displaying a first window; wherein the first window includes multiple candidate objects; responding to a first selection operation of the candidate objects in the first window, determining the candidate object selected by the first selection operation in the first window as the target object, thereby realizing the launch of the application that needs to be launched among the multiple applications associated with the target function of the input device according to the user's selection, and / or, the launch of the function module that needs to be launched among the multiple functional modules associated with the target function of the input device according to the user's selection.

[0007] In one possible implementation, if the target object belongs to a functional module, after the target object is started, the interaction method further includes: responding to the second operation of the input device on the first interface, executing the function of the target object, thereby realizing the quick use of the functional module.

[0008] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, the function of the target object includes: if the target object is an erasure tool and the second operation is an erasure operation performed by an input device on the first area of ​​the first interface, erasing the interface content in the first area, so that the user can quickly use the erasure function through the input device.

[0009] In one possible implementation, the function of executing the target object includes: if the target object is a selection tool and the second operation is a second selection operation of the input device acting on the second area of ​​the first interface, selecting the target content, so that the user can quickly use the selection function through the input device; wherein, the target content is the interface content in the second area.

[0010] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, after selecting the target content, the interaction method further includes: responding to the adjustment operation of the input device on the target content, moving and / or rotating the target content, so that the user can adjust the position and angle of the selected content through the input device.

[0011] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function of the target object includes: if the target object is a screenshot tool and the second operation is a screenshot operation performed by an input device on the third area of ​​the first interface, the interface content in the third area is captured to obtain a content screenshot, so that the user can quickly use the screenshot function through the input device.

[0012] In one possible implementation, after capturing the interface content in the third area and obtaining a screenshot, the interaction method further includes: displaying a second window; wherein the second window includes editing options and a save option; in response to a third selection operation of the editing option in the second window, displaying a third window; wherein the third window includes the interface content in the third area that is in an editable state; or, in response to a fourth selection operation of the save option in the second window, saving the screenshot, so that users can edit or save the screenshot according to their actual needs.

[0013] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function of the target object includes: if the target object is a magnifying glass tool, displaying the magnified page content in the fourth area, so that the user can quickly use the magnifying glass tool's magnification function through the input device; wherein, the fourth area is the area where the input device acts on the first interface.

[0014] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function of the target object includes: if the target object is a timer, displaying the timer in the fifth area and starting the timer to begin timing, so that the user can quickly use the timer function through the input device; wherein, the fifth area is any area of ​​the first interface; after starting the timer to begin timing, the interaction method also includes: responding to the stop timing operation of the input device on the timer, controlling the timer to stop timing, thereby realizing the use of the input device to control the timer to stop timing.

[0015] In one possible implementation, the interaction method further includes: displaying a fourth window in response to a third operation of the input device acting on a preset area in the second interface; wherein the fourth window includes a setting area for the timing duration; obtaining a target duration in response to a setting operation in the setting area; associating the target duration with a target function, so that when the timer is started based on the target function, timing starts based on the target duration, making it convenient for users to configure the timing duration for the input device according to their own needs.

[0016] In one possible implementation, the function of executing the target object includes: acquiring the pressure and / or duration of the input device acting on the first interface; adjusting the geometric properties of the target object according to the pressure and / or duration; and executing the function of the target object after adjusting the geometric properties, thereby realizing the quick adjustment of the size of the target object according to the pressure and / or duration of the input device acting on the interface.

[0017] In one possible implementation, if the target object belongs to an application, the interaction method further includes: displaying a fifth window in response to a fourth operation of the input device acting on a preset area in the third interface; wherein the fifth window includes application identifiers of multiple applications; and associating the application identifier selected by the fifth selection operation in the fifth window with the target function in response to a fifth selection operation of the application identifier in the fifth window, so that the user can configure the application that needs to be launched quickly for the input device according to their own needs.

[0018] Secondly, an interactive device is provided, the interactive device comprising:

[0019] The first determining module is used to determine the device identifier of the input device when a first operation triggered by the input device on the first interface is detected.

[0020] The second determining module is used to determine the target function configured by the input device based on the device identifier;

[0021] The third determination module is used to determine the target object from the objects to be launched based on the target function; wherein the objects to be launched include applications and / or functional modules;

[0022] The object startup module is used to start the target object.

[0023] Thirdly, an electronic device is provided, including a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, causing the electronic device to perform the interaction method in the first aspect or any possible implementation thereof.

[0024] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to execute the interaction method described in the first aspect or any possible implementation thereof.

[0025] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the interaction method described in the first aspect or any possible implementation thereof. Attached Figure Description

[0026] Figure 1 A schematic flowchart of an interaction method provided in an embodiment of this application is shown;

[0027] Figure 2 A schematic diagram illustrating the device identifier of an electronic device for recognizing input devices is shown;

[0028] Figure 3 This diagram illustrates a scenario where the target object is determined based on the target function.

[0029] Figure 4 This diagram illustrates a scenario where an electronic device performs an erasure function.

[0030] Figure 5 This diagram illustrates a scenario where an electronic device performs a selection function.

[0031] Figure 6 This illustration shows a scenario where a user adjusts the target content.

[0032] Figure 7 This diagram illustrates a scenario where an electronic device performs a screenshot function.

[0033] Figure 8 This illustration shows a scenario where a user edits a screenshot of content.

[0034] Figure 9 This diagram illustrates a scenario where an electronic device performs an interface content magnification function.

[0035] Figure 10 A schematic diagram illustrating a scenario where an electronic device performs a timing function is shown.

[0036] Figure 11 This diagram illustrates a scenario where a user sets the countdown duration.

[0037] Figure 12 This illustration shows a scenario where a user configures the applications to be launched for their input device.

[0038] Figure 13 An exemplary schematic diagram of a touch response system provided in an embodiment of this application is shown;

[0039] Figure 14 An exemplary cross-sectional view of an input device provided in an embodiment of this application is shown;

[0040] Figure 15 An exemplary encoding diagram of an input device provided in an embodiment of this application is shown;

[0041] Figure 16 It shows the way Figure 15 A schematic diagram of a local capacitance image when the input device triggers a click operation;

[0042] Figure 17 A schematic diagram illustrating the construction of a coordinate system provided in an embodiment of this application is shown;

[0043] Figure 18 This illustration shows a structural schematic diagram of an interactive device provided in an embodiment of this application;

[0044] Figure 19 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0045] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0046] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0047] Currently, when using smart devices, the process of opening specific functional controls is often cumbersome, requiring multiple steps. For example, in desktop software, users often need to first open the main menu and then navigate through layers of menus to reach the desired functional control. This not only increases operational complexity but also significantly reduces work efficiency. Especially in multitasking scenarios, frequently repeating this process greatly impacts the user experience, making what should be a simple task time-consuming and error-prone.

[0048] To address the aforementioned issues, this application provides an interaction method, apparatus, electronic device, and computer-readable storage medium. The interaction method provided in this application is applied to an electronic device with a capacitive screen. The electronic device is equipped with an input device (such as a smart stamp) for interaction. Users can configure the input device with their desired target functions and associate applications and / or functional modules in the electronic device with the target functions. When a user interacts with the electronic device using the input device, the electronic device identifies the target function of the input device and then determines the application and / or functional module that the user needs to launch based on the target function. This enables the rapid launch of applications and functional modules in the electronic device, facilitating quick use of applications and functional modules and improving the user's interaction experience with the electronic device.

[0049] The following is an embodiment of an interaction method provided in this application specification.

[0050] Figure 1A schematic flowchart of an interaction method provided in an embodiment of this application is shown, such as... Figure 1 As shown, the interactive method provided in this application is applied to electronic devices with capacitive screens, such as smart interactive flat panels, laptops, and tablets. These electronic devices carry various types of applications, such as whiteboard applications, chat applications, teaching applications, and browser applications. They also carry various functional modules (also called functional controls or gadgets), such as screenshot tools, timers (including countdown timers), calculators, and counters. Some applications carried by the electronic device also have functional modules; for example, whiteboard applications have erasers, pens, selection tools, and screenshot tools. It is worth noting that the "functional modules in electronic devices" described in this application include not only the functional modules carried by the electronic device but also the functional modules in the applications.

[0051] Electronic devices are equipped with input devices. When selling electronic devices, merchants will include an input device, and the electronic device will include at least one such input device for interacting with the electronic device. The input device is a type of passive interaction tool; for example, an input device could be a smart stamp, etc.

[0052] The input device has its own unique device identifier (called ID). Users can configure target functions for the input device according to their own usage needs, associate the target functions with the ID of the input device, associate the target functions of the input device with the application identifiers of at least some applications in the electronic device, or associate the target functions of the input device with the module identifiers of at least some functional modules in the electronic device, or associate the target functions of the input device with the application identifiers of at least some applications and the module identifiers of at least some functional modules in the electronic device.

[0053] Optionally, the input device can be configured with a target function through an electronic device or other terminal (e.g., a smartphone or computer), and the target function of the input device can be associated with the application identifier of at least some applications and / or the module identifier of at least some functional modules in the electronic device to generate configuration data. The configuration data is sent to a server for backup storage, and the electronic device retrieves the configuration data from the server for storage, so as to use it when the input device interacts with the electronic device.

[0054] It is worth noting that the configuration data stored in the server is updated in real time, which ensures that the configuration data obtained by the electronic device is up-to-date. This ensures that when the input device interacts with the electronic device, the applications and / or functional modules associated with the target function of the input device can be accurately activated.

[0055] When the input device is a smart stamp, smart stamps can be divided into non-pressable stamps and pressable stamps. When interacting with an electronic device using a non-pressable stamp, the user does not need to press the stamp. However, when interacting with an electronic device using a pressable stamp, the user needs to press the stamp; that is, using a pressable stamp involves an additional pressing operation compared to a non-pressable stamp. For example, when a user applies a non-pressable stamp to the capacitive screen of an electronic device (e.g., the non-pressable stamp makes contact with the capacitive screen), the user can directly control the non-pressable stamp to perform a sliding operation; that is, the non-pressable stamp triggers the sliding operation, and the electronic device responds to the sliding operation. When a user applies a pressable stamp to the capacitive screen of an electronic device (e.g., the pressable stamp makes contact with the capacitive screen), the user needs to press the stamp once first, then control the non-pressable stamp to perform a sliding operation, and after the sliding is complete, press the stamp again; that is, the pressable stamp triggers the sliding operation, and the electronic device responds to the sliding operation. Although using a shaped stamp involves an additional pressing operation compared to a non-shaped stamp, the response logic of the electronic device to operations triggered by both shaped and non-shaped stamps is the same. This application embodiment uses a non-shaped stamp as an example for illustration. The capacitive screen of the electronic device is a capacitive screen, such as a capacitive touchscreen.

[0056] An interactive method provided in this application includes the following steps:

[0057] S110: When it is detected that the input device triggers a first operation on the first interface, the device identifier of the input device is determined;

[0058] S120: Determine the target function of the input device configuration based on the device identifier;

[0059] S130: Determine the target object from the objects to be launched based on the target function; wherein, the objects to be launched include applications and / or functional modules;

[0060] S140: Start the target object.

[0061] In an exemplary embodiment, the interface currently displayed on the capacitive screen of the electronic device is referred to as the first interface. The first interface may be the desktop of the electronic device or the interface of a certain application.

[0062] If an input device triggers a first operation on the first interface displayed on the capacitive touchscreen, the ID of the input device is obtained. Here, the first operation can be understood as an operation involving contact with the first interface, such as a click operation.

[0063] The process of identifying the ID of the input device is as follows: The touch end (also called the contact end or contact surface) of the input device, which is used to contact the electronic device, is provided with multiple conductive elements. After the input device acts on the capacitive screen, the electronic device will identify the arrangement of the conductive elements that are connected among the multiple conductive elements, obtain the preset identifier corresponding to the arrangement, and thus obtain the ID of the input device.

[0064] For example, the touch terminal of the input device used for contacting electronic devices is provided with 8 conductive elements. Among the 8 conductive elements, there are 6 conductive elements that are connected and 2 conductive elements that are not connected. The conductive elements that are connected can be understood as conductive elements in the first grounding state, and the conductive elements that are not connected can be understood as conductive elements in the second grounding state. The first grounding state is used to indicate that the conductive element is grounded, and the second grounding state is used to indicate that the conductive element is not grounded. The grounding of the conductive element can be understood as the conductive element being connected to the ground wire to achieve grounding, or it can be understood as the conductive element being connected to the earth through the human body to achieve grounding.

[0065] A reference coordinate system is established with one corner of the capacitive touchscreen as the origin O, one side connecting to that corner as the X-axis, and the other side connecting to that corner as the Y-axis. When an input device is applied to the capacitive touchscreen, the electronic device can identify six conductive contacts, thus obtaining the position coordinates of these six conductive contacts in the reference coordinate system. For example... Figure 2 As shown, Figure 2 A schematic diagram showing the arrangement of conductive components that an electronic device identifies when they are connected is provided. Figure 2 The origin O of the reference coordinate system is the lower left corner of the capacitive touchscreen, the Y-axis is the left side of the touchscreen, and the X-axis is the bottom side of the touchscreen. After the input device is applied to the capacitive touchscreen, the electronic device can recognize six conductive elements, which are represented as conductive elements 2011-2016. The position coordinates of conductive elements 2011-5016 are P1, P2, P3, P4, P5, and P6, respectively. Then, a first matrix with M rows and N columns is constructed based on P1-P6, where M and N are positive integers, and at least one of M and N is greater than or equal to 2. The first matrix needs to include these six position coordinates. For example, if M = N = 4, the first matrix is ​​represented as R1, i.e.:

[0066]

[0067] In R1, "*" indicates that the element is empty.

[0068] After obtaining R1, a first identifier is used to represent the elements in the first matrix that are position coordinates, and a second identifier is used to represent the elements in the first matrix that are not position coordinates, resulting in a second matrix, denoted as R2. R2 represents the arrangement of these six conductive components. For example, if the first identifier is "1" and the second identifier is "0", then P1-P6 in R1 are all replaced with "1", and "*" is all replaced with "0", thus obtaining R2, i.e.:

[0069]

[0070] The arrangement of the conductive contacts on the touch screen of the input device. After obtaining R2, the preset identifier corresponding to R2 is the ID of the input device. For example, if the preset identifier corresponding to R2 is ID1, then the ID of the input device is ID1. Different input devices have different arrangements of the conductive contacts, thus allowing for the setting of a different ID for each input device.

[0071] Determining the target function configured for the input device includes: since the target function is configured for the input device in advance, the target function is associated with the ID of the input device. The electronic device stores configuration data about the input device in advance. After determining the ID of the input device, the electronic device queries the stored configuration data through the ID of the input device to obtain the function associated with the ID of the input device, that is, to obtain the target function configured for the input device.

[0072] The objects to be launched include applications and / or functional modules in the electronic device. After obtaining the target function configured on the input device, the configuration data is queried according to the target function configured on the input device to obtain the identifier corresponding to the target function configured on the input device, called the target object identifier. Then, the object whose object identifier is the target object identifier is obtained from the objects to be launched, that is, the target object is obtained. For example, if the target object identifier is identifier 1, and the object identifier of application 1 in the objects to be launched is identifier 1, then application 1 is the target object; as another example, if the target object identifier is identifier 2, and the object identifier of functional module 2 in the objects to be launched is identifier 2, then functional module 2 is the target object.

[0073] After identifying the target object from the list of objects to be launched, the target object is started. From the user's perspective, assuming the first interface is the desktop of an electronic device, for example, if the target object is a whiteboard application, the whiteboard application will open immediately after the user places the input device on the desktop; as another example, if the target object is a calculator, the calculator will open immediately after the user places the input device on the desktop; and as yet another example, if the target object is a countdown timer, the countdown timer will start and begin counting down after the user places the input device on the desktop, thus achieving rapid launch of applications and / or functional modules in the electronic device.

[0074] The interaction method provided in this application adopts the following technical solution: upon detecting that an input device triggers a first operation on a first interface, the device identifier of the input device is determined, then the target function configured on the input device is determined based on the device identifier, and finally, a target object is determined from the objects to be launched based on the target function. The objects to be launched include applications and / or functional modules, thereby launching the target object. This achieves the rapid launch of applications and / or functional modules associated with the input device in the electronic device when the user interacts with the electronic device using the input device. It simplifies the process of the user manually launching applications and / or functional modules, facilitates the user's quick use of applications and functional modules, and helps improve the user's interaction experience with the electronic device.

[0075] This application embodiment illustrates two target functions configured for an input device. For the same input device, after configuration data is configured, the input device corresponds to only one target function. The example illustrates target functions as launching an application and launching a gadget (also known as launching a function module or launching a function control). For instance, if the target function configured for the input device is launching an application, the input device is used to launch the application associated with that target function; if the target function configured for the input device is launching a gadget, the input device is used to launch the function module associated with that target function.

[0076] Furthermore, for the same input device, assuming its current target function and user information are already determined, if other functions are needed, the current target function of the input device needs to be changed to the other function via an electronic device or other terminal, and the identifier of the object associated with the target function also needs to be modified accordingly. For example, if the current target function of the input device is to launch an application, and the application associated with the application launch function is application 1, if the user wants to use the widget launch function, the current target function of the input device needs to be changed from the application launch function to the widget launch function via an electronic device or other terminal, and application 1 needs to be changed to the user-defined function module for the widget launch function (e.g., a countdown timer).

[0077] The following are Figure 1 The specific implementation methods of each step in the illustrated embodiment will be explained below:

[0078] In one possible implementation, determining the target object from the objects to be started based on the target function includes the following steps:

[0079] Retrieve multiple objects associated with the target function from the objects to be started, and obtain multiple candidate objects;

[0080] Display a first window; the first window includes multiple candidate objects;

[0081] In response to the first selection operation of the candidate objects in the first window, the candidate object selected by the first selection operation in the first window is determined as the target object.

[0082] For cases where the target function of an input device is associated with multiple objects, taking the example of the target function being the application launch function and the associated objects being applications, the explanation is as follows: Considering that although the target function is associated with multiple applications, the user may not necessarily need to launch all the applications associated with the target function at once, a method is provided that allows the user to select the actual application to launch from among the multiple applications associated with the target function. For example... Figure 3 As shown, Figure 3 This diagram illustrates a scenario where a target object is determined based on a target function. 101 represents the first interface, 200 represents the input device, and 301 represents the first window. Assuming the input device's target function (launching an application) is associated with three application identifiers, and the first interface is specifically the desktop of an electronic device, a user triggers a first operation (e.g., a click) on any area of ​​the first interface using the input device. The input device's ID is obtained, and its target function is determined to be the "launching an application" function. This leads to the three application identifiers associated with this function. These three application identifiers are then used to identify the corresponding applications from the list of objects to be launched: Application 1, Application 2, and Application 3. These three applications are then considered as candidate applications. The first window then displays Application 1, Application 2, and Application 3. The user triggers a first selection operation for Application 2 and Application 3, indicating that they have chosen these two applications as the target objects. Subsequently, Application 2 and Application 3 will be launched, thus enabling the launch of the required applications from multiple applications associated with the target function based on the user's selection. The checkmark "√" indicates that the mark is selected.

[0083] In one possible implementation, if the target object belongs to a functional module, after activating the target object, the interaction method further includes the following steps:

[0084] In response to a second operation performed by the input device on the first interface, the function of the target object is executed.

[0085] When the target object is a functional module, after activating the target object, the user can further trigger its function using an input device. Specifically, the user uses the input device to trigger a second operation on the first interface. After the electronic device responds to the second operation, it executes the target object's function, enabling quick access to the functional module. For example, if the target object is a selection tool, the selection function is executed; if the target object is a screenshot tool, the screenshot function is executed; if the target object is a magnifying glass, the content magnification function is executed; if the target object is a timer, the timer function (including countdown and up-and-down functions) is executed; if the target object is an eraser tool, the eraser function is executed; and so on. The second operation can be the same as or different from the first operation.

[0086] The application types of the first interface are pre-defined into two categories: designated applications and non-designated applications. Different application types result in different execution strategies for the electronic device to perform the functions of the target object. Designated applications are interactive collaborative applications where users can design and create, such as drawing applications and whiteboard applications. Non-designated applications can be understood as applications other than designated applications, such as browser applications. This application embodiment uses a whiteboard application as the designated application and a browser application as the non-designated application for illustration.

[0087] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, the above-mentioned execution of the target object's function includes the following steps:

[0088] If the target object is an erasing tool and the second operation is an erasing operation performed by an input device on the first area of ​​the first interface, the interface content in the first area is erased.

[0089] If the application type of the first interface is a specified application and the target object is an erasure tool (e.g., a whiteboard eraser), and the second operation is an erasure operation (e.g., a swipe operation, also called a move operation) performed by the input device on the first area of ​​the first interface, then the interface content in the first area will be erased. It can be understood that the input device is currently acting as an erasure tool; when the user swipes on the first interface using the input device, the interface content along the swipe path of the input device will be erased. Figure 4 As shown, Figure 4 This diagram illustrates a scenario where an electronic device performs an erasure function. 101 represents the first interface, 101-1 represents the updated first interface, and P1 represents the first area. The first interface displays the written text "5+2=7," and the user uses an input device ( Figure 4(Not shown in the image) The first area was slid to erase the written "2". After the first interface is updated, the written "2" is no longer displayed in the first area of ​​the updated first interface, making it convenient for users to quickly use the erase function through the input device.

[0090] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application or a non-specified application, the above-mentioned execution of the target object's function includes the following steps:

[0091] If the target object is a selection tool and the second operation is a second selection operation performed by an input device on the second area of ​​the first interface, the target content is selected by box selection; wherein, the target content is the interface content in the second area.

[0092] For applications belonging to the first interface that are either specified or non-specified applications, and whose target object is a selection tool, if the second operation is a second selection operation performed by the input device on the second area of ​​the first interface, then the interface content in the second area is selected by drawing a box, and the selected interface content is called the target content. The second selection operation can be a click operation or a swipe operation. If the second selection operation is a click operation, then the second area is the area where the input device contacts the first interface; if the second selection operation is a swipe operation, then the second area is the area traversed by the sliding trajectory of the input device after it contacts the first interface.

[0093] like Figure 5 As shown, Figure 5 This diagram illustrates a scenario where an electronic device performs a selection function. 101 represents the first interface, 101-2 represents the updated first interface, P2 represents the second area, and K1 represents the selection marker. The interface content displayed in the first interface includes "rectangular, circular, and triangular patterns," and the user uses an input device (…). Figure 5 (Not shown in the image) The input device slides across the "rectangle, circle and triangle pattern" on the first interface, thus indicating that the user has selected the "rectangle, circle and triangle pattern". The "rectangle, circle and triangle pattern" is then selected. After the first interface is updated, the selected "rectangle, circle and triangle pattern" is displayed in the first area of ​​the updated first interface, making it convenient for the user to quickly use the selection function through the input device.

[0094] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, after selecting the target content, the interaction method also includes the following steps:

[0095] In response to an adjustment operation performed by the input device on the target content, the target content is moved and / or rotated.

[0096] If the application type of the application to which the first interface belongs is a specified application and the target object is a selection tool, the response second operation is a second selection operation performed by the input device on the second area of ​​the first interface to obtain the selected target content. After that, if an adjustment operation is detected by the input device on the target content, the target content is adjusted according to the adjustment operation. Adjusting the target content includes moving and / or rotating the target content, allowing the user to adjust the position and angle of the selected content through the input device. If the adjustment operation is a move operation, the target content is moved; if the adjustment operation is a rotation operation, the target content is rotated; if the adjustment operation is both a move operation and a rotation operation, the target content is both moved and rotated.

[0097] like Figure 6 As shown, Figure 6 This diagram illustrates a scenario where a user adjusts the target content. Interface 101-3 shows the interface displaying the moved target content; interface 101-3 displays a "rectangular, circular, and triangular pattern" based on interface 101-2 after it has been moved downwards. Interface 101-4 shows the interface displaying the rotated target content; interface 101-4 displays a "rectangular, circular, and triangular pattern" based on interface 101-2 after it has been rotated clockwise.

[0098] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the above-mentioned execution of the target object's function includes the following steps:

[0099] If the target object is a screenshot tool and the second operation is a screenshot operation performed by an input device on the third area of ​​the first interface, the interface content in the third area is captured to obtain a content screenshot.

[0100] If the application type of the first interface is a non-specified application and the target object is a screenshot tool, and the second operation is a screenshot operation performed by the input device on the third area of ​​the first interface, then the interface content in the third area is captured to obtain a content screenshot, meaning the content screenshot includes the interface content in the third area. The screenshot operation includes click and swipe operations. The process of the user performing a screenshot operation using the input device is as follows: first, the user clicks on a certain location, which is considered the starting point; then, the user moves the input device away from the starting point. The user will see a small rectangle gradually enlarge as the input device moves away from the starting point. The position where the input device stops moving is the ending point. The interface content covered by the rectangle formed by the starting point and the ending point is the interface content in the third area. Therefore, the interface content in the third area is captured to obtain a content screenshot, which includes the interface content in the third area, allowing users to quickly use the screenshot function through the input device.

[0101] like Figure 7 As shown, Figure 7 This diagram illustrates a scenario where an electronic device performs a screenshot function. 101 represents the first interface, P3 represents the third area, and F represents the content screenshot. The content displayed on the first interface includes a short essay titled "Spring is Coming," which the user captures using an input device (…). Figure 7 (Not shown in the image) A screenshot operation was performed on the first interface. After the input device stopped performing the screenshot operation, the content of the interface covered by the rectangle formed by the start and end positions of the screenshot operation was "In the morning, under the slightly white sky, ..., spring has truly arrived.", thus obtaining the second paragraph of content in the screenshot, which reads "Spring is here."

[0102] In one possible implementation, after capturing the interface content in the third region and obtaining the content screenshot, the interaction method further includes the following steps:

[0103] A second window is displayed; this second window includes editing options and a save option.

[0104] In response to a third selection of editing options in the second window, a third window is displayed; wherein the third window includes the interface content in a third area that is in an editable state; or,

[0105] In response to the fourth selection of the save option in the second window, a screenshot of the saved content is taken.

[0106] If the application type of the first interface is a non-specified application and the target object is a screenshot tool, and if the second operation is a screenshot operation performed by the input device on the third area of ​​the first interface to capture the interface content in the third area, a second window is displayed. The second window displays editing and saving options for the screenshot. The user can edit the screenshot before saving it, or save it directly, according to their needs. For example... Figure 8 As shown, Figure 8 This diagram illustrates a scenario where a user edits a screenshot. 101 represents the first interface, K2 represents the rectangle formed by the start and end points of the screenshot operation, 302 represents the second window, and 303 represents the third window. If the electronic device detects a third selection of editing options in the second window, indicating that the user has chosen to edit, the third window is displayed. The third window shows the second paragraph of the text "Spring is Coming," which the user can edit.

[0107] If the electronic device detects the fourth selection operation of the save option in the second window, it means that the user has selected the save option. The device will then save the screenshot and prompt the user with the save path for the screenshot.

[0108] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the above-mentioned execution of the target object's function includes the following steps:

[0109] If the target object is a magnifying glass tool, the magnified page content in the fourth area is displayed; the fourth area is the area where the input device acts on the first interface, that is, the area where the input device contacts the first interface.

[0110] If the application type of the application belonging to the first interface is a non-specified application and the target object is a magnifying glass tool, and if the second operation is a click operation of the input device on the fourth area of ​​the first interface, the magnified page content in the fourth area will be displayed. For example... Figure 9 As shown, Figure 9 This diagram illustrates a scenario where an electronic device performs a magnification function on its interface. 101 represents the first interface, P4 represents the fourth area, and K3 represents the magnified page content. When a user clicks "Freedom" using an input device, the magnified version of "Freedom" is displayed, allowing the user to quickly access the magnification function of the magnifying glass tool via the input device.

[0111] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, executing the function of the target object includes the following steps:

[0112] If the target object is a timer, display the timer in the fifth area and start the timer to begin timing; the fifth area can be any area of ​​the first interface.

[0113] If the application type of the application belonging to the first interface is a non-specified application and the target object is a timer, and if the second operation is a click operation of the input device on the fifth area of ​​the first interface, the timer is displayed in the fifth area and started to begin timing, allowing the user to quickly use the timer function through the input device. If the timer is a positive timer, then the positive timing begins; if the timer is a countdown timer, then the countdown begins. Figure 10 As shown, Figure 10 This diagram illustrates a scenario where an electronic device performs a timing function. 101 represents the first interface, P5 represents the fifth area, and K4 represents the countdown timer. (Reference) Figure 10 In Figure a, the countdown timer has a duration of 60 seconds, and the countdown starts at 60 seconds.

[0114] After starting the timer as described above, the interaction method further includes the following steps:

[0115] In response to a stop operation on the timer performed by the input device, the timer is controlled to stop counting.

[0116] If an electronic device detects a stop operation (e.g., a click) performed by an input device on a timer, it controls the timer to stop, thus enabling the use of an input device to control the timer's stop. (Reference) Figure 10 In image b, the user clicked the countdown timer using the input device, and the countdown timer stopped at 30 seconds, meaning there are 30 seconds left.

[0117] In one possible implementation, the above interaction method further includes the following steps:

[0118] In response to a third operation performed by the input device on a preset area in the second interface, a fourth window is displayed; wherein the fourth window includes a setting area for the timing duration;

[0119] The target duration is obtained in response to the setting operation in the settings area;

[0120] Associate the target duration with the target function so that if the timer is started based on the target function, the timer will start based on the target duration.

[0121] Users can set the duration of a timer through an electronic device, and then associate the set duration with the target function of the input device. When the electronic device starts the timer based on the target function of the input device, the electronic device obtains the duration corresponding to the target function of the input device, and then controls the timer to keep time according to the duration corresponding to the target function of the input device.

[0122] The second interface can be understood as the interface for setting the timing duration. If the electronic device detects that the input device is acting on the preset area in the second interface and triggers the third operation, a fourth window is displayed. The fourth window includes a setting area for the timing duration, where the user can set the timing duration. The user can either directly input the duration or select a duration from the provided options, allowing the user to configure the timing duration for the input device according to their needs. Both directly inputting the duration and selecting a duration from the provided options are considered setting operations.

[0123] The set timing duration is called the target duration. This target duration is then associated with the target function of the input device. When the electronic device starts the timer based on the target function of the input device, the electronic device obtains the target duration corresponding to that function and then controls the timer to keep track of the time according to the target duration. The set timing duration can be either a positive countdown duration or a countdown duration.

[0124] like Figure 11 As shown, Figure 11The diagram illustrates a scenario where a user sets a countdown duration. 102 represents the second interface, P6 represents the preset area within the second interface, 304 represents the fourth window, and 200 represents the input device. If the electronic device detects that the input device is operating on the preset area of ​​the second interface and triggers a third operation, it displays the fourth window. The fourth window includes countdown duration options of 60 seconds and 30 seconds. If the user selects 30 seconds, the target duration is 30 seconds, and the target duration is then associated with the target function of the input device. After the electronic device starts the countdown timer based on the target function of the input device, the electronic device obtains the target duration corresponding to the target function of the input device as 30 seconds, and then controls the countdown timer to run in 30-second increments.

[0125] In one possible implementation, the aforementioned execution of the target object's functionality includes:

[0126] Acquire the pressure and / or duration of the input device acting on the first interface;

[0127] Adjust the geometric properties of the target object based on pressure and / or duration;

[0128] Performs the function to adjust the geometric properties of the target object.

[0129] When the target object belongs to a functional module, the user can adjust the geometric properties (such as size) of the target object through an input device, and the electronic device will then execute the function of the target object after the geometric properties have been adjusted.

[0130] After the input device acts on the first interface, the pressure and / or duration of the input device acting on the first interface are obtained. For example, if the input device acting on the first interface actually means that the input device comes into contact with the first interface, then the pressure of the input device acting on the first interface is the pressure applied by the input device to the first interface (which is also the pressure applied by the input device to the capacitive screen), and the duration of the input device acting on the first interface is the contact duration between the input device and the first interface (which is also the contact duration between the input device and the capacitive screen).

[0131] The process for obtaining the first duration of the input device's action on the first interface is as follows: Upon detecting that the input device is acting on the first interface, a timer is started to count the duration of the input device's action on the first interface (starting from 0). When the input device is no longer acting on the first interface, the timer stops counting. The timer's counted duration after stopping is the first duration. When the input device is not acting on the first interface, the pressure on the first interface is called the initial pressure (generally considered 0). When the difference between the detected pressure on the first interface and the initial pressure is greater than a preset value, it is considered that the first interface is under pressure from an external object. If the device identifier of the input device is detected, it is considered that the external object is the input device, thus confirming that the input device is acting on the first interface and applying pressure, and the timer starts counting. When the detected pressure on the first interface returns to the initial pressure, it is considered that the input device is no longer acting on the first interface, and the timer stops counting.

[0132] The pressure acquisition process of the input device acting on the first interface (which is also the pressure acquisition process of the capacitive screen): The capacitive screen includes multiple layers of capacitive sensors. Each layer of capacitive sensors can detect changes in capacitance. Beneath the screen is a layer of flexible material used to sense minute deformations on the screen surface, i.e., the minute displacements (usually measured in micrometers) that occur when the screen is pressed. When the screen is pressed, a certain pressure is applied, causing minute deformations on the screen surface and changing the capacitance values ​​between the layers of capacitive sensors. By measuring the capacitance changes and the deformation of the screen surface, the magnitude of the pressure applied to the capacitive screen is calculated.

[0133] When the screen is pressed, the surface capacitance change value C1 (i.e., the capacitance value detected by the outermost sensor), the deep capacitance change value C2 (i.e., the capacitance value detected by at least one sensor below the outermost layer), and the screen surface deformation Δd are obtained. The pressure F on the capacitive screen is obtained using the following pressure calculation formula:

[0134] F = k1 × (C1 - C2) + k2 × Δd;

[0135] Wherein, k1 represents the contribution weight of the difference in capacitance change between the surface and the deep layer (C1-C2) to the pressure, which reflects the ability to estimate the pressure through capacitance change, and k2 represents the contribution weight of the deformation Δd of the screen surface to the pressure, which reflects the ability to estimate the pressure through the physical deformation of the screen surface.

[0136] The system obtains the pressure exerted by the input device on the first interface, then retrieves the preset size corresponding to that pressure from a mapping relationship between pressure and preset size, thus obtaining the target size. Finally, it displays a target object with the target size. The mapping relationship between pressure and preset size includes multiple pressures and their corresponding preset sizes. The pressure exerted by the input device on the first interface cannot increase indefinitely; it is limited. Within this limited pressure range, pressure and size are positively correlated. That is, within the limited pressure range, the greater the pressure exerted by the input device on the first interface, the larger the size of the displayed target object. This allows for quick adjustment of the target object's size based on the pressure exerted by the input device on the interface.

[0137] The system obtains the duration of the input device's interaction with the first interface, then retrieves the preset size corresponding to that duration from a mapping relationship between duration and preset size, thus obtaining the target size. Finally, it displays the target object with the target size. The mapping relationship between duration and preset size includes multiple durations and their respective preset sizes. The duration of the input device's interaction with the first interface can be increased indefinitely, but the size of the functional module will not increase indefinitely. That is, once the duration of the input device's interaction with the first interface exceeds a set duration, the size of the functional module will not increase further and will always remain at its maximum size. Within the limited size range, the duration and the size of the functional module are positively correlated. In other words, within the limited size range, the longer the duration of the input device's interaction with the first interface, the larger the size of the displayed target object, thus enabling quick adjustment of the target object's size based on the duration of the input device's interaction with the interface.

[0138] The system obtains the pressure and duration of the input device applied to the first interface. Then, it retrieves the preset size corresponding to the pressure and duration from a mapping relationship between these two values ​​and preset sizes. The target size is then obtained, and the target object with the target size is displayed. This allows for quick adjustment of the target object's size based on the pressure and duration of the input device applied to the interface. The mapping relationship between pressure and duration and preset sizes includes multiple sets of durations and pressures, and multiple sets of preset sizes corresponding to each duration and pressure.

[0139] The function of a target object after its geometric properties are adjusted is the same as that of the target object before the adjustment. However, the response output of an electronic device after executing the function of the target object with adjusted geometric properties may be the same as or different from that of the target object before the adjustment. For example, if the target object is a whiteboard eraser, a large whiteboard eraser occupies a larger area of ​​the interface than a small whiteboard eraser, so executing the erase function of a large whiteboard eraser can erase more interface content at once. Another example is a paintbrush; a large paintbrush produces thicker strokes than a small paintbrush. Yet another example is a countdown timer; executing the countdown function of a large countdown timer will still show the countdown process to the user, only the size of the countdown numbers will be larger.

[0140] In one possible implementation, if the target object belongs to an application, the above interaction method further includes the following steps:

[0141] In response to a fourth operation performed by the input device on a preset area in the third interface, a fifth window is displayed; wherein the fifth window includes application icons for multiple applications;

[0142] In response to the fifth selection operation of the application identifier in the fifth window, the application identifier selected in the fifth selection operation in the fifth window is associated with the target function.

[0143] The third interface can be understood as an interface used to configure applications for the input device. If the electronic device detects that the input device is acting on a preset area in the third interface and triggers a fourth operation (such as a click), a fifth window is displayed. The fifth window displays application icons for multiple applications for the user to select. The user can select one application icon from the multiple application icons, or select two or more application icons. After the user selects an application icon, the selected application icon is associated with the target function of the input device. When the input device interacts with the electronic device, the application corresponding to the user's selected application icon can be quickly launched.

[0144] like Figure 12 As shown, Figure 12This diagram illustrates a scenario where a user configures an input device to launch an application. 103 represents the third interface, P7 represents a preset area within the third interface, 305 represents the fifth window, and 200 represents the input device. If the electronic device detects that the input device is operating on the preset area of ​​the third interface and triggers a fourth operation, the fifth window is displayed. The fifth window displays application identifiers, including application identifier 1 and application identifier 2. If the electronic device detects the fifth selection operation for application identifier 2 in the fifth window, indicating that the user has selected application identifier 2, then application identifier 2 is associated with the target function of the input device. When the input device interacts with the electronic device, the application corresponding to application identifier 2 can be quickly launched, allowing the user to configure the input device to launch applications according to their needs.

[0145] The following describes the process by which the electronic device provided in this application responds to a click operation triggered by an input device on the first interface.

[0146] Electronic devices include a capacitive touchscreen, a host computer, and a controller for controlling the capacitive touchscreen.

[0147] If the input device operates on the first interface (i.e., the input device operates on the capacitive screen), after the controller acquires the capacitive image from the capacitive screen, it sends the capacitive image to the host computer. The host computer then receives the capacitive image and decodes it based on the multiple touch points included in the image to obtain the encoding of the input device. It then executes the corresponding instruction (i.e., receives the instruction to perform a click operation), thereby enabling the host computer to respond specifically to the input device and achieve a click operation response. The touch point position information in the capacitive image refers to the position information of the conductive component in the input device on the capacitive screen after the input device operates on it (i.e., the touch point coordinates).

[0148] A capacitance image, also known as a capacitance map, is an image constructed based on the capacitance change at each capacitor node in a conductive array. A capacitance image is typically a grayscale image, where each pixel corresponds to a capacitor node, and the pixel value (grayscale value) of each pixel corresponds to the magnitude of the capacitance change at that node. Furthermore, since each capacitor node corresponds to a coordinate on the capacitive touchscreen, the position of each pixel in the capacitance image is the same as the position of its corresponding capacitor node on the touchscreen. In other words, there is a one-to-one correspondence between the pixels in the capacitance image and the pixels on the touchscreen, with each pixel in the capacitance image corresponding to a coordinate on the touchscreen.

[0149] To facilitate understanding of this application, the following is a brief introduction to the application scenarios and input devices involved in this application, in conjunction with the accompanying drawings.

[0150] See Figure 13 , Figure 13This diagram illustrates an exemplary touch response system according to an embodiment of this application. The system includes an electronic device 31 and an input device 32. The electronic device 31 integrates a capacitive touchscreen, which can display data such as images, videos, and text, and also receive and respond to click operations. Multiple conductive elements 321 can be disposed on the input device 32. These conductive elements 321 not only uniquely encode the input device 32 but also enable the input device 32 to perform click operations, forming multiple touch points 311 on the capacitive touchscreen. The electronic device 31 can decode the multiple touch points 311 to obtain a unique code corresponding to the input device 32, and execute the corresponding instruction based on the code to respond to the operation triggered by the input device 32.

[0151] See Figure 14 , Figure 14 An exemplary cross-sectional view of an input device provided in an embodiment of this application is shown. The outer contour of the input device 40 can be a cube, a cuboid, or other shapes, and this embodiment is not limited to these. The input device 40 has a touch end 41 and a grip end 42. A plurality of conductive elements 43 are provided on the input device 40, and each conductive element 43 has a portion protruding from the touch end 41. When the touch end 41 abuts against the capacitive screen, each conductive element 43 can make contact with the capacitive screen. The input device 40 also has a connecting part 44 inside. One end of the connecting part 44 is connected to each conductive element 43, and the other end is exposed from the grip end 42, so that the user's hand can contact the connecting part 44 when holding the input device 40.

[0152] Combination Figure 13 As shown, both the conductive element 43 and the connecting part 44 are made of conductive materials. When the user indirectly performs a click operation through the input device 40, the hand remains in contact with the connecting part 44 while holding the input device. The connecting part 44 short-circuits the hand to each conductive element 43. At this time, each conductive element 43 is equivalent to a finger of the user. When the touch end 41 touches the capacitive screen, each conductive element 43 can cause a change in the capacitance of the corresponding capacitor node on the capacitive screen, forming a touch point on the capacitive screen.

[0153] For example, when encoding an input device using multiple conductive elements, multiple first conductive elements can be provided on the input device. Based on these first conductive elements, multiple encoding regions are divided on the input device, and each encoding region corresponds one-to-one with multiple code elements in the encoding of the input device. For each code element, the specific value of the code element is characterized by the capacitance change caused by a second conductive element provided in the corresponding encoding region. Both the first and second conductive elements are conductive elements that are in contact with the input.

[0154] See Figure 15 , Figure 15An exemplary encoding diagram of an input device provided in an embodiment of this application is shown. (In conjunction with...) Figure 14 As shown, the touch terminal 41 is provided with a conductive element arrangement surface 411. N first conductive elements 431 (N is an integer greater than 2, for example, 3) and M second conductive elements 432 (M can be 0) are arranged on the conductive element arrangement surface 411. Multiple first conductive elements 431 form a preset pattern (the preset pattern can be a non-rotationally symmetric pattern), which can identify the input device as a recognizable input device. The M second conductive elements 432 are used to encode the input device. Three first conductive elements 431 can form a non-rotationally symmetric preset pattern "L". A coordinate system is constructed based on the three first conductive elements, and multiple encoding regions are divided within the coordinate system. Second conductive elements 432 are arranged within the encoding regions.

[0155] like Figure 15 As shown, a rectangular coordinate system is constructed with the center point of one of the first conductive components as the origin of the coordinate system, and the two directions from the center point of the first conductive component to the center points of the other two first conductive components as the directions of the two axes of the rectangular coordinate system. Multiple adjacent coding regions are divided in the coordinate system.

[0156] Optionally, the dimensions of each conductive element (including the first and second conductive elements) can be designed so that each conductive element can form a square touch point with a side length of K (K is, for example, 3 pixels) on the capacitive screen. Based on the dimensions of the square touch point, the conductive element arrangement surface 411 is divided into 16 square regions of the same size, each square region having a side length of H, where H is greater than or equal to K. The relative distance between the center points of two adjacent regions is H, and the number of the upper left corner of each region is the region number. Regions 1, 4, and 13 are reference regions, which are used to set the first conductive elements so that the three first conductive elements form a preset shape "L".

[0157] When placing conductive components within each region, the center point of the conductive component coincides with the center point of the region. The center point of the first conductive component in region 1 can be set as the origin of the coordinate system. The direction from the center point of region 1 to the center point of the first conductive component in region 4 is the positive direction of the X-axis in the coordinate system, and the direction from the center point of region 1 to the center point of the first conductive component in region 13 is the negative direction of the Y-axis in the coordinate system. Figure 1 In the constructed rectangular coordinate system, the symbol X2 represents the X-axis and the symbol Y2 represents the Y-axis.

[0158] Among them, regions 2 and 3, regions 5-12, and regions 14 and 15 are coding regions. Each coding region corresponds to a symbol in the encoding of the input device. The number in the lower right corner of each coding region is the region number of the coding region, which also represents the bit index of the coding region. The bit index is the position of the symbol corresponding to the coding region in the encoding.

[0159] When the conductive component arrangement surface 411 abuts against the capacitive touchscreen, both the first conductive component 431 and the second conductive component 432 can contact the capacitive touchscreen and form touch points on the capacitive touchscreen respectively. To avoid confusion, the touch point formed by the first conductive component 431 will be referred to as the first touch point, and the touch point formed by the second conductive component 432 will be referred to as the second touch point.

[0160] like Figure 15 As shown, when a click operation is performed via the input device, each area in the conductive element arrangement surface 411 is opposite to an area in the capacitive screen. The area in the capacitive screen corresponding to the reference area (hereinafter referred to as the positioning area) has a first touch point formed by the first conductive element, and the center point of the first touch point is located at the center point of the positioning area. When a second conductive element is provided in the encoding area, the area in the capacitive screen corresponding to the encoding area (hereinafter referred to as the decoding area) has a second touch point formed by the second conductive element, and the center point of the second touch point is located at the center of the decoding area. When no second conductive element is provided in the encoding area, there is no second touch point formed by the second conductive element in the decoding area corresponding to the encoding area.

[0161] Based on the above examples, the coordinates of the encoded region in this coordinate system can be represented by the coordinates of the center point of the encoded region in the coordinate system. The coordinates of the encoded region in this coordinate system can be called the preset coordinates. Since the relative positional relationship between the encoded region and the reference region is fixed and known, the relative positional relationship between the decoded region and the positioning region is also fixed and known.

[0162] For example, the encoding of the input device can be a binary number. When the encoding area has a second conductive element, the corresponding code element of the encoding area is 1; when the encoding area does not have a second conductive element, the corresponding code element of the encoding area is 0. The decoding area corresponds one-to-one with the encoding area. Therefore, for each encoding area, when a second touch point formed by the second conductive element exists within the decoding area, the corresponding code element of the decoding area is 1; when no second touch point formed by the second conductive element exists within the decoding area, the corresponding code element of the decoding area is 0. Furthermore, since the relative positional relationship between the decoding area and the first touch point is the same as the relative positional relationship between the corresponding encoding area and the first conductive element, the decoding area and the corresponding encoding area correspond to the same bit index.

[0163] like Figure 14As shown, the conductive component arrangement surface 411 has a total of 12 coding areas. Therefore, the code of the input device is a 12-bit binary number. There is no second conductive component in coding area 1, so the first bit of the binary number is 0. There is a second conductive component in coding area 2, so the second bit of the binary number is 1, and so on. Figure 15 The input device shown is coded as “010111101010”. It can be understood that when no second conductive element is provided in any of the coded areas, the input device is coded as “000000000000”, while when a second conductive element is provided in all coded areas, the input device is coded as “111111111111”.

[0164] When encoding different input devices, a unique 12-bit binary number (i.e., code) can be assigned to each input device. Then, for each code element in the binary number, the corresponding encoding region is determined from 12 encoding regions based on the bit index of the code element in the binary number. When the code element is 1, a second conductive element is set in the corresponding encoding region; when the code element is 0, no second conductive element is set in the corresponding encoding region, so as to achieve unique encoding of the input device.

[0165] like Figure 14 and Figure 15 As shown, the preset pattern formed by multiple first conductive elements 431 is an "L" shape, and the relative distance between any two first conductive elements 431 is a preset distance (hereinafter referred to as the preset distance). Based on this, when a click operation is performed through the input device and multiple touch points are formed on the capacitive screen, each first touch point is located within the positioning area corresponding to the reference area. The electronic device can identify each first touch point from the multiple touch points based on the relative positional relationship between the various touch points. The relative distance between two conductive elements refers to the distance between the center points of the two conductive elements, and the relative distance between two touch points refers to the distance between the center points of the two touch points.

[0166] Simultaneously, each second touch point is located within the decoding area corresponding to the encoding area. The electronic device can determine the bit index corresponding to the decoding area based on the relative positional relationship between the decoding area and multiple first touch points (i.e., positioning areas), and determine whether there is a second touch point within the decoding area. When there is a second touch point, the corresponding code element can be determined to be 1. When there is no second touch point (i.e., no second conductive element is provided), the corresponding code element can be determined to be 0. In this way, each code element in the encoding can be determined. The encoding of the input device can be obtained by combining all code elements and the bit index corresponding to each decoding area.

[0167] In practical applications, by properly setting the size of each conductive component, the size of the touch point formed when the conductive component contacts the capacitive screen can be controlled, and the conductive components can be prevented from interfering with each other. This ensures that the touch points formed by the conductive components on the capacitive screen do not interfere with each other, and each touch point can be clearly identified and distinguished by the electronic device.

[0168] It should be noted that the encoding method of the input device can be as follows: Figure 15 As shown, other methods can also be used, as long as they can uniquely identify the input device. The encoding method of the input device can include, but is not limited to, the examples above.

[0169] The electronic device responds to a click operation triggered by an input device on a first interface, including the following steps:

[0170] S210: The controller acquires the capacitance image from the capacitive touchscreen and sends the capacitance image to the host.

[0171] In one implementation, after acquiring the capacitance image of the capacitive touchscreen, the controller can directly send the complete capacitance image to the host. For example, during operation, the electronic device can periodically scan the conductive element array in the capacitive touchscreen. During each scan, the capacitance change at each capacitor node in the conductive element array can be acquired, and a capacitance image can be drawn based on the capacitance change at each capacitor node. After acquiring the capacitance image each time, the controller can directly send the capacitance image to the host.

[0172] The controller and the host can communicate via the USB Video Class (UVC) protocol. When communicating with the host via the UVC protocol, the controller can continuously send each captured capacitance image to the host.

[0173] S220: The host receives the capacitive image, decodes it based on multiple touch points included in the capacitive image to obtain the first target code of the input device, and executes the instruction corresponding to the first target code.

[0174] For example, the host receives a complete capacitance image sent by the controller. After receiving the complete capacitance image, the host can determine whether there are multiple touch points in the capacitance image. If there are multiple touch points in the capacitance image, the host can decode the multiple touch points. Since the multiple conductive elements on the input device correspond one-to-one with the multiple touch points in the capacitance image, decoding the multiple touch points is equivalent to decoding the multiple conductive elements.

[0175] See Figure 16 , Figure 16 It shows the way Figure 15The diagram shows a partial capacitance image when a click operation is triggered by the input device. In this capacitance image 70, the pixel value of each pixel represents the capacitance change at the corresponding capacitance node. A larger pixel value indicates a larger capacitance change at the corresponding capacitance node, and consequently, a higher pixel brightness. (Combined with...) Figure 15 As shown, when passing through Figure 15 When the input device shown performs a click operation, each conductive element in the conductive element arrangement surface 411 is in contact with the capacitive screen. Each conductive element can cause a change in the capacitance at the capacitor node at the contact position. Each first conductive element 431 can form a first touch point 71 in the capacitive image 70, and each second conductive element 432 can form a second touch point 72 in the capacitive image 70.

[0176] When the capacitance image includes Figure 16 When multiple touch points are shown, due to Figure 16 The multiple touch points shown are Figure 15 The multiple conductive components shown correspond one-to-one with each other. Figure 16 Decoding multiple touch points in the process is equivalent to... Figure 15 The multiple conductive components shown are decoded, and the resulting code is... Figure 15 The first target code of the input device shown.

[0177] In capacitive image, touch points refer to touch points on capacitive screens. Touch points are also called touch areas. A touch area typically consists of one or more pixels with pixel values ​​greater than a preset threshold. A pixel value greater than the preset threshold indicates that the capacitance at the corresponding capacitive node has changed significantly due to the presence of the input device. Generally, the pixel located at the center of the touch area (hereinafter referred to as the center pixel) has the largest pixel value, and the coordinates of the center pixel are usually the coordinates of the touch point on the capacitive screen.

[0178] In one implementation, when the controller sends a complete capacitance image to the host, the host can identify the capacitance image using a decoding model to determine the first target code of the input device. A neural network model can be pre-trained so that the trained neural network model (i.e., the decoding model) can identify the first target code from the capacitance image.

[0179] When training a neural network model, multiple training samples can be pre-acquired, each corresponding to a specific input device. Each training sample includes a sample image and a sample label for the corresponding input device. The sample image is a capacitance image when the input device performs a click operation, and the sample label is the first target encoding of the input device. Figure 15 Taking the input device shown as an example, it is possible to obtain the information in advance. Figure 15The capacitance image shown is generated when the input device performs a click operation. This capacitance image is used as a sample image, and the corresponding first target code "010111101010" is used as the sample label. Combining the sample image and the sample label yields a training sample. Similarly, one training sample can be obtained for each input device. When there are a large number of input devices, a large number of training samples can be acquired.

[0180] Simultaneously, an initial model can be constructed, which refers to an untrained neural network model for image recognition. Then, the initial model can be trained using the acquired training samples. During training, for each training sample, the sample image from the training sample can be input into the initial model to obtain the decoding result output by the initial model. Based on the deviation between the decoding result and the sample labels in the training samples, the model parameters of the initial model are adjusted. This process is repeated, training the initial model with a large number of training samples. When the accuracy of the decoding result output by the initial model reaches a preset accuracy, training stops, and the initial model is used as the final decoding model.

[0181] For example, a decoding model can be deployed in an electronic device, which can input the capacitance image into the decoding model each time it acquires it. When the capacitance image includes multiple touch points formed by multiple conductive elements on the input device, a decoding result output by the decoding model can be obtained, which is the first target code of the input device.

[0182] For example, the decoding model can be deployed on a server. After each acquisition of a capacitance image, the electronic device can upload the capacitance image to the server. Upon receiving the capacitance image, the server can input the capacitance image into the decoding model. When the capacitance image includes multiple touch points formed by multiple conductive parts on the input device, a first target code output by the decoding model can be obtained. The server can then send the first target code to the electronic device, thereby achieving decoding and obtaining the target code for the input device.

[0183] It is understandable that during the process of obtaining the first target code through the decoding model, when the capacitive image does not include multiple touch points formed by multiple conductive parts on the input device, the decoding result output by the decoding model does not contain the first target code. At this time, it can be determined that no input device performed a click operation, or the input device is an ordinary input device (such as a finger), and it is not necessary to decode to determine the first target code of the input device.

[0184] In another implementation, the input device can be decoded using template matching to determine a first target code for the input device based on the capacitance image. For example, for each input device, a capacitance image and the input device's code can be pre-obtained, with the capacitance image serving as an image template and the input device's code as a preset code corresponding to the image template. Thus, for multiple input devices, multiple image templates and preset codes corresponding to each image template can be obtained.

[0185] After acquiring a capacitance image, the electronic device can sequentially calculate the similarity between the capacitance image and each image template. When the similarity between a certain image template and the capacitance image is greater than a preset similarity, the image template can be determined as the target image template, and the preset code corresponding to the target image template is used as the first target code for inputting the device. The preset similarity can be a large value, such as 95%, 98%, 99%, or 100%.

[0186] In the process of determining the first target code through template matching, when the capacitive image does not include multiple touch points formed by multiple conductive parts on the input device, the target image template will not exist among the multiple image templates. In this case, it can be determined that no input device performed a click operation, and there is no need to decode to determine the first target code of the input device.

[0187] In another implementation, after acquiring the capacitance image, the host first determines whether the capacitance image includes N first touch points that constitute a preset non-rotationally symmetric shape. If so, a coordinate system is constructed in the capacitance image based on the N first touch points, and multiple decoding regions are determined from the constructed coordinate system based on multiple preset coordinates. Then, for each decoding region, the code element corresponding to the decoding region is determined according to whether there are touch points in the decoding region, and the code elements corresponding to the multiple decoding regions are combined according to the bit indices corresponding to the multiple decoding regions to obtain the first target code.

[0188] like Figure 15 As shown, in the conductive component arrangement surface 411, the side length of each square region is H. The size of the square region can be greater than or equal to the size of the first touch point and the second touch point. The distance between the center point of region 1 and the center point of region 4 is 3H, the distance between the center point of region 1 and the center point of region 13 is 3H, and the distance between the center point of region 4 and the center point of region 13 is...

[0189] After acquiring the capacitance image, the host computer can locate each pixel in the capacitance image whose pixel value is greater than a preset threshold. For each pixel whose pixel value is greater than the preset threshold, a square area with a side length of 3 pixels (both the first and second touch points are square touch points with a side length of 3 pixels, i.e., K) is defined centered on that pixel. If the pixel values ​​of other pixels are all greater than the preset threshold and less than the pixel value of that pixel, then the square area is determined to be either the first or second touch point. Next, for each pixel within the square area, the product between the pixel value and its coordinates is calculated, resulting in a product for each pixel. All product results are summed to obtain a first summation result, and the pixel values ​​of all pixels within the square area are summed to obtain a second summation result. Dividing the first summation result by the second summation result yields the center coordinates of the square area, which are the center coordinates of the touch point. In this way, all touch points in the capacitance image and the center coordinates of each touch point can be determined.

[0190] For example, such as Figure 15 As shown, when constructing a coordinate system based on three first conductive elements, the center point of the first conductive element in region 1 is the origin. The direction from the center point of the first conductive element in region 1 to the center point of the first conductive element in region 4 is the positive direction of the X-axis in the coordinate system, and the direction from the center point of the first conductive element in region 1 to the center point of the first conductive element in region 13 is the negative direction of the Y-axis in the coordinate system. Multiple preset coordinates represent the coordinates of the center points of multiple encoding regions in the coordinate system. These preset coordinates correspond one-to-one with the multiple encoding regions, and the multiple encoding regions correspond one-to-one with the multiple decoding regions. Therefore, the multiple preset coordinates also correspond one-to-one with the multiple decoding regions.

[0191] Based on the relative distances between the regions, it can be determined that the relative distance between the first touch point corresponding to region 1 and the first touch point corresponding to region 4 is approximately or equal to 3H, the relative distance between the first touch point corresponding to region 1 and the first touch point corresponding to region 13 is approximately or equal to 3H, and the relative distance between the first touch point corresponding to region 4 and the first touch point corresponding to region 13 is approximately or equal to 3H.

[0192] After determining that there are multiple touch points in the capacitance image using the above method, if the relative distance between two touch points is... These two touch points are then identified as the two first touch points corresponding to regions 4 and 13, respectively. If the relative distance between any touch point and both of these first touch points is 3H, then that touch point can be identified as the first touch point corresponding to region 1.

[0193] See Figure 17 , Figure 17A schematic diagram of the construction of a coordinate system provided in an embodiment of this application is shown. Figure 17 In the capacitance image 800 shown, each dashed box with a region number in the lower right corner represents a decoding region, and the bit index corresponding to the decoding region is the same as that of the encoding region. Combined with... Figure 15 As shown, touch point 81 is a first touch point formed by the first conductive element in region 1, touch point 82 is a first touch point formed by the first conductive element in region 4, and touch point 83 is a first touch point formed by the first conductive element in region 13.

[0194] After determining that touch point 81 is the first touch point corresponding to region 1, the center of touch point 81 can be used as the origin of the coordinate system. For touch points 82 and 83, a first vector 84 pointing from the center of touch point 81 to the center of touch point 82, and a second vector 85 pointing from the center of touch point 81 to the center of touch point 83 are determined. If the cross product between the first vector 84 and the second vector 85 is negative, then the first vector 84 is located on the X2 axis of the coordinate system, and the second vector 85 is located on the Y2 axis of the coordinate system. Therefore, touch point 82 can be determined as the first touch point corresponding to region 4, and touch point 83 as the first touch point corresponding to region 13.

[0195] Next, the direction from the center point of touch point 81 to the center point of touch point 82 can be determined as the X2 axis direction in the coordinate system, and the direction from the center point of touch point 81 to the center point of touch point 83 can be determined as the Y2 axis direction in the coordinate system. This allows the construction of a Cartesian coordinate system composed of the X2 and Y2 axes. Simultaneously, the angle α between the X1 and X2 axes in the original coordinate system of the capacitive image can be calculated and determined.

[0196] It can be understood that after determining the first vector 84 and the second vector 85, if the cross product between the first vector 84 and the second vector 85 is determined to be positive, then the first vector 84 can be determined to be located on the Y2 axis of the coordinate system, and the second vector 85 can be determined to be located on the X2 axis of the coordinate system. Therefore, touch point 82 can be determined to be the first touch point corresponding to region 13, and touch point 83 can be determined to be the first touch point corresponding to region 4. Subsequently, the direction from the center point of touch point 81 to the center point of touch point 83 can be determined as the X2 axis direction of the coordinate system, and the direction from the center point of touch point 81 to the center point of touch point 82 can be determined as the Y2 axis direction of the coordinate system.

[0197] Combination Figure 15As shown, each encoded region corresponds to a decoded region, and the preset coordinates of the encoded region are the same as the preset coordinates of the corresponding decoded region. In determining the decoded region based on the coordinate system, after the coordinate system is constructed, the preset coordinates of each encoded region can be obtained. Based on these preset coordinates, the position of the center point of the corresponding decoded region is determined from the coordinate system. Then, a square region with a side length of H centered on the center point is determined as the decoded region. The angle between two opposite sides of this square region and the X1 axis is angle α, and the other two sides are perpendicular to the line segment between touch points 81 and 82. Similarly, each other decoded region can be determined.

[0198] Taking decoding regions 1 and 2 as examples, if there is no second touch point in decoding region 1, then the code element corresponding to decoding region 1 can be determined to be 0, and the bit index corresponding to this code element can be determined to be 1. If there is a second touch point 86 in decoding region 2, then the code element corresponding to decoding region 2 can be determined to be 1, and the bit index corresponding to this code element can be determined to be 2. Thus, the first and second bits in the first target code are determined to be 0 and 1, respectively. Similarly, each code element and its corresponding index can be determined. By combining the code elements and bit indices corresponding to all decoding regions, the target code of the input device can be obtained.

[0199] It should be understood that the above are merely illustrative examples, and the specific methods for obtaining the first target code based on capacitance image decoding may include, but are not limited to, the examples above.

[0200] In this embodiment, the controller in the electronic device acquires the capacitive image of the capacitive screen and sends the capacitive image to the host. After receiving the capacitive image, the host decodes the multiple touch points included in the capacitive image to obtain the first target code of the input device and executes the instruction corresponding to the first target code. This not only enables the electronic device to respond to the input device that performs the click operation, but also expands the application scenarios of the capacitive screen and meets the needs of users in some specific scenarios.

[0201] Optionally, the step of sending a capacitive image to the host may include: extracting a region image including multiple touch points from the capacitive image; sending the region image to the host so that the host can decode the region image to obtain a first target code.

[0202] In one implementation, when sending a capacitance image to the host, the controller can only send images of the areas within the capacitance image that include multiple touch points, thereby reducing the amount of data transmitted. For example... Figure 17 As shown, when passing through Figure 15 When the input device shown performs a click operation, multiple touch points can be formed in the capacitive image 800. The multiple touch points include a first touch point and a second touch point, and there may also be other touch points.

[0203] When extracting a region image 80 containing multiple touch points from the capacitive image 800, the upper-left corner coordinate A and lower-right corner coordinate B of the region image 80 can first be determined from the multiple touch points. The x-coordinate of the upper-left corner coordinate A is the minimum x-coordinate among all the x-coordinates of the multiple touch points plus a first preset offset (the first preset offset is a negative value), and the y-coordinate is the maximum y-coordinate among all the y-coordinates of the multiple touch points plus a second preset offset (the second preset offset is a positive value); the x-coordinate of the lower-right corner coordinate B is the maximum x-coordinate among all the x-coordinates of the multiple touch points plus a third preset offset (the third preset offset is a positive value), and the y-coordinate is the minimum y-coordinate among all the y-coordinates of the multiple touch points plus a fourth preset offset (the fourth preset offset is a negative value).

[0204] After determining the coordinates A of the upper left corner and B of the lower right corner of the region image 80, the controller can extract a rectangular region located between the upper left corner coordinates A and the lower right corner coordinates B from the capacitance image 800 to obtain the region image 80. After obtaining the region image 80, the controller can send the region image 80 to the host.

[0205] Correspondingly, the host can decode the multiple touch points included in the region image 80 to obtain the first target code of the input device. In the process of obtaining the first target code based on the region image 80, the host can also use the decoding method described in the aforementioned embodiments to decode the multiple touch points in the region image 80 to obtain the first target code.

[0206] In practical applications, the first preset offset, the second preset offset, the third preset offset, and the fourth preset offset can be reasonably set so that the controller can capture the area image including all the first touch points and the second touch points from the capacitive image regardless of how the input device rotates on the capacitive screen.

[0207] In this embodiment of the application, during the process of sending a capacitance image to the host, the controller extracts a region image from the capacitance image and sends the region image to the host, which can reduce the amount of data during the transmission process and improve the data transmission efficiency.

[0208] Optionally, the step of sending a region image to the host may include:

[0209] The pixel value of the first pixel is modified to one of 1 and 0, and the pixel value of the second pixel is modified to the other of 1 and 0, so as to obtain a region image in the form of a binary map;

[0210] Send a region image in binary form to the host so that the host can decode the region image in binary form to obtain the first target code.

[0211] Here, the first pixel refers to a pixel among multiple touch points included in the region image, and the second pixel refers to all other pixels except the first pixel. For example... Figure 16 As shown, when the pixel value of a pixel in a touch point is greater than or equal to a preset threshold, the first pixel is the pixel in the region image whose pixel value is greater than or equal to the preset threshold, and the second preset pixel is the pixel in the region image whose pixel value is less than the preset threshold.

[0212] For example, before sending the region image to the host, the controller can iterate through each pixel in the region image. If the pixel value (i.e., grayscale value) is greater than or equal to a preset threshold, the pixel is determined to be a pixel within the touch point, and its pixel value can be modified to 1. If the pixel value is less than the preset threshold, the pixel is determined to be a pixel outside the touch point, and its pixel value can be modified to 0. At this point, the grayscale region image can be modified into a binary region image. The controller can then send the binary region image to the host.

[0213] Correspondingly, when decoding a region image based on a binary image, the host can use template matching to decode and obtain the first target code, or it can use a decoding model to identify the first target code from the region image, or it can decode and obtain the first target code by constructing a coordinate system. Specifically, when decoding by constructing a coordinate system, the host can identify pixels with a value of 1 in the binary region image as touch points, and pixels with a value of 0 as non-touch points. Touch points in the region image are determined based on pixels with a value of 1, and then the region image is decoded to obtain the first target code.

[0214] In this embodiment of the application, during the process of sending a region image to the host, the controller first converts the region image into a binary image and then sends the binary image to the host, which can reduce the amount of data during the transmission process and improve data transmission efficiency.

[0215] Optionally, when sending a complete capacitance image to the host, the controller can also convert the capacitance image into a binary capacitance image and send the binary capacitance image to the host. Correspondingly, the host can decode the input device based on the binary capacitance image to obtain the first target code of the input device.

[0216] Optionally, before sending the region image to the host, the method may further include:

[0217] Based on the capacitance image, determine the reference coordinates of the reference point of the input device in the capacitive screen;

[0218] Insert reference coordinates at a preset position in the region image so that the host can extract and use the reference coordinates from the preset position in the region image.

[0219] For example, when through Figure 15 When encoding the input device using the encoding method shown, the reference point on the input device can be the center point of the conductive element arrangement surface 411, and the reference coordinates are the coordinates of the center point of the conductive element arrangement surface 411 on the capacitive screen (i.e., the capacitive image). Of course, the reference point can also be located at other positions on the input device.

[0220] For example, after acquiring the capacitance image, the controller can determine N first touch points from the capacitance image using the aforementioned method, and construct a coordinate system based on the N first touch points. After constructing the coordinate system, a center point C can be determined from this coordinate system. The coordinates of the center point C in this coordinate system are preset (1.5H, -1.5H), and the coordinates (1.5H, -1.5H) are the coordinates of the center point of the conductive component arrangement surface 411 in the coordinate system formed by the X2 axis and the Y2 axis. Then, the controller can determine the coordinates of the center point C in the original coordinate system of the capacitance image, which are the reference coordinates.

[0221] Before sending a region image to the host, the controller can insert reference coordinates at the end (i.e., a preset position) of all data included in the region image. Alternatively, before sending a binary region image to the host, reference coordinates can be inserted at the beginning (i.e., a preset position) of all data included in the binary region image. Correspondingly, after receiving the region image, the host can extract the reference coordinates from the preset positions (beginning or end) of all data included in the region image.

[0222] The host computer can determine the position of a reference point on the input device within the capacitive touchscreen based on reference coordinates. It can then execute corresponding instructions from the input device at that position, such as displaying images or text corresponding to the input device. In practical applications, the controller can also insert reference coordinates at preset positions within the capacitive touchscreen when sending a complete capacitive image to the host computer.

[0223] It should be understood that the preset position may include, but is not limited to, the beginning or the end, and the method of using the reference coordinates may include, but is not limited to, the examples above.

[0224] In this embodiment of the application, before sending the area image to the host, the reference coordinates of multiple touch points are inserted at a preset position in the area image, which makes it easier for the host to determine the actual position of the input device in the capacitive screen through the reference coordinates.

[0225] Optionally, the step of cropping an image of a region including multiple touch points from a capacitance image may include:

[0226] When a capacitive image includes multiple sets of multiple touch points, multiple region images are extracted from the capacitive image; wherein, each of the multiple region images corresponds one-to-one with each of the multiple sets of multiple touch points.

[0227] In one embodiment, during the process of cropping a region image from a capacitance image, the controller can determine the number of input devices present on the capacitive screen based on the capacitance image. When it is determined that there are multiple input devices on the capacitive screen, a region image corresponding to each input device can be cropped from the capacitance image. Each region image includes a set of touch points formed by the corresponding input device. The set of touch points includes multiple touch points formed by multiple conductive elements on the input device.

[0228] For example, in adopting Figure 15 When encoding the input device using the encoding method shown, after acquiring the capacitance image, the controller first uses the method described above to detect and determine each group of first touch points included in the capacitance image, with each group of first touch points corresponding to one input device. For each group of first touch points, a region image corresponding to each group of first input devices can be extracted to obtain a corresponding region image.

[0229] like Figure 17 As shown, for each group of first touch points, a coordinate system is constructed based on the first touch point, and the reference coordinates of the input device (i.e., center point C) are determined from the coordinate system. Then, a square region is determined from the capacitance image based on the center point C. The center point of the square region coincides with the center point C. The bottom and top edges of the square region are parallel to the X1 axis in the original coordinate system of the capacitance image, and the two sides of the square region are parallel to the Y1 axis in the original coordinate system of the capacitance image. The side length of the square region is greater than or equal to the length of the diagonal of the conductive component arrangement surface 411. In this way, an area image including multiple touch points corresponding to the input device can be captured.

[0230] Similarly, when the controller sends the area image corresponding to each input device to the host, it can insert the reference coordinates of the reference point of the input device at a preset position in the area image.

[0231] The above are merely illustrative examples. Methods for extracting the region image corresponding to each input device from the capacitance image may include, but are not limited to, the examples above. This embodiment does not limit these methods.

[0232] In this embodiment of the application, when the capacitive image includes multiple sets of touch points of multiple input devices, the controller captures the area image corresponding to each input device, which enables the host to decode the first target code of each input device based on the area image corresponding to each input device, and execute the instruction corresponding to each input device based on the first target code of each input device to realize the response to each input device.

[0233] Optionally, the step of sending a capacitive image to the host may include: determining the coordinates of each touch point in the capacitive image to obtain multiple touch point coordinates; and sending the multiple touch point coordinates to the host so that the host can decode the first target code based on the multiple touch point coordinates.

[0234] For example, after acquiring the capacitive image, the controller can determine each first touch point and each second touch point included in the capacitive image using the aforementioned method, and determine the coordinates of each first touch point in the capacitive image (i.e., touch point coordinates), and the coordinates of each second touch point in the capacitive image (i.e., touch point coordinates). Then, an array can be created that includes the touch point coordinates of each first touch point and the touch point coordinates of each second touch point, and then the array can be sent to the host.

[0235] Correspondingly, after receiving the array, the host can construct a coordinate system based on the touch point coordinates of each first touch point included in the array, and determine each decoding region from the coordinate system based on the preset coordinates corresponding to each decoding region. Then, for each decoding region, if the touch point coordinates of a second touch point are located within that decoding region, the corresponding code element for that decoding region is determined to be 1; otherwise, if no touch point coordinates are located within that decoding region, the corresponding code element is determined to be 0. In this way, each code element included in the first target encoding of the input device can be determined, thus achieving decoding of the input device.

[0236] In this embodiment of the application, when sending a capacitance image to the host, the coordinates of multiple touch points are extracted from the capacitance image and sent to the host to avoid sending image data to the host, which can reduce the amount of data sent and improve the data sending efficiency.

[0237] Optionally, during the process of sending region images to the host, the controller can also configure a unique identifier for each region image. For example, after acquiring a capacitance image, the controller can assign a unique identifier to each region image of an input device captured from the capacitance image and insert the identifier at a preset position in the region image. Correspondingly, after receiving a region image, the host can extract the identifier from the preset position in the region image and distinguish between multiple received region images based on the identifier.

[0238] Optionally, the method may further include: determining the reference coordinates of the reference point of the input device in the capacitive screen based on the capacitive image, and / or determining the rotation angle of the input device in the capacitive screen; and sending the rotation angle and / or reference coordinates to the host.

[0239] The rotation angle of the input device within the capacitive screen can be the deflection angle between the reference line of the input device and the X-axis in the original coordinate system of the capacitive image. For example... Figure 17 As shown, when using Figure 15 When encoding the input device using the encoding method shown, the reference line of the input device can be the X2 axis of the constructed coordinate system, and the rotation angle is the deflection angle α between the X2 axis and X1.

[0240] like Figure 17 As shown, after acquiring the capacitive image, the controller can determine the touch point coordinates of each touch point from the capacitive image. Simultaneously, it can determine N first touch points from multiple touch points, construct a coordinate system based on these N first touch points in the capacitive image, and then determine the angle α between the X2 axis of the constructed coordinate system and the X1 axis of the original coordinate system of the capacitive image. This angle α is then defined as the rotation angle of the input device within the capacitive screen.

[0241] For example, when sending an array of touch point coordinates of multiple touch points to the host, a rotation angle can be inserted at the first specified position of the array to send the rotation angle to the host. Of course, the controller can also send the rotation angle to the host individually.

[0242] Furthermore, when sending an array of touch point coordinates from multiple touch points to the host, the controller can also insert reference coordinates at a second specified position in the array to send reference coordinates to the host. Of course, the controller can also send reference coordinates to the host independently.

[0243] The host computer can execute commands corresponding to the input device based on reference coordinates and / or rotation angle. For example, when displaying an object corresponding to the input device, the host computer can first determine the reference coordinates from the capacitive screen and display the object at those coordinates. Furthermore, during the display of the object, the rotation angle of the object can be controlled based on the rotation angle. Specific methods of using reference coordinates and rotation angle may include, but are not limited to, the examples above.

[0244] In this embodiment, the controller determines the reference coordinates and / or rotation angle of the input device in the capacitive screen based on the capacitive image, and sends the rotation angle and reference coordinates to the host, which facilitates the host to execute the corresponding instructions of the input device based on the rotation angle and reference coordinates.

[0245] Optionally, the step of sending a capacitive image to the host may include: decoding a first target code based on multiple touch points included in the capacitive image; and sending the first target code to the host.

[0246] In one implementation, after acquiring the capacitance image, the controller can directly decode the multiple touch points included in the capacitance image to obtain the first target code of the input device, and then directly send the first target code to the host. Correspondingly, after receiving the first target code, the host can directly execute the instruction corresponding to the first target code to realize the response to the input device. The process by which the controller determines the first target code based on the capacitance image can refer to the process by which the host determines the first target code based on the capacitance image, and will not be described in detail here.

[0247] After acquiring the capacitive image, the controller can also determine the reference coordinates of the reference point of the input device in the capacitive screen and / or determine the rotation angle of the input device in the capacitive screen based on the capacitive image. During the process of sending the first target code to the host, the controller can simultaneously send the first target code, reference coordinates and rotation angle to the host.

[0248] In this embodiment, the controller obtains the first target code of the input device based on capacitance image recognition and directly sends the first target code to the host, which can reduce the amount of data transmission between the controller and the host and improve data transmission efficiency.

[0249] In this configuration, upon receiving a region image, the host can determine the rotation angle based on the region image when needed. Alternatively, the controller can determine the rotation angle based on the capacitance image, insert the rotation angle at a preset position in the region image, and the host can extract the rotation angle from the preset position in the region image after receiving it.

[0250] The process of an electronic device responding to a sliding operation triggered by an input device on the first interface requires obtaining a second target code from the input device. This second target code consists of a first target code and a sliding identifier. The sliding identifier indicates that a sliding operation has occurred when the input device is applied to the first interface. Whether a sliding operation has occurred when the input device is applied to the first interface can be determined by examining two capacitance images taken at adjacent time points. For example, comparing the position of the touch point in the capacitance image at time t with the position of the touch point in the capacitance image at time t+1, if the positions of the touch points at time t and t+1 are different and the distance between them is greater than a preset distance, then it is considered that a sliding operation has occurred when the input device is applied to the first interface. Thus, the second target code can be obtained from the first target code and the sliding identifier. The host executes the instruction corresponding to the second target code (i.e., receives the instruction to execute the sliding operation), enabling the host to respond specifically to the input device and thus achieve the sliding operation response.

[0251] The process of an electronic device responding to a rotation operation triggered by an input device on a first interface requires obtaining a third target code for the input device. This third target code consists of a first target code and a rotation identifier. The rotation identifier indicates that the input device has rotated relative to the first interface when it is applied. This can be determined by examining two capacitance images taken at adjacent time points. For example, by obtaining the first position coordinates of the touch point in the capacitance image at time t and the second position coordinates of the touch point in the capacitance image at time t+1, and calculating the cross product of the first and second position coordinates, if the cross product is not zero, it indicates that the input device has rotated relative to the first interface. Thus, the third target code can be obtained from the first target code and the rotation identifier. The host executes the instruction corresponding to the third target code (i.e., receives the instruction to perform the rotation operation), enabling the host to respond specifically to the input device and thus achieve the rotation operation response.

[0252] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0253] Figure 18 This application provides a schematic diagram of the structure of an interactive device according to an embodiment of the present application. Figure 18 As shown, the interactive device 1800 includes:

[0254] The first determining module 1810 is used to determine the device identifier of the input device when a first operation triggered by the input device on the first interface is detected.

[0255] The second determining module 1820 is used to determine the target function of the input device configuration based on the device identifier;

[0256] The third determining module 1830 is used to determine the target object from the objects to be started based on the target function; wherein the objects to be started include applications and / or functional modules;

[0257] Object startup module 1840 is used to start the target object.

[0258] In one possible implementation, the third determining module 1830 is specifically used to obtain multiple objects associated with the target function from the objects to be started, and obtain multiple candidate objects; display a first window; wherein the first window includes multiple candidate objects; and in response to a first selection operation of the candidate objects in the first window, determine the candidate object selected by the first selection operation in the first window as the target object.

[0259] In one possible implementation, the interactive device further includes:

[0260] The function execution unit is used to execute the function of the target object in response to a second operation performed by the input device on the first interface.

[0261] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, the function execution unit includes:

[0262] The first execution unit is used to erase the interface content in the first area if the target object is an erasing tool and the second operation is an erasing operation performed by an input device on the first area of ​​the first interface.

[0263] In one possible implementation, the functional execution unit includes:

[0264] The second execution unit is used to select target content if the target object is a selection tool and the second operation is a second selection operation of the input device acting on the second area of ​​the first interface; wherein the target content is the interface content in the second area.

[0265] In one possible implementation, if the application type of the application to which the first interface belongs is a specified application, the second execution unit is further configured to move and / or rotate the target content in response to the adjustment operation of the input device on the target content.

[0266] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function execution unit includes:

[0267] The third execution unit is used to capture the interface content in the third area and obtain a content screenshot if the target object is a screenshot tool and the second operation is an input device acting on the third area of ​​the first interface.

[0268] In one possible implementation, the third execution unit is further configured to display a second window; wherein the second window includes editing options and a save option; in response to a third selection operation of the editing options in the second window, a third window is displayed; wherein the third window includes interface content in a third area that is in an editable state; or, in response to a fourth selection operation of the save option in the second window, a screenshot of the content is saved.

[0269] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function execution unit includes:

[0270] The fourth execution unit is used to display the magnified page content in the fourth area if the target object is a magnifying glass tool; wherein, the fourth area is the area where the input device operates on the first interface.

[0271] In one possible implementation, if the application type of the application to which the first interface belongs is a non-specified application, the function execution unit includes:

[0272] The fifth execution unit is used to display the timer in the fifth area and start the timer if the target object is a timer; wherein the fifth area is any area of ​​the first interface;

[0273] The fifth execution unit is also used to control the timer to stop timing in response to a stop timing operation performed by the input device on the timer.

[0274] In one possible implementation, the interactive device further includes:

[0275] The first configuration unit is configured to display a fourth window in response to a third operation performed by the input device on a preset area in the second interface; wherein the fourth window includes a setting area for the timing duration; in response to a setting operation in the setting area, a target duration is obtained; and the target duration is associated with a target function so that, when the timer is started based on the target function, timing begins based on the target duration.

[0276] In one possible implementation, the functional execution unit includes:

[0277] The sixth execution unit is used to acquire the pressure and / or duration of the input device acting on the first interface; adjust the geometric properties of the target object according to the pressure and / or duration; and execute the functions of the target object after adjusting the geometric properties.

[0278] In one possible implementation, if the target object belongs to an application, the interactive device further includes:

[0279] The second configuration unit is used to display a fifth window in response to a fourth operation performed by the input device on a preset area in the third interface; wherein the fifth window includes application identifiers of multiple applications; and in response to a fifth selection operation of the application identifiers in the fifth window, the application identifier selected by the fifth selection operation in the fifth window is associated with a target function.

[0280] It should be noted that the interactive device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the interactive method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the interactive device and interactive method embodiments provided in the above embodiments belong to the same concept. Therefore, for details not disclosed in the device embodiments of this application, please refer to the above-described interactive method embodiments of this application, which will not be repeated here.

[0281] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0282] Figure 19This application provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 19 As shown, the electronic device 1900 includes a memory 1901 and a processor 1902. The memory 1901 stores executable program code 19011, and the processor 1902 is used to call and execute the executable program code 19011 to perform an interactive method.

[0283] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each module can correspond to a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0284] When each functional module is divided according to its corresponding function, the electronic device may include: a first determining module, a second determining module, a third determining module, and an object initiation module. It should be noted that all relevant content regarding the steps involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0285] The electronic device provided in this embodiment is used to execute the above-described interaction method, and therefore can achieve the same effect as the above-described implementation method.

[0286] When using integrated units, the electronic device may include a processing module and a storage module. The processing module is used to control and manage the operation of the electronic device. The storage module is used to support the execution of relevant program code and data by the electronic device.

[0287] The processing module may be a processor or a controller, which can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0288] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned related method steps to implement an interactive method in the above embodiment.

[0289] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement an interactive method as described in the above embodiment.

[0290] In addition, the electronic device provided in the embodiments of this application may specifically be a chip, component or module. The electronic device may include a connected processor and a memory. The memory is used to store instructions. When the electronic device is running, the processor may call and execute the instructions to make the chip execute an interactive method in the above embodiments.

[0291] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding interaction method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding interaction method provided above, and will not be repeated here.

[0292] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0293] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0294] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An interaction method, characterized in that, The interaction method includes: Upon detecting that an input device triggers a first operation on the first interface, the device identifier of the input device is determined; The target function configured for the input device is determined based on the device identifier; The target object is determined from the objects to be launched based on the target function; wherein, the objects to be launched include applications and / or functional modules; Start the target object.

2. The interaction method according to claim 1, characterized in that, The step of determining the target object from the objects to be started based on the target function includes: Obtain multiple objects associated with the target function from the objects to be started, and obtain multiple candidate objects; Display a first window; wherein the first window includes the plurality of candidate objects; In response to a first selection operation of a candidate object in the first window, the candidate object selected by the first selection operation in the first window is determined as the target object.

3. The interaction method according to claim 1, characterized in that, If the target object belongs to a functional module, after activating the target object, the interaction method further includes: In response to a second operation performed by the input device on the first interface, the function of the target object is executed.

4. The interaction method according to claim 3, characterized in that, If the application type of the application to which the first interface belongs is a specified application, the function of executing the target object includes: If the target object is an erasing tool and the second operation is an erasing operation performed by the input device on a first area of ​​the first interface, the interface content in the first area is erased.

5. The interaction method according to claim 3, characterized in that, The function of executing the target object includes: If the target object is a selection tool and the second operation is a second selection operation performed by the input device on the second area of ​​the first interface, the target content is selected by box selection; wherein, the target content is the interface content in the second area.

6. The interaction method according to claim 5, characterized in that, If the application type of the application to which the first interface belongs is a specified application, after selecting the target content, the interaction method further includes: In response to an adjustment operation performed by the input device on the target content, the target content is moved and / or rotated.

7. The interaction method according to claim 3, characterized in that, If the application type of the application to which the first interface belongs is a non-specified application, the function of executing the target object includes: If the target object is a screenshot tool and the second operation is a screenshot operation performed by the input device on the third area of ​​the first interface, the interface content in the third area is captured to obtain a content screenshot.

8. The interaction method according to claim 7, characterized in that, After capturing the interface content in the third region to obtain a screenshot, the interaction method further includes: A second window is displayed; wherein, the second window includes editing options and a save option; In response to a third selection operation of the editing options in the second window, a third window is displayed; wherein the third window includes interface content in the third area that is in an editable state; or, In response to the fourth selection of the save option in the second window, the content screenshot is saved.

9. The interaction method according to claim 3, characterized in that, If the application type of the application to which the first interface belongs is a non-specified application, the function of executing the target object includes: If the target object is a magnifying glass tool, the magnified page content in the fourth area is displayed; wherein, the fourth area is the area where the input device operates on the first interface.

10. The interaction method according to claim 3, characterized in that, If the application type of the application to which the first interface belongs is a non-specified application, the function of executing the target object includes: If the target object is a timer, the timer is displayed in the fifth area and started to begin timing; wherein, the fifth area is any area of ​​the first interface; After starting the timer to begin timing, the interaction method further includes: In response to the input device's action to stop the timer, the timer is controlled to stop counting.

11. The interaction method according to claim 10, characterized in that, The interaction method also includes: In response to a third operation performed by the input device on a preset area in the second interface, a fourth window is displayed; wherein the fourth window includes a setting area for the timing duration; In response to the setting operation in the setting area, the target duration is obtained; The target duration is associated with the target function so that when the timer is started based on the target function, the timer starts based on the target duration.

12. The interaction method according to claim 1, characterized in that, The function of executing the target object includes: The pressure and / or duration of the input device acting on the first interface are obtained; Adjust the geometric properties of the target object according to the pressure and / or the duration; Perform the function on the target object after adjusting its geometric properties.

13. The interaction method according to claim 1, characterized in that, If the target object belongs to an application, the interaction method further includes: In response to a fourth operation performed by the input device on a preset area in the third interface, a fifth window is displayed; wherein the fifth window includes application identifiers for multiple applications; In response to the fifth selection operation of the application identifier in the fifth window, the application identifier selected in the fifth selection operation in the fifth window is associated with the target function.

14. An interactive device, characterized in that, The interactive device includes: The first determining module is used to determine the device identifier of the input device when a first operation triggered by the input device on the first interface is detected. The second determining module is used to determine the target function configured by the input device based on the device identifier; The third determining module is used to determine the target object from the objects to be launched based on the target function; wherein the objects to be launched include applications and / or functional modules; The object startup module is used to start the target object.

15. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor for calling and running the executable program code from the memory, causing the electronic device to perform the interaction method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the interaction method as described in any one of claims 1 to 13.

Images