Device operation method, electronic device system, device and storage medium

Abstract

The invention discloses a device operation method, an electronic device system, a device and a storage medium, and belongs to the technical field of computers. The method comprises: if a first device and a second device are in a communication connection state and the first device is in a target mode, the first device sends a first message to the second device to instruct the second device to identify the first device as a pen device. Afterwards, the first device obtains a contact position coordinate in the screen sensing area of the first device, and sends a second message carrying the contact position coordinate to the second device to instruct the second device to execute the pen operation according to the contact position coordinate. According to the method, the user can operate the second equipment by taking the first equipment as the pen equipment of the second equipment, so that the operation flexibility of the second equipment is improved, and the use convenience of the user is improved.

Description

[0001] This application is a divisional application of the Chinese patent application with application number 202211668078.1 filed on December 23, 2022, and invention name “Device operation method, electronic device system, device and storage medium”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a device operating method, an electronic device system, a device and a storage medium. Background Art

[0003] With the continuous development of computer technology, capacitive screens are widely used. For example, electronic devices such as mobile phones and tablet computers all have capacitive screens. Users can use their fingers or capacitive pens to conveniently and quickly control electronic devices with capacitive screens. For example, users can use their fingers or capacitive pens to write, draw, etc. on the screen of a tablet computer, or users can use their fingers or capacitive pens to operate the screen content of a tablet computer. However, currently, users can only operate on the electronic device itself using their fingers or capacitive pens, and cannot control other devices. Summary of the invention

[0004] The present application provides a device operation method, an electronic device system, a device and a storage medium, which can use a first device as a pen device of a second device to operate the second device, thereby improving the operation flexibility of the second device. The technical solution is as follows:

[0005] In a first aspect, a device operation method is provided. The method is applied to a first device having a capacitive screen. In the method, if the first device is in a communication connection state with the second device and the first device is in a target mode, the first device sends a first message to the second device to instruct the second device to identify the first device as a pen device. Afterwards, the first device obtains the contact position coordinates in the screen sensing area of ​​the first device, and sends a second message carrying the contact position coordinates to the second device to instruct the second device to perform a pen operation according to the contact position coordinates.

[0006] Target mode refers to a mode in which other devices are operated through the first device. In the target mode, the first device is equivalent to the pen device of the other device. Specifically, in the target mode, the user can use the human body or a capacitive pen to touch the screen sensing area of ​​the first device to implement the corresponding pen operation on the other device.

[0007] The screen sensing area of ​​the first device is an area that can sense contact by a human body or a capacitive stylus in the target mode.

[0008] For example, the contact position coordinates are the coordinates of the contact position between the human body and the screen sensing area of ​​the first device, or the contact position coordinates are the coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the first device.

[0009] The first message is used to indicate that the first device is a pen device. The pen device refers to a device externally connected to the second device and used to implement a stylus function.

[0010] After receiving the first message sent by the first device, the second device can identify the first device as a pen device. After the second device identifies the pen device, it can perform a pen operation according to the contact position coordinates of the pen device. The pen operation refers to an operation that can be implemented with a pen device, for example, the pen operation can be an operation such as writing or drawing, or can be an operation related to the screen content of the second device, which is not limited in this application.

[0011] In the present application, when a first device with a capacitive screen is in a communication connection state with a second device and the first device is in a target mode, the first device can send a first message to the second device so that the second device can identify the first device as a pen device. After that, the user can operate the second device through the first device. Specifically, the user can use the human body or a capacitive pen to touch the screen sensing area of ​​the first device. The first device can obtain the contact position coordinates in the screen sensing area of ​​the first device, and then send the contact position coordinates to the second device, so that the second device can perform a pen operation according to the contact position coordinates. In this way, the user can use the first device as a pen device of the second device to operate the second device, thereby improving the operational flexibility of the second device and improving the user's convenience.

[0012] As an example, if the first device and the second device are in a communication connection state and the first device is in target mode, the operation of the first device sending the first message to the second device may be: if the first device starts the target mode when the first device and the second device are in a communication connection state, the first device sends the first message to the second device.

[0013] In this case, the first device first establishes a communication connection with the second device, and then the first device starts the target mode when the communication connection between the first device and the second device is completed and the communication connection is in the communication connection state. After the first device starts the target mode, it can directly send the first message to the second device that has established the communication connection, so that the second device can identify the first device as a pen device.

[0014] Optionally, if the first device starts the target mode when the first device and the second device are in a communication connection state, the operation of the first device sending the first message to the second device may be: if the first device detects that the first device and the second device have established a communication connection, then the start entry of the target mode is displayed when the first device is not in the target mode; if the first device detects a specified operation for the start entry of the target mode, then the target mode is started and the first message is sent to the second device.

[0015] In the present application, the first device can automatically display the start entry of the target mode after establishing a communication connection with the second device, so that the user can quickly start the target mode through the start entry, and then can conveniently operate other devices through the first device in the target mode.

[0016] As another example, if the first device and the second device are in a communication connection state and the first device is in target mode, the operation of the first device sending the first message to the second device may be: if the first device and the second device establish a communication connection when the first device is in target mode, the first device sends the first message to the second device.

[0017] In this case, the first device starts the target mode first, and then establishes a communication connection with the second device when the first device is in the target mode. After the first device and the second device have established a communication connection, the two are in a communication connection state, and the first device can directly send a first message to the second device that has established a communication connection, so that the second device can identify the first device as a pen device.

[0018] Optionally, if the first device establishes a communication connection with the second device when the first device is in target mode, the operation of the first device sending the first message to the second device may be: if the first device detects a specified operation for a start entry of the target mode, the target mode is started, and a reminder message is displayed when the first device is not communicatively connected to other devices to remind the first device to establish a communication connection with other devices; if the first device detects that the first device establishes a communication connection with the second device, the first message is sent to the second device.

[0019] In the present application, the first device can automatically display a reminder message after starting the target mode to remind the user to connect the first device to other devices for communication, so that the user can conveniently operate other devices through the first device in the target mode.

[0020] Optionally, before the first device sends the second message carrying the contact position coordinates to the second device, the first device may also send a third message carrying the maximum coordinate value of the screen sensing area of ​​the first device to the second device.

[0021] In the present application, after the first device sends the third message to the second device, the second device can determine the mapping relationship between the position coordinates in the screen sensing area of ​​the first device and the position coordinates in the screen operation area of ​​the second device based on the maximum coordinate value of the screen sensing area of ​​the first device and the maximum coordinate value of the screen operation area of ​​the second device carried in the third message. Afterwards, after receiving the second message sent by the first device, the second device can map the contact position coordinates carried in the second message to the target position coordinates in the screen operation area of ​​the second device based on the mapping relationship, and then can perform pen operations based on the target position coordinates.

[0022] Optionally, before the first device sends the second message carrying the contact position coordinates to the second device, the contact position coordinates may be converted into a data format recognizable by the second device.

[0023] In the present application, since the operating systems of the first device and the second device may be different, the data format of the position coordinates that the first device can recognize may be different from the data format of the position coordinates that the second device can recognize. Therefore, before the first device sends the acquired contact position coordinates to the second device, it can convert the contact position coordinates to a data format that the second device can recognize. In this way, the second device can normally recognize the contact position coordinates after receiving the contact position coordinates sent by the first device.

[0024] Optionally, the contact position coordinates are the coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the first device. In this case, before the first device obtains the contact position coordinates in the screen sensing area of ​​the first device, it can also establish a communication connection with the capacitive pen. In this case, after the first device sends the first message to the second device, it can also receive pressure data sent by the capacitive pen, and the pressure data is obtained by the capacitive pen detecting the pen tip pressure of the capacitive pen. Afterwards, the first device can send a second message carrying the contact position coordinates and pressure data to the second device.

[0025] In this application, the first device can not only send the contact position coordinates to the second device, but also send the pressure data sent by the capacitive pen to the second device. In this case, when the second device performs a writing operation based on the contact position coordinates, it can also change the thickness of the writing handwriting according to the change in writing force reflected by the pressure data, so as to improve the user's writing experience.

[0026] In a second aspect, a device operation apparatus is provided, wherein the device operation apparatus has the function of implementing the device operation method in the first aspect. The device operation apparatus comprises at least one module, wherein the at least one module is used to implement the device operation method provided in the first aspect.

[0027] In a third aspect, a device operation apparatus is provided, wherein the structure of the device operation apparatus includes a processor and a memory, wherein the memory is used to store a program that supports the device operation apparatus to execute the device operation method provided in the first aspect, and to store data involved in implementing the device operation method described in the first aspect. The processor is configured to execute the program stored in the memory. The device operation apparatus may also include a communication bus, wherein the communication bus is used to establish a connection between the processor and the memory.

[0028] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the device operation method described in the first aspect.

[0029] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the device operation method described in the first aspect.

[0030] In a sixth aspect, an electronic device system is provided, the electronic device system comprising a first device and a second device, the first device having a capacitive screen;

[0031] The first device is used to: when the first device and the second device are in a communication connection state and the first device is in a target mode, send a first message to the second device;

[0032] The second device is used to: after receiving the first message sent by the first device, identify the first device as a pen device;

[0033] The first device is further used to: obtain the contact position coordinates in the screen sensing area of ​​the first device, and send a second message carrying the contact position coordinates to the second device;

[0034] The second device is further used to: after receiving the second message sent by the first device, perform a pen operation according to the contact position coordinates.

[0035] Optionally, the pen operation is a writing operation, and performing the pen operation according to the contact position coordinates includes:

[0036] Mapping the contact position coordinates to target position coordinates in the screen operation area of ​​the second device according to the maximum coordinate value of the screen sensing area of ​​the first device and the maximum coordinate value of the screen operation area of ​​the second device;

[0037] The writing handwriting is drawn in the screen operation area of ​​the second device according to the target position coordinates.

[0038] In a seventh aspect, an electronic device system is provided, the electronic device system comprising: a first device, a second device and a capacitive stylus, the first device having a capacitive screen;

[0039] The first device is used to: when the first device and the second device are in a communication connection state and the first device is in a target mode, send a first message to the second device;

[0040] The second device is used to: after receiving the first message sent by the first device, identify the first device as a pen device;

[0041] The capacitive stylus is used to: output a voltage pulse to a screen sensing area of ​​the first device through the tip of the capacitive stylus;

[0042] The first device is further used to: determine the contact position coordinates between the tip of the capacitive stylus and the screen sensing area of ​​the first device according to the voltage pulse, and send a second message carrying the contact position coordinates to the second device;

[0043] The second device is further used to: after receiving the second message sent by the first device, perform a pen operation according to the contact position coordinates.

[0044] Optionally, the capacitive stylus is provided with a pressure sensor, and the first device is in a communication connection state with the capacitive stylus;

[0045] The capacitive pen is further used to: detect the pen tip pressure of the capacitive pen through the pressure sensor to obtain pressure sensing data, and send the pressure sensing data to the first device.

[0046] Optionally, the sending a second message carrying the contact position coordinates to the second device includes:

[0047] Sending the second message carrying the contact position coordinates and the pressure sensing data to the second device;

[0048] The pen operation is a writing operation, and performing the pen operation according to the contact position coordinates includes:

[0049] Mapping the contact position coordinates to target position coordinates in the screen operation area of ​​the second device according to the maximum coordinate value of the screen sensing area of ​​the first device and the maximum coordinate value of the screen operation area of ​​the second device;

[0050] The writing handwriting is drawn in the screen operation area of ​​the second device according to the target position coordinates and the pressure sensing data.

[0051] In an eighth aspect, an electronic device is provided, the electronic device having a capacitive screen, a communication module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the following steps:

[0052] Establishing a communication connection with a second device through the communication module;

[0053] Start target mode;

[0054] sending a first message to the second device through the communication module to instruct the second device to identify the electronic device as a pen device;

[0055] Acquiring the contact position coordinates in the screen sensing area of ​​the capacitive screen;

[0056] A second message carrying the contact position coordinates is sent to the second device through the communication module to instruct the second device to perform a pen operation according to the contact position coordinates.

[0057] The technical effects obtained by the above-mentioned second, third, fourth, fifth, sixth, seventh and eighth aspects are similar to the technical effects obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0059] Figure 2 is a block diagram of a software system of an electronic device provided in an embodiment of the present application;

[0060] Figure 3 is a structural schematic diagram of an active capacitive stylus provided in an embodiment of the present application;

[0061] Figure 4 is a schematic diagram of an electronic device system provided by an embodiment of the present application;

[0062] Figure 5 is a flow chart of a device operation method provided by an embodiment of the present application;

[0063] Figure 6 is a schematic diagram of a screen sensing area of ​​a first device provided in an embodiment of the present application;

[0064] Figure 7 It is a schematic diagram of starting a target mode after a first device establishes a communication connection with a second device, provided by an embodiment of the present application;

[0065] Figure 8is a schematic diagram of a screen sensing area of ​​a first device and a screen operating area of ​​a second device provided in an embodiment of the present application;

[0066] Fig. 9 is a schematic diagram of interaction between a first device and a second device provided in an embodiment of the present application;

[0067] Fig.10 is a flow chart of a device operation method provided by an embodiment of the present application;

[0068] Fig.11 It is a schematic diagram of a device operation provided in an embodiment of the present application;

[0069] Fig.12 is another device operation schematic diagram provided in an embodiment of the present application;

[0070] Fig.13 This is another device operation schematic diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0072] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0073] The phrases such as "one embodiment" or "some embodiments" described in the present application mean that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the present application. Therefore, the phrases such as "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments" that appear in the differences in the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. In addition, the terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0074] The electronic device involved in the embodiments of the present application is described below.

[0075] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 1 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0077] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0078] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0079] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0080] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0081] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193 and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0082] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0083] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0084] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0085] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0086] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0087] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0088] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created by the electronic device 100 during use (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0089] The electronic device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D and the application processor.

[0090] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0091] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is an integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0092] Next, the software system of the electronic device 100 will be described.

[0093] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software system of the electronic device 100.

[0094] Figure 2 1 is a block diagram of a software system of an electronic device 100 provided in an embodiment of the present application. Figure 2 The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system includes an application layer, an application framework layer, an Android runtime (Android runtime) and a system layer, as well as a kernel layer.

[0095] The application layer can include a series of applications. Figure 2 As shown, the application may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, short message and other applications.

[0096] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make these data accessible to applications. These data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build the display interface of the application, and the display interface can be composed of one or more views, for example, including a view for displaying a text message notification icon, including a view for displaying text, and including a view for displaying pictures. The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of an icon or scroll bar text, such as a notification of an application running in the background. The notification manager can also be a notification that appears on the screen in the form of a dialog window, such as a text message in the status bar, a reminder sound, an electronic device vibrating, an indicator light flashing, etc.

[0097] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0098] The system library can include multiple functional modules, such as: surface manager, media library, 3D graphics processing library (such as: OpenGL ES), 2D graphics engine (such as: SGL), etc. The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0099] The kernel layer is the layer between hardware and software. The kernel layer includes display driver, camera driver, audio driver, sensor driver, etc.

[0100] The application scenarios involved in the embodiments of the present application are described below.

[0101] The embodiment of the present application is applied to an electronic device having a capacitive screen. The electronic device may be Figure 1 to Figure 2 The electronic device 100 described in the embodiment. For example, the electronic device can be an electronic device with a capacitive screen, such as a capacitive screen mobile phone, a tablet computer, a touch-screen notebook, etc.

[0102] When users operate electronic devices with capacitive screens, they can operate directly with their fingers or with a capacitive pen. Among them, a capacitive pen is a stylus that can be used on a capacitive screen, and can include a passive capacitive pen and an active capacitive pen. Among them, a passive capacitive pen uses the conductive material of the capacitive pen tip to imitate a human finger to complete human-computer interaction. An active capacitive pen completes human-computer interaction by transmitting a signal at the tip of the capacitive pen.

[0103] Next, the touch control principle of the capacitive screen involved in the embodiment of the present application is described:

[0104] The structure of the capacitive screen is mainly to coat a transparent thin film conductor layer on the glass screen, and then add a protective glass outside the conductor layer. The double glass design can protect the conductor layer. Among them, the conductor layer uses one or more layers of nano indium tin oxide (Indium Tin Oxides, ITO) thin film to make the X-axis and Y-axis electrode matrix. The X-axis and Y-axis electrode matrices are coupled to each other to form mutual capacitance.

[0105] For a finger, if it touches the capacitive screen, a coupling capacitor is formed between the finger and the surface of the capacitive screen, causing a slight change in the current, which is equivalent to a change in mutual capacitance. By scanning the X-axis and Y-axis electrode matrix and detecting the change in capacitance at the contact point, the finger contact position can be calculated.

[0106] Similarly, for a passive capacitive pen, if the tip of the passive capacitive pen touches the capacitive screen, a coupling capacitor is formed between the tip of the passive capacitive pen and the surface of the capacitive screen, causing a slight change in the current, which is equivalent to a change in mutual capacitance. By scanning the X-axis and Y-axis electrode matrix and detecting the change in capacitance at the contact point, the contact position of the passive capacitive pen tip can be calculated.

[0107] For active capacitive stylus, such as Figure 3 As shown, the active capacitive stylus may include a power supply 301, a processor 302, a pressure sensor 303, etc. Optionally, the processor 302 may be a micro controller unit (MCU), and of course, the processor 302 may also be a central processing unit (CPU), an application-specific integrated circuit (ASIC), etc., which is not limited in the embodiment of the present application.

[0108] In this case, if the tip of the active capacitive pen touches the capacitive screen, the processor 302 may output a voltage pulse at the tip of the pen. The touch position of the tip of the active capacitive pen may be determined based on the voltage pulse. For example, the touch position of the tip of the active capacitive pen may be obtained by locating the intersection of the capacitance peaks on the X-axis and Y-axis electrode matrices.

[0109] In addition, the pressure sensor 303 can detect the pressure of the pen tip and send the detected pressure data to the processor 302. After receiving the pressure data sent by the pressure sensor 303, the processor 302 can send the pressure data to an electronic device that is connected to the active capacitive pen for communication (i.e., an electronic device to which the capacitive screen that the pen tip of the active capacitive pen contacts belongs), such as a mobile phone, a tablet computer, etc. After receiving the pressure data, the electronic device can sense the writing force of the active capacitive pen, and then can change the thickness of the handwriting according to the change of the writing force, so as to improve the writing experience of the user.

[0110] However, currently users can only operate on the electronic device itself using their fingers or capacitive pens, but cannot control other devices.

[0111] To this end, the embodiment of the present application provides a device operation method, which can use the first device as a pen device of the second device when the first device and the second device are in a communication connection state and the first device is in a target mode, and operate the second device by operating the first device with a finger or a capacitive stylus. In this way, the operation flexibility of the second device can be improved, and the convenience of use for users can be improved.

[0112] The device operating system provided in the embodiment of the present application is described below.

[0113] Figure 4 is a schematic diagram of an electronic device system provided by an embodiment of the present application. Figure 4 , the electronic device system includes: a first device 401 and a second device 402.

[0114] The first device 401 and the second device 402 can communicate via a wired connection or a wireless connection. Figure 4 As shown, the first device 401 and the second device 402 can communicate via a USB connection. Of course, the first device 401 and the second device 402 can also communicate via other connections (including but not limited to Bluetooth connection, WLAN connection, etc.), which is not limited in this embodiment of the present application.

[0115] The first device 401 may be Figure 1 to Figure 2 The electronic device 100 described in the embodiment. The first device 401 can be an electronic device with a capacitive screen, for example, the first device 401 can be a capacitive screen mobile phone, a tablet computer, a touch-screen notebook, etc. The second device 402 can be an electronic device with a capacitive screen, or an electronic device without a capacitive screen, for example, the second device 402 can be a laptop computer, a desktop computer, etc., which is not limited in the embodiment of the present application.

[0116] Optionally, the operating systems of the first device 401 and the second device 402 may be the same or different. For example, the operating system of the first device 401 may be an Android system, and the operating system of the second device 402 may be a Windows system, for example, Figure 4 As shown, the first device 401 may be a tablet computer whose operating system is an Android system, and the second device 402 may be a laptop computer whose operating system is a Windows system.

[0117] In the embodiment of the present application, the first device 401 can be used as a pen device of the second device 402. In this case, the user can use a finger or a capacitive stylus to operate on the first device 401, and thereby operate the second device 402. In this way, the use scenario of the second device 402 can be expanded, the operation flexibility of the second device 402 is improved, and the user's convenience of use is improved. Moreover, in this case, the user can operate the second device 402 with different operating systems or different screen types through operations on the first device 401, thereby improving the user's use experience.

[0118] The device operation method provided in the embodiment of the present application is described below.

[0119] Figure 5 is a flow chart of a device operation method provided by an embodiment of the present application. Figure 5 , the method may include the following steps:

[0120] Step 501: If the first device and the second device are in a communication connection state and the first device is in a target mode, the first device sends a first message to the second device to instruct the second device to identify the first device as a pen device.

[0121] The first device is an electronic device with a capacitive screen. The first device can sense the contact of a human body (mainly a finger) or a capacitive stylus on it. That is, the user can use the human body or the capacitive stylus to touch the screen of the first device to operate the first device.

[0122] The first message is used to indicate that the first device is a pen device. Optionally, the first message may carry a pen identifier, and the pen identifier is used to identify the first device as a pen device. A pen device refers to a device that is externally connected to a second device and is used to implement a stylus function. For example, the pen device may be a human interface device (HID), in which case the pen identifier carried by the first message may be a HID descriptor, and the HID descriptor may be USAGE (Stylus), which is used to describe that the first device is a pen device.

[0123] After receiving the first message sent by the first device, the second device can identify the first device as a pen device. Optionally, after the second device identifies the first device as a pen device, a management option for the pen device will exist in the device manager of the second device. After the second device identifies the pen device, it can perform a pen operation according to the contact position coordinates of the pen device. The pen operation refers to an operation that can be implemented with a pen device. For example, the pen operation can be an operation such as writing or drawing, or an operation related to the screen content of the second device. This embodiment of the present application does not limit this.

[0124] Optionally, the first device and the second device may be communicatively connected via a variety of methods. For example, the first device and the second device may be communicatively connected via USB, Bluetooth, or WLAN, etc. This embodiment of the present application is not limited to this.

[0125] Target mode refers to a mode in which other devices are operated through the first device. When the first device is in target mode, the first device is equivalent to the pen device of other devices, so that other devices can be operated through the first device. Specifically, in target mode, the user can use the human body or a capacitive stylus to touch the screen sensing area of ​​the first device to implement corresponding pen operations on other devices.

[0126] The screen sensing area of ​​the first device is an area that can sense the contact of a human body or a capacitive stylus in the target mode. For example, Figure 6 As shown in FIG. 6( a) of FIG. 6 , the screen sensing area of ​​the first device may be a part of the entire screen area of ​​the first device 601; or Figure 6 As shown in FIG. 5( b ) , the screen sensing area of ​​the first device may be the entire screen area 602 of the first device.

[0127] If the first device and the second device are in a communication connection state and the first device is in target mode, it means that the first device needs to be used as a pen device of the second device to operate the second device. In this case, the first device can send a first message to the second device so that the second device can identify the first device as a pen device.

[0128] Optionally, the operation of step 501 may include the following two situations:

[0129] The first case: if the first device starts the target mode when the first device and the second device are in a communication connection state, the first device sends a first message to the second device.

[0130] In this case, the first device first establishes a communication connection with the second device, and then the first device starts the target mode when the communication connection between the first device and the second device is completed and the communication connection is in the communication connection state. After the first device starts the target mode, it can directly send the first message to the second device that has established the communication connection, so that the second device can identify the first device as a pen device.

[0131] For example, Figure 7 As shown in (a) of FIG. 1 , a user can connect the first device 701 and the second device 702 via a USB cable. In this case, the first device 701 and the second device 702 will establish a communication connection. After the first device 701 and the second device 702 have established a communication connection, the two are in a communication connection state. Then, as an example, Figure 7As shown in FIG. 7( b), the first device 701 may automatically pop up a function selection box 703, in which a target mode option is displayed. The user clicks the target mode option to trigger a start instruction for the target mode. As another example, Figure 7 As shown in FIG. 7( c ), the user can click the target mode button in the pull-down notification bar 704 of the first device 701 to trigger a start instruction for the target mode. After the first device 701 detects the start instruction, it can start the target mode and then send a first message to the second device that has established a communication connection.

[0132] The second situation: if the first device establishes a communication connection with the second device when the first device is in the target mode, the first device sends a first message to the second device.

[0133] In this case, the first device starts the target mode first, and then establishes a communication connection with the second device when the first device is in the target mode. After the first device and the second device have established a communication connection, the two are in a communication connection state, and the first device can directly send a first message to the second device that has established a communication connection, so that the second device can identify the first device as a pen device.

[0134] For example, the user can click the target mode button in the drop-down notification bar of the first device to trigger a start instruction for the target mode. After the first device detects the start instruction, it can start the target mode. Optionally, if the first device does not detect other devices connected to the first device after starting the target mode, the first device can display a reminder message to remind the user that the first device needs to be connected to the other devices. After that, the user can connect the first device and the second device via a USB cable. In this case, the first device and the second device will establish a communication connection. After the first device and the second device have established a communication connection, the two are in a communication connection state, and the first device can send a first message to the second device that has established a communication connection.

[0135] Optionally, the target mode startup entry (including but not limited to the target mode options and target mode buttons described above) in the embodiment of the present application can be set in various interfaces or windows of the first device. For example, the target mode startup entry can be set in a function selection box popped up by the first device, or can be set in a drop-down notification bar of the first device, or can be set in an application interface of a specific application installed on the first device, or can be set in a system setting interface of the first device. Of course, it can also be set in other interfaces or windows, and the embodiment of the present application does not limit this.

[0136] Step 502: The first device obtains the contact position coordinates in the screen sensing area of ​​the first device, and sends a second message carrying the contact position coordinates to the second device to instruct the second device to perform a pen operation according to the contact position coordinates.

[0137] Optionally, the contact position coordinates may be the coordinates of the contact position between the human body (including but not limited to fingers) and the screen sensing area of ​​the first device. Alternatively, the contact position coordinates may be the coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the first device. The capacitive pen may include a passive capacitive pen and an active capacitive pen. Among them, the passive capacitive pen uses the conductive material of the capacitive pen tip to imitate human fingers to complete human-computer interaction. The active capacitive pen completes human-computer interaction by transmitting signals at the capacitive pen tip.

[0138] It should be noted that the contact position coordinates in the embodiment of the present application are determined based on the screen sensing area of ​​the first device as the overall coordinate range.

[0139] The user can use the human body or a capacitive pen to touch the screen sensing area of ​​the first device. In this case, the first device can obtain the coordinates of the contact position between the human body or the capacitive pen and the screen sensing area of ​​the first device, that is, obtain the contact position coordinates in the screen sensing area of ​​the first device.

[0140] For example, the structure of the capacitive screen of the first device is mainly to coat a transparent thin film conductor layer on the glass screen, and then add a protective glass outside the conductor layer. The double glass design can protect the conductor layer. Among them, the conductor layer uses one or more layers of ITO film to make X-axis and Y-axis electrode matrices. The X-axis and Y-axis electrode matrices are coupled to each other to form mutual capacitance.

[0141] For a human body, if the human body contacts the screen sensing area of ​​the capacitive screen of the first device, a coupling capacitor is formed between the human body and the surface of the capacitive screen, causing a slight change in the current, which is equivalent to a change in mutual capacitance. The first device detects the change in capacitance of the contact point by scanning the X-axis and Y-axis electrode matrices, and can calculate the coordinates of the human body contact position in the screen sensing area.

[0142] Similarly, for a passive capacitive pen, if the tip of the passive capacitive pen contacts the screen sensing area of ​​the capacitive screen of the first device, the tip of the passive capacitive pen and the surface of the capacitive screen form a coupling capacitor, causing a slight change in current, which is equivalent to a change in mutual capacitance. The first device can calculate the coordinates of the pen tip contact position in the screen sensing area by scanning the X-axis and Y-axis electrode matrices to detect the change in capacitance at the contact point.

[0143] For an active capacitive pen, a voltage pulse will be output after the pen tip of the active capacitive pen contacts the screen sensing area of ​​the capacitive screen of the first device. The first device can determine the coordinates of the contact position between the pen tip of the active capacitive pen and the screen sensing area of ​​the first device (i.e., the pen tip contact position coordinates in the screen sensing area) based on the voltage pulse. For example, the first device can obtain the pen tip contact position coordinates in the screen sensing area by locating the intersection of the capacitance peaks on the X-axis and Y-axis electrode matrices.

[0144] Optionally, in step 502, after the first device obtains the contact position coordinates in the screen sensing area of ​​the first device, it can first convert the contact position coordinates into a data format recognizable by the second device, and then carry the converted contact position coordinates in the second message to the second device.

[0145] Since the operating systems of the first device and the second device may be different, the data format of the position coordinates that can be recognized by the first device may be different from the data format of the position coordinates that can be recognized by the second device. Therefore, before sending the acquired contact position coordinates to the second device, the first device may convert the contact position coordinates to a data format that can be recognized by the second device. In this way, the second device can normally recognize the contact position coordinates after receiving the contact position coordinates sent by the first device.

[0146] The first device may convert the acquired contact position coordinates by a preset conversion rule. The conversion rule may be preset, for example, the conversion rule may be preset by a technician according to the operating system type of the first device and the operating system type of the second device. Of course, the conversion rule may also be set in other ways, which is not limited in the embodiments of the present application.

[0147] Optionally, if the contact position coordinates in the screen sensing area of ​​the first device are the coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the first device, the first device may also establish a communication connection with the capacitive pen before step 502. In this case, optionally, when the capacitive pen is in a communication connection state with the first device, the capacitive pen may continuously detect the pen tip pressure of the capacitive pen to obtain pressure data, and the pressure data may reflect the writing force of the user when using the capacitive pen. The capacitive pen may send the pressure data to the first device.

[0148] In this way, when the first device and the second device are in a communication connection state and the first device is in target mode, the first device can not only continuously obtain the contact position coordinates in the screen sensing area of ​​the first device, but also continuously obtain the pressure data sent by the capacitive pen.

[0149] In this case, as an example, the first device can not only carry the latest contact position coordinates in the second message and send it to the second device, but also carry the latest pressure data in the second message and send it to the second device. That is to say, in the embodiment of the present application, the second message can only carry the contact position coordinates, or can carry pressure data in addition to the contact position coordinates. Of course, in addition to this, the second message can also carry other data related to the pen operation, which is not limited in the embodiment of the present application.

[0150] Furthermore, after receiving the second message sent by the first device, the second device can perform pen operations according to the contact position coordinates carried in the second message. For example, the second device can perform operations such as writing and drawing according to the contact position coordinates, or can perform operations on the screen content of the second device according to the contact position coordinates.

[0151] In this case, if the second message also carries pressure data, the second device can refer to the pressure data when performing pen operations according to the contact position coordinates. For example, if the second device performs a writing operation according to the contact position coordinates, the second device can also implement functions such as changing the thickness of the writing handwriting according to the change in writing force reflected by the pressure data to improve the user's writing experience.

[0152] Optionally, when the second device performs a pen operation based on the contact position coordinates, it can first map the contact position coordinates to target position coordinates in the screen operation area of ​​the second device, and then perform the pen operation at the target position in the screen operation area of ​​the second device based on the target position coordinates.

[0153] The screen operation area of ​​the second device is an area on the screen of the second device that can be operated by the pen device. For example, the screen operation area of ​​the second device can be the entire screen area of ​​the second device, or a part of the entire screen area of ​​the second device, which is not limited in the embodiments of the present application.

[0154] Since the size of the screen sensing area of ​​the first device may be different from the size of the screen operation area of ​​the second device, after the second device receives the contact position coordinates in the screen sensing area of ​​the first device, it is necessary to first map the contact position coordinates to the target position coordinates in the screen operation area of ​​the second device, and then perform pen operations according to the target position indicated by the target position coordinates in the screen operation area of ​​the second device. For example, you can write, draw, etc. at the target position in the screen operation area of ​​the second device, or you can operate the controls at the target position in the screen operation area of ​​the second device.

[0155] Optionally, the operation of the second device mapping the contact position coordinates to the target position coordinates in the screen operation area of ​​the second device may be: the second device maps the contact position coordinates to the target position coordinates in the screen operation area of ​​the second device based on the maximum coordinate value of the screen sensing area of ​​the first device and the maximum coordinate value of the screen operation area of ​​the second device.

[0156] As an example, the maximum coordinate value of the screen sensing area of ​​the first device may be sent by the first device to the second device. For example, before step 502, the first device may send the maximum coordinate value of the screen sensing area of ​​the first device to the second device.

[0157] The maximum coordinate value of the screen sensing area of ​​the first device can reflect the coordinate range of the screen sensing area of ​​the first device, and the maximum coordinate value of the screen operation area of ​​the second device can reflect the coordinate range of the screen operation area of ​​the second device. Therefore, the mapping relationship between the position coordinates in the screen sensing area of ​​the first device and the position coordinates in the screen operation area of ​​the second device can be determined based on the maximum coordinate value of the screen sensing area of ​​the first device and the maximum coordinate value of the screen operation area of ​​the second device. According to the mapping relationship, the contact position coordinates can be mapped to the target position coordinates in the screen operation area of ​​the second device.

[0158] For example, Figure 8 As shown in Figure (a) in FIG. 1 , the maximum coordinate value of the screen sensing area 801 of the first device is (10,6). Figure 8 As shown in (b) of FIG. 1 , the maximum coordinate value of the screen operation area 802 of the second device is (20, 12), then the mapping relationship between the position coordinates (Xa, Ya) in the screen sensing area 801 of the first device and the position coordinates (Xb, Yb) in the screen operation area 802 of the second device is: Xb = 2Xa, Yb = 2Ya. In this case, Figure 8 As shown, assuming that the contact position coordinates in the screen sensing area 801 of the first device received by the second device are (2, 3), then the second device can map the contact position coordinates (2, 3) to the target position coordinates (4, 6) in the screen operation area 802 of the second device according to the mapping relationship.

[0159] In an embodiment of the present application, when a first device with a capacitive screen is in a communication connection state with a second device and the first device is in target mode, the first device can send a first message to the second device so that the second device can identify the first device as a pen device. After that, the user can operate the second device through the first device. Specifically, the user can use the human body or a capacitive pen to touch the screen sensing area of ​​the first device. The first device can obtain the contact position coordinates in the screen sensing area of ​​the first device, and then send the contact position coordinates to the second device, so that the second device can perform a pen operation according to the contact position coordinates. In this way, the user can use the first device as a pen device of the second device to operate the second device, thereby improving the operational flexibility of the second device and improving the user's convenience.

[0160] For ease of understanding, the following Fig. 9 and Fig.10 The operation method of the above-mentioned device is exemplified.

[0161] See also Fig. 9 , assuming that the target mode is the drawing board mode, the first device is a tablet computer with an Android operating system, a drawing board application and a tablet driver are installed in the tablet computer, and optionally, a touch panel (TP) sensor is also provided in the tablet computer. Assume that the second device is a personal computer (PC) with a Windows operating system. Assume that the user uses an active capacitive stylus to operate the tablet computer, and the active capacitive stylus is provided with a pressure sensor.

[0162] Fig.10 is a flow chart of a device operation method provided by an embodiment of the present application. Fig.10 , the method may include the following steps:

[0163] Step 1001: The user starts the sketchpad mode in the tablet computer through the sketchpad mode start entry.

[0164] For example, after the tablet detects the user's designated operation (including but not limited to click operation, sliding operation, voice operation, gesture operation, somatosensory operation, etc.) on the start entrance of the sketchpad mode, it can start (also called calling up) the sketchpad application. After the sketchpad application is started, the tablet enters the sketchpad mode. In this case, the touch area in the application interface of the sketchpad application is the screen sensing area of ​​the tablet.

[0165] Step 1002: When the tablet computer and the PC have established a communication connection, the tablet driver sends a first message carrying a pen identifier to the PC.

[0166] After receiving the first message, the PC can identify the first device as a pen device according to the pen identifier in the first message.

[0167] Step 1003: The drawing board application sends the maximum coordinate value of the screen sensing area of ​​the tablet computer to the PC.

[0168] The screen sensing area of ​​the tablet computer is the touch area in the application interface of the sketchpad application.

[0169] After the PC receives the maximum coordinate value of the screen sensing area of ​​the tablet computer sent by the tablet computer, it can determine the mapping relationship between the position coordinates in the screen sensing area of ​​the tablet computer and the position coordinates in the screen operation area of ​​the PC based on the maximum coordinate value of the screen sensing area of ​​the tablet computer and the maximum coordinate value of the screen operation area of ​​the PC.

[0170] Step 1004: The user uses a capacitive stylus to write on the screen sensing area of ​​the tablet computer.

[0171] In this case, if Fig. 9 As shown, the tip of the capacitive stylus can emit voltage pulses. In addition, the pressure sensor in the capacitive stylus can detect the tip pressure of the capacitive stylus, obtain pressure sensing data, and send the pressure sensing data to the tablet computer.

[0172] Step 1005: The drawing board application obtains the contact position coordinates in the screen sensing area of ​​the tablet computer, and receives the pressure sensing data sent by the capacitive pen.

[0173] like Fig. 9 As shown, after the tip of the capacitive pen emits a voltage pulse, the TP sensor can detect the contact position of the tip of the capacitive pen according to the voltage pulse to obtain the contact position coordinates. The TP sensor can send the contact position coordinates to the drawing board application. After the drawing board application obtains the contact position coordinates, it can convert the contact position coordinates into a data format that can be recognized by the PC. In addition, the drawing board application can also obtain the pressure sensing data sent by the capacitive pen.

[0174] Step 1006: The drawing board application sends the contact position coordinates and pressure sensing data to the PC.

[0175] After the PC receives the contact position coordinates and the pressure sensing data, it can first map the contact position coordinates to the target position coordinates in the screen operation area of ​​the PC based on the mapping relationship between the position coordinates in the screen sensing area of ​​the tablet computer and the position coordinates in the screen operation area of ​​the PC. Then, the PC can perform corresponding pen operations based on the target position coordinates in the screen operation area of ​​the PC and the pressure sensing data.

[0176] Combine the following Figures 11 to 13 The operation method of the above-mentioned device is exemplified.

[0177] In the embodiment of the present application, the first device can be used as a pen device of the second device to operate the second device. Before operating the second device through the first device, the first device needs to establish a communication connection with the second device, and the first device needs to start the target mode.

[0178] As an example, Fig.11 As shown in FIG. 1( a ), after the user connects the first device 1101 and the second device 1102 via a USB cable, the first device 1101 and the second device 1102 establish a communication connection. In this case, Fig.11 As shown in FIG. 1( b), if the first device 1101 is not in the target mode, the first device 1101 displays a function selection box 1103, and the function selection box 1103 includes a target mode option. Afterwards, if the user performs a selection operation on the target mode option, the first device 1101 starts the target mode, such as Fig.11 As shown in FIG. 5( c ) , the first device 1101 displays a touch interface 1104 in the target mode, and the touch interface 1104 is the screen sensing area of ​​the first device 1101 .

[0179] As another example, Fig.12 As shown in FIG. 10( a), the user performs a selection operation on the target mode button in the pull-down notification bar 1105 of the first device 1101, and the first device 1101 starts the target mode, such as Fig.12 As shown in FIG. 1( b), the first device 1101 displays a touch interface 1104 in the target mode, and the touch interface 1104 is the screen sensing area of ​​the first device 1101. In this case, if the first device 1101 is not connected to other devices for communication, then Fig.12 As shown in FIG. 1( c ), the first device 1101 may display a reminder message 1106 to remind the user to connect the first device 1101 to another device for communication. Fig.12 As shown in Figure (d), after the user connects the first device 1101 and the second device 1102 via a USB cable, the first device 1101 and the second device 1102 establish a communication connection.

[0180] Through the above process, the first device 1101 and the second device 1102 are in a communication connection state and the first device 1101 is in target mode, then the first device 1101 can send a first message to the second device 1102. After receiving the first message, the second device 1101 can identify the first device 1101 as a pen device.

[0181] In this case, if Fig.13As shown, the user uses a capacitive pen to perform writing operations on the screen sensing area 1104 of the first device 1101. The first device 1101 can obtain the contact position coordinates of the tip of the capacitive pen and the screen sensing area 1104 of the first device 1101, and then carry the contact position coordinates in the second message to the second device 1102. After receiving the second message, the second device 1102 can map the contact position coordinates to the target position coordinates in the screen operating area 1107 of the second device 1102 according to the maximum coordinate value of the screen sensing area 1104 of the first device 1101 and the maximum coordinate value of the screen operating area 1107 of the second device 1102, and draw the writing handwriting in the screen operating area 1107 of the second device 1102 according to the target position coordinates. In this way, the screen operating area 1107 of the second device 1102 can display the writing handwriting consistent with the screen sensing area 1104 of the first device 1101. In this way, when a user writes on the screen sensing area 1104 of the first device 1101 with a capacitive stylus, corresponding writing traces may appear on the screen operation area 1107 of the second device 1102, thereby improving user convenience.

[0182] In some embodiments, a pressure sensor may also be provided in the capacitive pen. In this case, the capacitive pen may establish a communication connection with the first device 1101. The capacitive pen may detect the pen tip pressure of the capacitive pen through the pressure sensor, obtain pressure data, and send the pressure data to the first device 1101. In this case, the first device 1101 may carry the contact position coordinates and the pressure data in the second message and send them to the second device 1102. After the second device 1102 receives the second message, the contact position coordinates may be mapped to the target position coordinates in the screen operation area 1107 of the second device 1102 according to the maximum coordinate value of the screen sensing area 1104 of the first device 1101 and the maximum coordinate value of the screen operation area 1107 of the second device 1102, and the writing handwriting may be drawn in the screen operation area 1107 of the second device 1102 according to the target position coordinates and the pressure data. Specifically, the original writing handwriting may be drawn in the screen operation area 1107 of the second device 1102 according to the target position coordinates, and then the line thickness or color depth of the original writing handwriting may be adjusted according to the pressure data.

[0183] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (such as: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (such as: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)).

[0184] The above are optional embodiments provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the technical scope disclosed in the present application shall be included in the protection scope of the present application.

Claims

1. A device operation method, characterized in that: Applied to a tablet computer with a capacitive screen, the method comprises: If the tablet computer is in a communication connection state with the second device and the tablet computer is in a target mode, the tablet computer sends a first message carrying a human-machine interface device HID descriptor to the second device to instruct the second device to identify the tablet computer as a pen device, where the pen device refers to a HID externally connected to the second device for realizing a stylus function, and the target mode refers to a mode for operating other devices through the device, and the tablet computer can be used as a pen device when in the target mode; The tablet computer acquires the contact position coordinates in the screen sensing area of ​​the tablet computer; The tablet computer sends a second message carrying the contact position coordinates to the second device to instruct the second device to perform a pen operation according to the contact position coordinates, where the pen operation refers to an operation that can be implemented in the second device using the pen device.

2. The method according to claim 1, characterized in that If the tablet computer is in a communication connection state with the second device and the tablet computer is in a target mode, the tablet computer sends a first message carrying a human-machine interface device HID descriptor to the second device, including: If the tablet computer starts the target mode when the tablet computer and the second device are in a communication connection state, the tablet computer sends the first message to the second device.

3. The method according to claim 2, characterized in that If the tablet computer starts the target mode when the tablet computer and the second device are in a communication connection state, the tablet computer sends the first message to the second device, including: If the tablet computer detects that the tablet computer has established a communication connection with the second device, displaying a start entry for the target mode when the tablet computer is not in the target mode; If the tablet computer detects a designated operation for the start-up entry of the target mode, the target mode is started and the first message is sent to the second device.

4. The method according to claim 1, characterized in that If the tablet computer is in a communication connection state with the second device and the tablet computer is in a target mode, the tablet computer sends a first message carrying a human-machine interface device HID descriptor to the second device, including: If the tablet computer establishes a communication connection with the second device when the tablet computer is in the target mode, the tablet computer sends the first message to the second device.

5. The method according to claim 4, characterized in that If the tablet computer establishes a communication connection with the second device when the tablet computer is in the target mode, the tablet computer sends the first message to the second device, including: If the tablet computer detects a designated operation for the start-up entry of the target mode, the target mode is started, and a reminder message is displayed when the tablet computer is not in communication connection with other devices, so as to remind the user to connect the tablet computer to other devices in communication; If the tablet computer detects that a communication connection is established between the tablet computer and the second device, the first message is sent to the second device.

6. The method according to any one of claims 1 to 5, characterized in that: Before the tablet computer sends the second message carrying the contact position coordinates to the second device, the method further includes: The tablet computer sends a third message carrying the maximum coordinate value of the screen sensing area of ​​the tablet computer to the second device.

7. The method according to any one of claims 1 to 6, characterized in that: Before the tablet computer sends the second message carrying the contact position coordinates to the second device, the method further includes: The tablet computer converts the contact position coordinates into a data format recognizable by the second device.

8. The method according to any one of claims 1 to 7, characterized in that: The contact position coordinates are coordinates of the contact position between the human body and the screen sensing area of ​​the tablet computer, or the contact position coordinates are coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the tablet computer.

9. The method according to any one of claims 1 to 7, characterized in that: The contact position coordinates are the coordinates of the contact position between the capacitive pen and the screen sensing area of ​​the tablet computer; Before the tablet computer obtains the contact position coordinates in the screen sensing area of ​​the tablet computer, the method further includes: The tablet computer establishes a communication connection with the capacitive stylus; After the tablet computer sends the first message to the second device, the method further includes: The tablet computer receives pressure sensing data sent by the capacitive pen, where the pressure sensing data is obtained by the capacitive pen detecting the pen tip pressure of the capacitive pen; The tablet computer sends a second message carrying the contact position coordinates to the second device, including: The tablet computer sends the second message carrying the contact position coordinates and the pressure sensing data to the second device.

10. The method according to any one of claims 1 to 9, characterized in that: The second device is an electronic device whose operating system is Windows system.

11. An electronic equipment system, characterized in that: The electronic device system comprises a tablet computer and a second device, wherein the tablet computer has a capacitive screen; The tablet computer is used to: send a first message carrying a human-machine interface device HID descriptor to the second device when the tablet computer and the second device are in a communication connection state and the tablet computer is in a target mode, wherein the target mode refers to a mode in which other devices are operated by the tablet computer, and the tablet computer can be used as a pen device when in the target mode; The second device is used to: after receiving the first message sent by the tablet computer, identify the tablet computer as a pen device, where the pen device refers to a HID externally connected to the second device for implementing a stylus function; The tablet computer is further used to: obtain the contact position coordinates in the screen sensing area of ​​the tablet computer, and send a second message carrying the contact position coordinates to the second device; The second device is further configured to: after receiving the second message sent by the tablet computer, perform a pen operation according to the contact position coordinates, where the pen operation refers to an operation that can be implemented in the second device using the pen device.

12. The electronic equipment system according to claim 11, wherein: The pen operation is a writing operation, and performing the pen operation according to the contact position coordinates includes: Mapping the contact position coordinates to target position coordinates in the screen operation area of ​​the second device according to the maximum coordinate value of the screen sensing area of ​​the tablet computer and the maximum coordinate value of the screen operation area of ​​the second device; The writing handwriting is drawn in the screen operation area of ​​the second device according to the target position coordinates.

13. The electronic equipment system according to claim 11 or 12, characterized in that: The second device is an electronic device whose operating system is Windows system.

14. An electronic equipment system, characterized in that: The electronic device system comprises: a tablet computer, a second device and a capacitive stylus, wherein the tablet computer has a capacitive screen; The tablet computer is used to: send a first message carrying a human-machine interface device HID descriptor to the second device when the tablet computer and the second device are in a communication connection state and the tablet computer is in a target mode, wherein the target mode refers to a mode in which other devices are operated by the tablet computer, and the tablet computer can be used as a pen device when in the target mode; The second device is used to: after receiving the first message sent by the tablet computer, identify the tablet computer as a pen device, where the pen device refers to a HID externally connected to the second device for implementing a stylus function; The capacitive stylus is used to: output a voltage pulse to a screen sensing area of ​​the tablet computer through the tip of the capacitive stylus; The tablet computer is further used to: determine the contact position coordinates between the tip of the capacitive pen and the screen sensing area of ​​the tablet computer according to the voltage pulse, and send a second message carrying the contact position coordinates to the second device; The second device is further configured to: after receiving the second message sent by the tablet computer, perform a pen operation according to the contact position coordinates, where the pen operation refers to an operation that can be implemented in the second device using the pen device.

15. The electronic equipment system according to claim 14, characterized in that: The capacitive stylus is provided with a pressure sensor, and the tablet computer is in a communication connection state with the capacitive stylus; The capacitive pen is also used to: detect the pen tip pressure of the capacitive pen through the pressure sensor to obtain pressure sensing data, and send the pressure sensing data to the tablet computer.

16. The electronic equipment system according to claim 15, characterized in that: The sending a second message carrying the contact position coordinates to the second device includes: Sending the second message carrying the contact position coordinates and the pressure sensing data to the second device; The pen operation is a writing operation, and performing the pen operation according to the contact position coordinates includes: Mapping the contact position coordinates to target position coordinates in the screen operation area of ​​the second device according to the maximum coordinate value of the screen sensing area of ​​the tablet computer and the maximum coordinate value of the screen operation area of ​​the second device; The writing handwriting is drawn in the screen operation area of ​​the second device according to the target position coordinates and the pressure sensing data.

17. The method according to any one of claims 14 to 16, characterized in that: The second device is an electronic device whose operating system is Windows system.

18. An electronic device, characterized in that: The electronic device is a tablet computer, which has a capacitive screen, a communication module, a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the following steps: Establishing a communication connection with a second device through the communication module; Starting the target mode, wherein the target mode refers to a mode in which other devices are operated through the device, and the tablet computer can be used as a pen device when in the target mode; Sending a first message carrying a human-machine interface device (HID) descriptor to the second device through the communication module to instruct the second device to identify the tablet computer as a pen device, where the pen device refers to a HID externally connected to the second device for implementing a stylus function; Acquiring the contact position coordinates in the screen sensing area of ​​the capacitive screen; A second message carrying the contact position coordinates is sent to the second device through the communication module to instruct the second device to perform a pen operation according to the contact position coordinates, where the pen operation refers to an operation that can be implemented in the second device using the pen device.

19. A computer device, characterized in that: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by the processor.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 10.

Images