Element display method and terminal equipment

Abstract

The invention relates to the technical field of terminals, and provides an element display method and terminal device.The method comprises the steps that the state of touch equipment is detected; when the state is a suspension state, displaying a projection area of the touch device on a display screen; receiving a first operation; and in response to the first operation, displaying a first element of the control in the projection area. According to the method, the control element selection efficiency can be improved, and the drawing experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method and a terminal device for displaying elements. Background Art

[0002] Digital painting refers to the use of modern digital technology to perform painting, coloring and other painting operations on a computer. Digital painting can use a variety of tools and painting techniques, perform complex layer and effect processing, and can also be modified and adjusted unlimited times without being restricted by time and space. For example, users can quickly perform operations such as "drawing", "color smearing", "color transformation", and "canvas rotation" using digital painting. In digital painting, hand-pen coordination is a common painting state; for example, when a user is painting, one hand is used to switch painting tools, and the other hand holds the pen to paint; however, when the user wants to switch painting tools, he needs to manually call out the toolbar and then click the desired target tool in the toolbar. This way of selecting control elements is inefficient and affects the painting experience. Summary of the invention

[0003] The present application provides a method and a terminal device for displaying elements, which can improve the efficiency of selecting control elements and enhance the drawing experience.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a method for displaying an element is provided, the method comprising: detecting a state of a touch device; when the state is a suspended state, displaying a projection area of ​​the touch device; receiving a first operation; and displaying a first element of a control in the projection area in response to the first operation.

[0006] The above method can be executed by a terminal device including a display screen, or by a module (such as a processor, a chip, or a chip system, etc.) applied to a terminal device including a display screen, or by a logic node, a logic module or software that can realize all or part of the functions of the terminal device. In the present application, when the terminal device detects that the touch device (such as a stylus) is in a suspended state, a projection area is generated; thereafter, the first element in the control is displayed in the projection area according to the first operation from the user (such as a sliding gesture), and the first element in the projection area is the currently selected element. Compared with the user manually operating to call out the control and then manually selecting the element in the control (such as the first element), the present embodiment can complete the selection of the first element through one operation (i.e., the first operation), thereby improving the efficiency of selecting the control element and enhancing the painting experience. In addition, when the user is working, the attention is usually focused on the projection area of ​​the touch device, and the control elements are displayed in the projection area. The user can select the required control element without diverting attention, thereby improving the user experience.

[0007] In a possible implementation manner, displaying the projection area of ​​the touch device includes completely displaying the shape of the projection area, or displaying a mark indicating the projection area.

[0008] In a possible implementation, if the touch device is a stylus pen, the projection area is an area on the display screen that is closer to the stylus tip when the stylus tip is close to the display screen.

[0009] In a possible implementation, the method further includes: in response to the first operation, displaying a second element of the control outside the projection area.

[0010] In some cases, the element displayed for the first time in the projection area may not be the element required by the user. The terminal device displays the second element in the control outside the projection area according to the first operation. The user can browse other elements in the control except the first element (for example, the second element) to determine the required element, thereby improving the user experience.

[0011] In a possible implementation, the method further includes: receiving a second operation; and displaying a second element of the control in the projection area in response to the second operation.

[0012] Compared with the prior art, in which the user needs to select the required element from the candidate elements of the control by clicking the touch device, in this embodiment, the terminal device directly displays the element in the control corresponding to the second operation in the projection area based on the second operation (for example, a gesture operation). The element in the projection area is the currently selected element, and the element switching can be completed without moving the touch device, thereby improving the efficiency of element switching.

[0013] In a possible implementation, the above-mentioned controls include: a color palette control, a brush type control, a brush thickness control, a color absorption control or an erasing control.

[0014] In one possible implementation, when the control is a color wheel control and when the first operation is a gesture operation, in response to the first operation, a first element of the control is displayed in the projection area, including: in response to the gesture operation, displaying the first element of the color wheel control in the projection area, wherein the first element is a color in the color wheel control and the color wheel control is ring-shaped.

[0015] In some cases, a ring-shaped color wheel control can make it convenient for users to rotate the color wheel control to select colors through gesture operations, which is efficient and provides a good user experience.

[0016] In one possible implementation, when the control is a brush type control and when the first operation is a gesture operation, in response to the first operation, a first element of the control is displayed in the projection area, including: in response to the gesture operation, displaying the first element of the brush type control in the projection area, wherein the first element is a brush type in the brush type control, and the brush type control is in the shape of a bar.

[0017] In some designs, a strip-shaped brush type control is convenient for users to slide the brush type control to select a brush through gesture operations (for example, moving left and right or up and down, etc.), and it also conforms to user operating habits and provides a good user experience.

[0018] In one possible implementation, when the control is a brush thickness control and when the first operation is a gesture operation, a first element of the control is displayed in the projection area in response to the first operation, including: in response to the gesture operation, displaying the first element of the brush thickness control in the projection area, wherein the first element is a brush of a thickness in the brush thickness control, and the brush thickness control is in the shape of a strip.

[0019] In some designs, a strip-shaped brush thickness control not only makes it convenient for users to slide the brush thickness control through gesture operations (such as slow sliding operations, etc.) to select the brush thickness, but also conforms to user operating habits and provides a good user experience.

[0020] In one possible implementation, when the control is a color absorbing control and when the first operation is a gesture operation, in response to the first operation, a first element of the control is displayed in the projection area, including: in response to the gesture operation, displaying the first element of the color absorbing control in the projection area, wherein the first element is the color corresponding to the target position on the display screen.

[0021] In one possible implementation, when the control is an erase control and when the first operation is a gesture operation, a first element of the control is displayed in the projection area in response to the first operation, including: in response to the gesture operation, displaying the first element of the erase control in the projection area, wherein the first element is an erase tool in the erase control and the erase control is in the shape of a bar.

[0022] Generally speaking, a strip-shaped eraser control is more in line with user habits, convenient and fast, and provides a good user experience.

[0023] In a possible implementation, the gesture operation is one of a pinch operation, a slide operation, a tap operation, a rotation operation, or a press operation.

[0024] In a possible implementation manner, the first operation and the second operation are operations of the same type.

[0025] In this embodiment, when the first operation and the second operation are operations of the same type, the user can complete the call-out of the control and the switching of elements in the control through the same type of operation, thereby improving the operational convenience and user experience.

[0026] In a possible implementation, the method further includes: detecting a state of the touch device; and when the state is a contact state, performing an operation according to an element in the projection area.

[0027] When the terminal device detects that the touch device is in contact state, it means that the user has finished selecting the element. The terminal device will perform operations (such as drawing operations or coloring operations) according to the elements in the projection area, without the user having to manually confirm the selected elements again, thereby improving the user experience.

[0028] In a possible implementation, the touch device is a stylus pen, and the projection area is an area where the intersection of the axis of the stylus pen in a tilted state and the display interface of the terminal device is located.

[0029] Compared with the vertical projection area that may be blocked by the stylus, in this embodiment, the projection area is located at the intersection of the axis of the inclined stylus and the display interface, which is not easily blocked by the stylus, thereby improving the user experience.

[0030] In a second aspect, an embodiment of the present application provides a terminal device, the terminal device comprising a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program from the memory, so that the terminal device executes any one of the methods in the first aspect.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes any one of the methods in the first aspect.

[0032] In a fourth aspect, an embodiment of the present application provides a computer program product, the computer program product comprising: a computer program code, when the computer program code is executed by a terminal device, the terminal device executes any one of the methods in the first aspect.

[0033] In the fifth aspect, an embodiment of the present application provides a chip system, which includes a processing circuit and a storage medium, in which computer program instructions are stored; when the computer program instructions are executed by the processing circuit, the method described in the first aspect and various possible implementations of the first aspect is implemented.

[0034] Optionally, the processing circuit in the above chip system can be replaced by a processor, and the storage medium can be replaced by a memory. Optionally, the chip system can also include a communication interface, and the communication interface is used to realize communication between the chip system and an external device.

[0035] The beneficial effects of the technical solutions in the second, third, fourth and fifth aspects of the present application are the same as the beneficial effects of the technical solution in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of the structure of a terminal device 100 provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the software structure of a terminal device 100 provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a drawing device provided in an embodiment of the present application;

[0039] Figure 4 A flowchart of a method 400 for displaying elements provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of a projection area provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of another projection area provided in an embodiment of the present application;

[0042] Figure 7 A gesture operation schematic diagram provided in an embodiment of the present application;

[0043] Figure 8 A schematic diagram of a scenario of calling out a control and selecting an element in the control through the same gesture operation provided in an embodiment of the present application;

[0044] Fig. 9 A schematic diagram of another scenario of calling out a control and selecting an element in the control through the same gesture operation provided in an embodiment of the present application;

[0045] Fig.10 A schematic diagram of a scenario of calling out a control and selecting an element in a control through two different gesture operations provided in an embodiment of the present application;

[0046] Fig.11 A schematic diagram of another scenario of calling out a control and selecting an element in the control through two different gesture operations provided in an embodiment of the present application;

[0047] Fig.12A schematic diagram of a scenario of calling out a control and selecting an element in a control through two different types of operations provided in an embodiment of the present application;

[0048] Fig.13 A schematic diagram of a scenario of calling out different elements in a control through different gesture operations provided in an embodiment of the present application;

[0049] Fig.14 A schematic diagram of another scenario of calling out different elements in a control through different gesture operations provided in an embodiment of the present application;

[0050] Fig.15 A schematic diagram of a scenario of calling up a color suction control and selecting an element in the color suction control through two different gesture operations provided in an embodiment of the present application;

[0051] Fig.16 A schematic diagram of a scenario for calling out a control and selecting an element in the control provided in an embodiment of the present application;

[0052] Fig.17 A schematic diagram of another scenario of calling out a control and selecting an element in the control provided in an embodiment of the present application;

[0053] Fig.18 A schematic diagram of a scene of selecting an element in a control provided in an embodiment of the present application;

[0054] Fig.19 A schematic diagram of another scenario of calling out a control and selecting an element in the control provided in an embodiment of the present application;

[0055] Fig. 20 A schematic diagram of a scene showing a projection area identification provided in an embodiment of the present application;

[0056] Fig.21 A schematic diagram of a scene for dynamically displaying a projection area provided in an embodiment of the present application;

[0057] Fig. 22 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0059] Figure 1 A schematic structural diagram of a terminal device 100 is shown.

[0060] The terminal device 100 may include at least one of a digital tablet, a hand-drawn tablet, a handwriting tablet, a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a personal digital assistant (PDA). The embodiment of the present application does not impose any special restrictions on the specific type of the terminal device 100.

[0061] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) connector 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor 180, a button 190, an indicator 191, a subscriber identification module (SIM) card interface 192, Bluetooth 193, a positioning device 194 and a display screen 194, etc.

[0062] It is understandable that the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include Figure 1 More or fewer components, or combining some components, or splitting some components, or different arrangements of components. Figure 1 The components in the present invention may be implemented in hardware, software, or a combination of software and hardware.

[0063] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0064] The processor 110 can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0065] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 may be a cache memory. The memory may store instructions or data that have been used or are frequently used by the processor 110. If the processor 110 needs to use the instruction or data, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 110 may be connected to a touch sensor, an audio module, a wireless communication module, a display, a camera, and other modules through at least one of the above interfaces.

[0067] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0068] The USB connector 130 is an interface that complies with the USB standard specification and can be used to connect the terminal device 100 and peripheral devices. The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display screen 194 and the wireless communication module 160. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0069] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0070] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. In some embodiments, at least some functional modules of the mobile communication module 150 may be arranged in the same device as at least some modules of the processor 110.

[0071] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0072] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), near field communication (NFC), etc., which are applied to the terminal device 100. In some embodiments, the antenna 1 of the terminal device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the terminal device 100 can communicate with the network and other terminal devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), etc.

[0073] The terminal device 100 can realize the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0074] The external memory interface 120 can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external memory card. Or files such as music and videos are transferred from the terminal device to the external memory card.

[0075] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (e.g., a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (e.g., audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional methods or data processing of the terminal device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0076] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0077] The sensor 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., which are used to convert various signals from the outside world into electrical signals or other required forms of information output.

[0078] The key 190 may include a power button, a volume button (for example, a user may adjust the volume of a broadcast message through the volume button), etc. The key 190 may be a mechanical key or a touch key. The terminal device 100 may receive a key input and generate a key signal input related to the user settings and function control of the terminal device 100.

[0079] The indicator 191 can be an indicator light, which can be used to indicate the charging status, power change, and can also be used to indicate information (for example, the indicator light can indicate the broadcast information received by the Bluetooth application software through Bluetooth 193), missed calls, notifications, etc.

[0080] Optionally, the terminal device may further include a SIM card interface 192 for connecting a SIM card. The SIM card can be connected to and separated from the terminal device 100 by inserting it into the SIM card interface 192 or pulling it out from the SIM card interface 192. The terminal device 100 may support one or more SIM card interfaces 192. The SIM card interface 192 may support NanoSIM cards, MicroSIM cards, SIM cards, and the like. Multiple cards may be inserted into the same SIM card interface 192 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 192 may also be compatible with different types of SIM cards. The SIM card interface 192 may also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card may be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0081] The terminal device 100 can realize the camera function through the camera module 193, ISP, video codec, GPU, display screen 194, application processor AP, neural network processor NPU, etc.

[0082] The camera module 193 may be used to collect color image data and depth data of the photographed object, for example, user gestures, etc. The ISP may be used to process the color image data collected by the camera module 193 .

[0083] The display screen 194 is used to display data such as a drawing interface and a user operation interface. For example, the display screen 194 can be used to display information such as the user's drawing content. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or more display screens 194. In some embodiments, the display screen may be a foldable or scroll-like display screen.

[0084] The terminal device 100 may also adopt other architectures, and the embodiment of the present invention may also take the Harmony system as an example to exemplarily illustrate the software architecture of the terminal device 100. It should be understood that the solution provided in this application may also be applied to other types of operating systems such as the Android operating system, the Apple operating system, and the Windows operating system.

[0085] Figure 2 It is a schematic diagram of the software structure of the terminal device 100 according to an embodiment of the present application.

[0086] In some implementations, the Harmony system includes four layers, namely, from bottom to top, a kernel layer, a system basic service layer, a framework layer, and an application layer.

[0087] The Harmony system adopts a multi-kernel design, optionally including the Linux kernel, Hongmeng microkernel and the lightweight IoT operating system kernel (Lite Operating System, LiteOS). Through this design, devices with different device capabilities can choose the appropriate system kernel. The kernel layer also includes the Kernal Abstract Layer, which provides basic kernel capabilities for other Harmony layers, such as process management, thread management, memory management, file system management, network management, and peripheral management.

[0088] The system basic service layer is the core capability set of the Harmony system, which supports the Harmony system to provide services to application services through the framework layer in the scenario of multi-device deployment. This layer can optionally include the following parts:

[0089] System basic capability subsystem set: provides basic capabilities for the operation, scheduling, migration and other operations of distributed applications on multiple devices in the Harmony system, and consists of distributed soft bus, distributed data management and file management, distributed task scheduling, Ark runtime, distributed security and privacy protection, etc. Among them, the Ark runtime provides C / C++ / JavaScript multi-language runtime and basic system class libraries, and also provides runtime for Java programs statically compiled using the Ark compiler (i.e., the part developed in Java language in the application or framework layer).

[0090] Basic software service subsystem set: Provides public and general software services for the Harmony system, and consists of subsystems such as graphics and images, distributed media, distributed AI, multi-mode input, mobile sensing platform (MSDP) & device virtualization technology (DV), event notification, telephone service, and design for product non-functional attributes (DFX). The basic software service subsystem set can be tailored according to the functional granularity based on the deployment environment of different device forms.

[0091] Enhanced software service subsystem set (see Figure 2 Enhanced software part (within the dotted line): Provides differentiated capability-enhanced software services for different devices for the Harmony system, and consists of subsystems such as tablet business software, smart screen business software, car business software, and Internet of Things (IoT) business software. The enhanced software service subsystem set can be tailored at the subsystem granularity according to the deployment environment of different device forms, and each subsystem can be tailored at the functional granularity.

[0092] Harmony Driver Foundation (HDF) and Hardware Abstraction Layer (HAL): are the foundation for the openness of the Harmony system hardware ecosystem, providing hardware capability abstraction to the hardware upward and providing a development framework and operating environment for various peripheral drivers downward.

[0093] Hardware service subsystem set: Provides common and adaptive hardware services for the Harmony system, and consists of hardware service subsystems such as pan-sensor, location, power, USB, biometrics, etc. The hardware service subsystem set can be tailored according to the functional granularity based on the deployment environment of different device forms.

[0094] Proprietary hardware service subsystem (see Figure 2 Proprietary hardware part (with dotted lines in the middle): Provides differentiated hardware services for different devices for the Harmony system, which may include tablet proprietary hardware services, car proprietary hardware services, wearable proprietary hardware services, IoT proprietary hardware services and other subsystems. The proprietary hardware service subsystem can be tailored according to the subsystem granularity, and each subsystem part can be tailored according to the functional granularity.

[0095] The framework layer provides the Harmony system's applications with a user program framework and meta-capability framework in multiple languages, such as Java / C / C++ / JavaScript, as well as a multi-language framework application programming interface (API) that opens up various software and hardware services to the outside world.

[0096] The application layer includes system applications and third-party applications (or extended applications), which can include camera, gallery, calendar, call, drawing application, navigation, WLAN, music, video, short message and other applications. Applications in the Harmony system are built based on atomic ability (AA) and feature ability (FA).

[0097] The following has Figure 1 and Figure 2 Taking the drawing device (eg, electronic drawing board) of the structure shown as an example, combined with the method of displaying elements provided in the present application, the workflow of the software system and the hardware system of the terminal device 100 is exemplarily described.

[0098] Figure 3 An application scenario suitable for this application is shown. Figure 3 As shown in (a), the painting device includes a display screen, and shortcut icons of various applications are displayed on the display interface 301 of the display screen; for example, the user can double-click the shortcut icon of the painting application 302 on the display interface 301, and the painting application 302 will respond to the user's double-click operation and generate a painting application start request; the painting application 302 sends the painting application start request to the event notification management service subsystem of the framework layer through the application layer; the event notification management service subsystem generates an event for starting the painting application according to the received painting application start request, and then sends the event for starting the painting application to the painting driver module of the Hongmeng driver framework layer through the hardware abstraction layer; at this time, the painting driver module generates an instruction for starting the painting application according to the event for starting the painting application, and sends the instruction for starting the painting application to the hardware driver module; the hardware driver module starts the painting application 302 according to the instruction for starting the painting application.

[0099] After the painting application 302 is started, the Figure 3 The drawing interface 303 shown in (b) in the figure, the user can perform various drawing-related operations on the drawing interface 303, such as drawing lines, applying colors, etc.; Figure 3As shown in (b) in the figure, the drawing interface 303 includes a toolbar 304 and a drawing area 305. The user can select the drawing tool (e.g., brush color, brush stroke, etc.) that he wants to use through the toolbar 304. After the user selects the drawing tool, he can draw a graphic on the drawing area 305. When drawing a graphic, the user can call out the required control (e.g., color wheel control or eraser control) through a series of operations (e.g., first operation, second operation, etc.) in the suspended state of the touch device (e.g., stylus) and select the required element (e.g., color, eraser) on the control. For example, the user selects red as the color of the stylus on the color wheel control.

[0100] It should be noted that the following embodiments of the present application will be based on Figure 1 and Figure 2 Taking the terminal device 100 of the structure shown as an example, the method of displaying elements provided by the embodiment of the present application is specifically described in combination with the accompanying drawings and application scenarios. It should be understood that the terminal device to which this method is applicable is not limited to Figure 1 and Figure 2 The hardware and software structure shown in the figure can be adjusted according to the specific application scenario in actual application. Figure 1 and Figure 2 The hardware and software system architecture shown may be modified, and this application does not limit this.

[0101] Next, we still have Figure 1 and Figure 2 Taking the terminal device with the structure shown as an example, the method of displaying elements provided by the present application is described in detail in combination with different embodiments.

[0102] As can be seen from the background technology, in a painting scene, a user can draw various graphics, images, and other contents through a painting device (e.g., a digital tablet). However, when a user wants to switch painting tools, he needs to manually call out the toolbar and then select the desired target tool in the toolbar. This method of selecting control elements is inefficient and affects the painting experience. Therefore, the present application proposes a method for displaying elements, such as Figure 4 As shown; this method can improve the efficiency of selecting control elements and enhance the drawing experience.

[0103] Before introducing the method for displaying elements provided by the present application, a brief description is first given of the execution subject involved in the embodiments of the present application. In the embodiments of the present application, a terminal device including a display screen is used as an example to illustrate the method provided by the present application, but the present application does not limit the execution subject.

[0104] For example, the execution subject involved in the embodiments of the present application may be a terminal device including a display screen, or a chip, a chip system, or a processor applied to a terminal device including a display screen, or a logical node, a logical module, or software that can realize all or part of the functions of the terminal device. For example, the terminal device may be a digital tablet or an electronic drawing board.

[0105] The following embodiments are described by taking the execution subject of the method 400 for displaying elements as a terminal device including a display screen as an example. Figure 4 As shown, it is a flowchart of a method 400 for displaying elements provided in an embodiment of the present application. The method 400 includes S401 to S404. These steps are described in detail below.

[0106] S401, the terminal device detects the state of the touch device.

[0107] The touch control device and the terminal device may communicate with each other via wired or wireless means, which is not limited in this application. For example, the touch control device and the terminal device may communicate with each other via Bluetooth.

[0108] Touch devices include but are not limited to styluses, touch screen pens or styluses.

[0109] The state of the touch device may be a touch-screen state, a floating state or an off-screen state, wherein the touch-screen state may refer to direct contact between the touch device and the display interface of the terminal device, the floating state may refer to the distance between the touch device and the display interface of the terminal device being less than or equal to a preset distance, and the off-screen state may refer to the distance between the touch device and the display interface of the terminal device being greater than a preset distance, wherein the preset distance may be 2 cm or 3 cm, etc.

[0110] The terminal device determines whether to generate a projection area by detecting the current state of the touch device, so as to respond to the operation from the user side to execute the control call. For example, when the terminal device detects that the touch device is in a suspended state, a projection area can be generated near the touch device; when the terminal device receives a first operation from the user side, a control can be displayed according to the first operation.

[0111] S402: When the state is a suspended state, the terminal device displays a projection area of ​​the touch device on the display screen.

[0112] It should be noted that, in a possible implementation, displaying the projection area of ​​the touch device includes fully displaying the shape of the projection area, or displaying a logo indicating the projection area; wherein the terminal device indicates the location of the projection area by displaying the logo. The logo may be a solid dot or an arrow, etc., and can be designed according to specific circumstances in actual applications, and this application does not limit this; for example, if the projection area is a square area, the terminal device can fully display the square projection area, or display a solid dot to represent the square projection area. It should be understood that this application does not limit the specific shape and size of the projection area. This can be understood in conjunction with the following interface embodiments.

[0113] It should also be noted that when the terminal device displays the projection area, the projection area can be fixedly displayed in the target area, or the projection area or the logo of the projection area can be dynamically displayed to remind the user to select an element in the control. For example, the terminal device can display the projection area in a gradual manner at a certain frequency, and refer to the following interface embodiment for details.

[0114] The projection area may refer to an area with a certain area; the projection area may be a regular area or an irregular area, which is not limited in this application. For example, the projection area is a circular area, a square area, or an irregular polygonal area. The projection area may be a two-dimensional area or a three-dimensional area, which is not limited in this application.

[0115] It should be noted that the projection area can be in the vicinity of the touch device. In one example, the projection area can be in the area directly below the tip of the stylus, or in the area of ​​the extension line of the tip of the stylus, or in other areas near the stylus. The present application does not limit this. For a specific understanding, please refer to the interface embodiments below, which will not be repeated here.

[0116] In another example, if the touch device is a stylus, the projection area can be in an area on the display screen that is closer to the stylus tip when the stylus tip is close to the display screen. This application does not limit the specific shape and size of this area, and you can refer to the interface implementation examples below. Among them, "the pen tip is close to the display screen" can be understood as a preset distance between the pen tip and the display screen, and the preset distance can be 1 or 2 cm, etc.; "the area close to the pen tip" can be understood as 0.5 or 1 cm away from the pen tip, etc. This application does not limit the distance value, and it can be adjusted according to the actual situation in actual application.

[0117] It should also be noted that the projection area is also called the selection area. When the terminal device calls out a certain control, an element in the control will appear in the projection area as a preset element; the element in the projection area is the element currently to be executed by the touch device by the terminal device by default. In other words, the element moved to the projection area is the selected element.

[0118] For example, when the terminal device brings up the color wheel control, the black color in the color wheel control is in the projection area by default. At this time, the color of the tip of the stylus is black. When the user needs to select other colors in the color wheel, the other colors in the color wheel (for example, green) need to be moved or rotated to the projection area; when the green color in the color wheel is in the projection area, the color of the tip of the stylus changes from the original color to green, that is, the current color of the stylus is green.

[0119] In a possible embodiment, the touch device is a stylus pen, and the projection area is an area where the intersection of the axis of the stylus pen in a tilted state and the display interface of the terminal device is located.

[0120] The stylus in an inclined state may mean that the stylus forms a certain angle with the display interface of the terminal device, for example, 30° or 45°. The stylus in a vertical state may mean that the stylus forms a 90° angle with the display interface of the terminal device.

[0121] For example, when the user holds the stylus pen, the stylus pen may form an angle of 30° or 60° with the display interface of the terminal device.

[0122] Normally, the terminal device displays the projection area at the intersection of the axis of the tilted stylus and the display interface of the terminal device. For example, the terminal device displays the projection area in a circular area with the intersection as the center and R as the radius. This method of displaying the projection area can ensure that the projection area is within the user's field of view and avoid the stylus from blocking the projection area, so that the user can select elements in the control within the field of view. For details, please refer to the following Figure 6 The relevant description of the interface embodiment shown will not be repeated here.

[0123] It can be seen that compared to the vertical projection area that may be blocked by the stylus, in this embodiment, the projection area is located at the intersection of the axis of the tilted stylus and the display interface, which is not easily blocked by the stylus, thereby improving the user experience.

[0124] From the above examples, it can be seen that the projection area of ​​the touch device and the display area of ​​the elements in the control will overlap. When the element in the control enters the projection area, the terminal device will determine that the element is selected and will control the touch device to perform operations according to the element. This area sharing allows users to quickly switch between elements near their line of sight, which not only allows users to focus more on the painting itself, but also has high switching efficiency and good user experience.

[0125] When the terminal device detects that the touch device is in a suspended state, the terminal device will display the generated projection area on the display interface; after the user sees the projection area, he can call out the target control (for example, the color wheel control) within the field of view through some gestures and other operations, and select the required target element (for example, the target color) through the projection area without distracting his attention.

[0126] S403: The terminal device receives a first operation.

[0127] Among them, the first operation may refer to a gesture operation or a voice operation; the gesture operation may refer to an operation in which the user calls out a control through a preset gesture, for example, the user calls out a palette control through a sliding operation; the voice operation may refer to an operation in which the user calls out a control through a voice command, for example, the user calls out a palette control by voice inputting the voice command "start palette control".

[0128] In an optional implementation, the first operation is a gesture operation, and the second operation is a gesture operation, wherein the gesture operation includes but is not limited to one of a pinch operation, a slide operation, a tap operation, a rotation operation, or a press operation.

[0129] For example, the pinch operation may refer to pinching with two fingers or pinching with three or more fingers, and this application does not limit this.

[0130] A sliding operation may refer to an operation performed by sliding a finger up, down, left, or right at a preset distance (for example, 2 cm) from the display interface. A sliding operation may also be a slow sliding operation or a fast sliding operation; for example, a user uses four fingers of the left hand except the thumb to quickly slide upward to invoke a brush-type control. It should be noted that a slow sliding operation may be divided into slow sliding upward, slow sliding downward, slow sliding to the left, or slow sliding to the right; similarly, a fast sliding operation may be divided into fast sliding upward, fast sliding downward, fast sliding to the left, or fast sliding to the right. In an optional implementation, slow sliding operations and fast sliding operations may have different control display forms. For example, during a slow sliding operation, the target control is displayed to the user, and during a fast sliding operation, the complete target control is hidden, and only the elements in the projection area are displayed. Please refer to the following text Fig.17 and Fig.18 The related interface embodiments will not be described in detail here.

[0131] Among them, the slow sliding operation or the fast sliding operation can be defined by the speed at which the hand leaves the screen. For example, the speed at which the hand leaves the screen for the slow sliding operation can be less than 100vp / (or 300pixels / s); the speed at which the hand leaves the screen for the fast sliding operation can be greater than or equal to 100vp / s (or 300pixels / s).

[0132] The tapping operation may refer to an operation performed by tapping a display interface of a terminal device with a finger, for example, a user clicks the display interface with the index finger of the left hand to call an erase control.

[0133] The rotation operation may refer to an operation performed by rotating a finger clockwise or counterclockwise on the display interface of the terminal device. For example, a user may rotate the index finger of the left hand clockwise on the display interface to invoke a color wheel control.

[0134] A pressing operation may refer to an operation performed by long pressing or short pressing a finger on the display interface of a terminal device for a preset time (e.g., 2 seconds or 3 seconds, etc.). For example, a user may call up the brush thickness control by long pressing the index finger of his left hand on the above display interface for 3 seconds.

[0135] It should be noted that the above gesture operation may refer to an operation in which the user touches the display interface, or may refer to an operation in which the user is a certain distance away from the display interface (eg, 1 cm or 2 cm, etc.).

[0136] Exemplarily, the user triggers the target control (eg, the color wheel control) through a gesture operation on the display interface of the terminal device; the terminal device determines the control to be called by receiving the first operation (eg, the gesture operation) from the user.

[0137] Compared with the operation of the combination shortcut key method, in this application, the user can quickly switch elements through some gesture operations (such as pinching operation or sliding operation, etc.), with high switching efficiency and good user experience.

[0138] S404: The terminal device displays a first element of the control in the projection area in response to the first operation.

[0139] It should be noted that the above controls include but are not limited to: color palette control, brush type control, brush thickness control, color absorption control or erase control. Among them, the color palette control (also known as color control) is used to provide various colors (such as red or black, etc.) that can be selected by the touch device (such as a stylus); the brush type control is used to provide various types of brushes (such as fountain pen brushes) that can be selected by the touch device; the brush thickness control is used to provide various brush thicknesses (such as brushes of No. 3 or No. 5, etc.); the color absorption control is used to provide the touch device with various colors corresponding to a specific position on the screen (such as purple or pink, etc.); the erase control is used to provide various erasing tools (such as a round eraser or a square eraser, etc.) that can be selected by the touch device.

[0140] Among them, the first element is an element in the control. It should be noted that usually a control (for example, a color palette control) includes at least two elements. The meaning of the first element is different in different controls. For example, in a color palette control, elements refer to different colors, such as the first element can refer to red or green, etc.; in a brush type control, elements refer to brushes of different types, for example, the first element can refer to a pencil brush, a rigid brush, or a brush, etc.

[0141] In an optional embodiment, the first element generally refers to an element displayed in the projection area after the pointer appears on the display interface of the terminal device.

[0142] After the terminal device detects that the touch device is in a suspended state, it will generate a projection area near the touch device. At this time, the actual properties of the current touch device (for example, color properties and / or brush properties) will be presented in the projection area. For example, the actual properties of the current touch device in the projection area are black markers. At this time, the actual properties of the current touch device in the projection area can be presented as the brush stroke effect of a black marker with a transparency of 50%.

[0143] As an example, when the user sees the projection area, if the first element displayed in the projection area happens to be the element the user wants to select, the user does not need to do anything and can directly touch the screen with the stylus; if the first element displayed in the projection area is not the element the user wants to select, the user will use various operations (such as gesture operations, etc.) to switch the element (such as the first element) of the target control displayed in the projection area; accordingly, after receiving the first operation from the user, the terminal device will first determine the control that the user wants to trigger based on the first operation, and then display the first element pre-set in the control in the projection area.

[0144] For example, the first operation received by the terminal device is a quick swipe downward operation; the terminal device brings up the color wheel control based on the quick swipe downward operation, and the black color in the color wheel control (the first element here refers to black) will appear in the projection area by default. At this time, as the user swipes downward, the color wheel control will also rotate clockwise, and other elements in the color wheel control (such as red or orange, etc.) will slide to the initial projection area of ​​the stylus as the hand moves. When an element moves to the projection area, the element will be enlarged (such as red); in an optional implementation method, when the stylus is lifted by the user and is in an off-screen state, the color of the stylus is switched to the color that appears in the current projection area, and the user no longer needs to select a color from the color wheel control.

[0145] It should be noted that when an element moves into the projection area, the element may be enlarged or have a stroke, or may have other display forms to distinguish between the selected element in the projection area and the unselected element outside the projection area.

[0146] It can be seen that when the terminal device detects that the touch device (for example, a stylus) is in a suspended state, a projection area is generated; thereafter, the first element in the control is displayed in the projection area according to the first operation from the user (for example, a sliding gesture), and the first element in the projection area is the currently selected element. Compared with the user manually calling out the control and then manually selecting an element in the control (for example, the first element), this embodiment can complete the selection of the first element through one operation (that is, the first operation), thereby improving the efficiency of selecting control elements and enhancing the drawing experience. In addition, when working, the user's attention is usually focused on the projection area of ​​the touch device. When the control elements are displayed in the projection area, the user can select the desired control element without diverting attention, thereby improving the user experience.

[0147] In one possible implementation, when the control is a color wheel control and when the first operation is a gesture operation, the terminal device displays the first element of the control in the projection area in response to the first operation, including: the terminal device displays the first element of the color wheel control in the projection area in response to the gesture operation, wherein the first element is a color in the color wheel control and the color wheel control is ring-shaped.

[0148] For the description of the gesture operation, please refer to the description in S403 above, which will not be repeated here.

[0149] The color wheel control includes at least two colors, for example, the color wheel control includes red, blue and green; the first element may be red or blue or green.

[0150] It should be noted that the color wheel control can be in a ring shape, a square shape or other shapes, and this application does not limit this. When the color wheel control is in a ring shape, the user can rotate the color wheel control to select a color through gesture operation, which is convenient and fast, and the user experience is good.

[0151] In one example, the gesture operation is a sliding operation, which can be a slow sliding operation or a fast sliding operation; for example, the user triggers the color wheel control by sliding slowly to the left (or right or up or down) or sliding quickly to the left (or right or up or down); after the terminal device detects the slow sliding operation to the left (or right or up or down) or the fast sliding operation to the left (or right or up or down) from the user side through the display screen, the color wheel control is called in response to the operation, and the first element (for example, red) in the color wheel control is displayed in the projection area. Of course, the gesture operation that triggers the color wheel control can also be a pinching operation or a tapping operation, etc., which is not limited in this application.

[0152] In another possible implementation, when the control is a brush type control and when the first operation is a gesture operation, the terminal device displays a first element of the control in the projection area in response to the first operation, including: the terminal device displays the first element of the brush type control in the projection area in response to the gesture operation, wherein the first element is a brush type in the brush type control, and the brush type control is in the shape of a bar.

[0153] The related description of the gesture operation can also refer to the related description in the previous S403, which will not be repeated here.

[0154] The brush type control includes at least two brush types. For example, the brush type control includes a pencil brush, a fountain pen brush, and a brush brush. The first element may be a pencil brush, a fountain pen brush, or a brush brush.

[0155] The strip shape may refer to a vertical (or horizontal) rectangle or column shape, wherein the size and dimension (two-dimensional or three-dimensional) of the rectangle or column shape are not limited in this application.

[0156] It should be noted that the brush type control can be in a strip shape, a ring shape or other shapes, and this application does not limit this. When the brush type control is in a strip shape, the user can move the brush type control left or right or up and down through gestures to select a brush, which conforms to the user's operating habits and provides a good user experience.

[0157] In one example, the gesture operation is a slow swipe operation, which can be a left (or right or up or down) operation; for example, the user triggers a brush type control by a left slow swipe operation; after the terminal device detects the left slow swipe operation from the user side through the display screen, the brush type control is called in response to the left slow swipe operation, and the first element in the brush type control (for example, a brush) is displayed in the projection area.

[0158] In another possible implementation, when the control is a brush thickness control and when the first operation is a gesture operation, the terminal device displays a first element of the control in the projection area in response to the first operation, including: the terminal device displays the first element of the brush thickness control in the projection area in response to the gesture operation, wherein the first element is a brush of a certain thickness in the brush thickness control, and the brush thickness control is in the shape of a strip.

[0159] The related description of the gesture operation can also refer to the related description in the previous S403, which will not be repeated here.

[0160] The brush thickness control includes at least two brush thicknesses. The brush thickness can usually be represented by the font size. For example, the brush thickness control includes a brush of thickness 1, a brush of thickness 3, and a brush of thickness 5; the first element can be a brush of thickness 1, a brush of thickness 3, or a brush of thickness 5.

[0161] It should be noted that the brush thickness control can be in a strip shape, or in a square shape or other shapes, and this application does not limit this. When the brush thickness control is in a strip shape, the user can also use gestures to move the brush thickness control left and right or up and down to select the brush thickness, which is in line with the user's operating habits and is convenient and fast.

[0162] In one example, the gesture operation is a slow swipe operation, which can be a left (or right or up or down) operation; for example, the user triggers the brush thickness control by a slow swipe upward operation; after the terminal device detects the slow swipe upward operation from the user side through the display screen, the brush thickness control is called in response to the slow swipe upward operation, and the first element in the brush thickness control (for example, thickness 3) is displayed in the projection area.

[0163] In one possible implementation, when the control is a color-absorbing control and when the first operation is a gesture operation, the terminal device displays a first element of the control in the projection area in response to the first operation, including: the terminal device displays the first element of the color-absorbing control in the projection area in response to the gesture operation, wherein the first element is the color corresponding to the target position on the display screen.

[0164] The related description of the gesture operation can also refer to the related description in the previous S403, which will not be repeated here.

[0165] The target position may refer to a certain area in the picture, which can be understood by referring to the following interface embodiments. The user can absorb the color of a certain area in the picture by triggering the color absorption control.

[0166] In one example, the gesture operation is a pinch operation, for example, the user triggers the color absorption control through a pinch operation; after the terminal device detects the pinch operation from the user side through the display screen, the color absorption control is called in response to the pinch operation to absorb the color of the target position, and the successfully absorbed color (for example, the first element) will be displayed in the projection area.

[0167] In one possible implementation, when the control is an erasing control and when the first operation is a gesture operation, the terminal device displays a first element of the control in the projection area in response to the first operation, including: the terminal device displays the first element of the erasing control in the projection area in response to the gesture operation, wherein the first element is an erasing tool in the erasing control, and the erasing control is in the shape of a bar.

[0168] The related description of the gesture operation can also refer to the related description in the previous S403, which will not be repeated here.

[0169] The erasing control includes at least two erasing tools, for example, the erasing control includes a square eraser, an elliptical eraser and a circular eraser; the first element may be a square eraser, an elliptical eraser or a circular eraser.

[0170] It should be noted that the erase control can be in a strip shape, or in a square shape or other shapes, which is not limited in this application. When the erase control is in a strip shape, it is usually more in line with the user's usage habits, convenient and fast, and has a good user experience.

[0171] In one example, the gesture operation is a tapping operation, which may be a single-click operation or a double-click operation, or other tapping operations; for example, the user triggers the erase control through a single-click operation; after the terminal device detects the tapping operation from the user side through the display screen, the erase control is called in response to the tapping operation, and the first element in the erase control (for example, a circular eraser) is displayed in the projection area.

[0172] In a possible implementation, the method further includes: the terminal device displays a second element of the control outside the projection area in response to the first operation.

[0173] The second element is an element other than the first element in the control; the second element usually refers to an element displayed outside the projection area after the control appears on the display interface of the terminal device.

[0174] It should be noted that, similar to the first element, the meaning of the second element is different in different controls; for example, in a color palette control, the first element may refer to black, and the second element may refer to red; in a brush type control, the first element may refer to a fountain pen brush, and the second element may refer to a brush brush.

[0175] After the terminal device receives the first operation from the user, it will first determine the control that the user wants to call based on the first operation, and then display the first element pre-set in the control in the projection area, and display other elements in the control except the first element (for example, the second element, the third element, etc.) in the area outside the projection area, so that the user can view the status of each element in the control.

[0176] For example, control A is a color palette control, which includes N elements, where the first element is black, the second element is red, ..., the Nth element is purple, and N is a positive integer; when the user calls the color palette control through a first operation (for example, a quick slide operation), black will appear in the projection area by default, and red, ..., purple will appear in the area outside the projection area; if the user wants to choose purple as the color of the stylus, the purple can be moved to the projection area through a gesture. At this time, the color corresponding to the projection area (for example, purple) will be different from the states of other elements. For example, the purple corresponding to the projection area can be in an enlarged state or have a stroke state to distinguish it from other elements; when the user lifts the stylus or the tip of the stylus touches the drawing board, the color of the stylus switches to purple.

[0177] In some cases, the element displayed for the first time in the projection area may not be the element required by the user. The terminal device displays the second element in the control outside the projection area according to the first operation. The user can browse other elements in the control except the first element (for example, the second element) to determine the required element, thereby improving the user experience.

[0178] In a possible implementation, the method further includes: the terminal device receiving a second operation; and the terminal device displaying a second element of the control in the projection area in response to the second operation.

[0179] The second operation may refer to a gesture operation or a voice operation; for example, the user brings up the brush type control by swiping right; or for another example, the user brings up the brush type control by voice inputting the voice command "start brush type tool".

[0180] It should be noted that the first operation and the second operation may be operations of the same type or operations of different types.

[0181] Example 1, in some embodiments, the first operation and the second operation are operations of the same type.

[0182] When the first operation and the second operation are operations of the same type, the first operation and the second operation can be the same or different. When the first operation and the second operation are the same, the user can complete the control call and the selection of elements in the control through the same operation; when the first operation and the second operation are different, the user can first call out the target control through the first operation, and then select the target element in the target control through the second operation. You can refer to the following interface embodiment for understanding, which will not be repeated here.

[0183] In one example, the terminal device calls out an element in the control according to the second operation, and at this time, the first operation and the second operation are the same operation. Exemplarily, the terminal device receives the second operation and displays the second element in the control in the projection area according to the second operation, wherein the second operation may refer to a gesture operation, and the second element may refer to a callable element corresponding to the second operation; the user may directly call an element in the control (e.g., the second element) through a gesture operation, and the terminal device may directly display the element in the control corresponding to a gesture operation (e.g., the second element) in the projection area without displaying the element in the entire control, so as to improve the user experience.

[0184] For example, a quick swipe downward (or a quick swipe upward) corresponds to the previous color (or the next color) in the color palette control, such as red (or blue); at this time, other colors in the color palette control and the color palette base will not appear; after the terminal device receives information related to the quick swipe downward (or the quick swipe upward) operation, it will respond to the quick swipe downward (or the quick swipe upward) operation and display the red (or blue) in the color palette control in the projection area. At this time, the color of the tip of the stylus pen switches to red (or blue), and the user can continue painting with the red (or blue) stylus pen.

[0185] When the first operation and the second operation are operations of the same type, the user can complete the call-out of the control and the switching of elements in the control through the same type of operation, thereby improving the operation convenience and user experience.

[0186] Example 2: In some embodiments, the first operation and the second operation are operations of different types. For example, the first operation is a voice operation, and the second operation is a gesture operation.

[0187] In another example, the terminal device first calls out the target control according to the first operation, and then selects the target element from the target control according to the second operation. At this time, the first operation and the second operation can be different operations of the same type or different operations of different types.

[0188] Example 1, the first operation is a quick swipe operation in a gesture operation (for example, a quick swipe upwards or a quick swipe downwards), and the first operation is used to call out a brush type control, and the second operation is a slow swipe operation in a gesture operation (for example, a slow swipe upwards or a slow swipe downwards), and the second operation is used for the user to select the required brush from the brush type control; for example, the terminal device calls out the brush type control in response to a quick swipe operation from the user side (for example, a quick swipe to the left or a quick swipe to the right), and then determines the pen type brush (for example, a fountain pen type brush) selected by the user from the brush type control in response to a slow swipe operation from the user side (for example, a slow swipe to the left or a slow swipe to the right).

[0189] Example 2: The terminal device calls out the brush type control in response to a quick swipe operation from the user side, and then responds to a voice operation from the user side. For example, the user inputs the voice command "Select a pen-type brush" through voice. At this time, the terminal device determines that the user wants to select a pen-type brush from the brush type control based on the voice command input by the user. Finally, the terminal device will display the pen-type brush in the projection area in response to the user's currently selected "Pen-type brush".

[0190] Compared with the prior art, in which the user needs to select the required element from the candidate elements of the control by clicking on the touch device, in this embodiment, the terminal device directly displays the element in the control corresponding to the second operation in the projection area based on the second operation (for example, a gesture operation). The element in the projection area is the currently selected element, and the element switching can be completed without moving the touch device, thereby improving the efficiency of element switching. In addition, when there is no need to switch elements, there is no need to display too many control elements to reduce page occlusion.

[0191] In a possible implementation, the method further includes: the terminal device detecting a state of the touch device; when the state is a contact state, the terminal device performing an operation according to an element in the projection area.

[0192] As can be seen from the above description, the state of the touch device can be a touch screen state, a floating state or an off-screen state, wherein the touch screen state can also be called a contact state; when the terminal device detects that the touch device is in a floating state, it will respond to operations from the user side and display controls and elements in the controls. For details, please refer to the relevant description of the above method 400, which will not be repeated here. When the terminal device detects that the touch device is in an off-screen state, it will not respond to the relevant operations called by the user side controls.

[0193] When the terminal device detects that the touch device is in contact, it means that the user has selected an element in the control, and the terminal device can control the touch device to perform operations according to the elements in the projection area. For example, the user uses the first operation to bring up the color wheel control, and uses the second operation to move the red color in the color wheel control to the projection area generated by the touch device. When the user presses the stylus pen to the display interface of the terminal device, the terminal device will detect that the stylus pen is in contact; at this time, the brush color drawn by the stylus pen on the display interface is red.

[0194] In this embodiment, when the terminal device detects that the touch device is in a contact state, it means that the user has finished selecting the element. The terminal device will perform an operation (such as a drawing operation or a coloring operation) based on the element in the projection area, without the user having to manually confirm the selected element again, thereby improving the user experience.

[0195] The method 400 and possible implementations are introduced in detail above. The method 400 and possible implementations are further described below by taking a touch device as a stylus as an example and combining interface embodiments of different application scenarios.

[0196] As can be seen from the relevant description about the projection area in method 400, the shape of the projection area can be circular or other shapes; Figure 5 A schematic diagram of a projection area suitable for the present application is shown. Figure 5 As shown in (a) in FIG. 1 , when the terminal device detects that the stylus pen 501 is in a suspended state, a projection area is generated on the drawing interface 303 of the display interface, and the projection area is a circular projection area 502; for example, Figure 5 As shown in (b), when the terminal device detects that the stylus 501 is in a suspended state, the terminal device can also generate a square projection area 503 on the drawing interface 303 of the display interface; subsequently, the elements in the control called out by the user through gestures and other operations can be selected through the circular projection area 502 or the square projection area 503.

[0197] It should be noted that, in some embodiments, the projection area generated by the terminal device on the drawing interface 303 can be located on the extension line of the axis of the stylus pen 501, or at the projection position directly below the tip of the stylus pen 501 (i.e., the vertical projection position of the tip on the display interface), or at other areas near the tip of the stylus pen 501 (for example, the area within the user's field of view), and this application does not limit this.

[0198] For example, Figure 6As shown in (a) of FIG. 3 , the projection area 502 generated by the terminal device on the drawing interface 303 is located at the position where the extension line 601 of the axis of the stylus 501 intersects with the reference plane, where the reference plane represents the drawing interface 303; for example, Figure 6 As shown in (b), the projection area 502 generated by the terminal device on the drawing interface 303 is located at the projection position directly below the tip of the stylus 501, wherein the projection position directly below may refer to the position where the tip of the stylus 501 intersects the reference plane through the vertical projection line 602, and the vertical projection line 602 forms a 90° angle with the reference plane.

[0199] Figure 7 Some gesture operations applicable to this application are shown; Figure 7 As shown in (a) of FIG. 5 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call out the desired control (e.g., a brush type control) by single-clicking (or double-clicking) 701 with the index finger on the drawing interface 303; for another example, Figure 7 As shown in (b) in FIG. 5 , when the terminal device generates a projection area 502 near the position of the stylus 501 , the user can call out the desired control (eg, the color wheel control) in the drawing interface 303 by fast sliding or slow sliding 702 .

[0200] Figure 8 and Fig. 9 A scenario of calling out a control and selecting an element in the control through the same gesture operation is shown; for example, Figure 8 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can use the upward quick sliding operation 801 in the drawing interface 303 to bring up the color palette control 802, and the color palette control 802 is in a ring shape. At this time, the black color 803 in the color palette control 802 is displayed in the projection area 502 by default, that is, at this time, the color of the tip of the stylus 501 is black 803; Figure 8 As shown in (b) in FIG. 5 , the user rotates the color wheel control 802 by quickly sliding upward 801 to move the red color 804 in the color wheel control 802 to the projection area 502 . At this time, the color of the tip of the stylus 501 is red 804 .

[0201] For example, Fig. 9 As shown in (a) in FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call out the brush type control 902 by slowly sliding rightward 901 in the drawing interface 303. The brush type control 902 is in the form of a horizontal strip. At this time, the pencil-type brush 903 in the brush type control 902 is displayed in the projection area 502 by default. That is, at this time, the brush that can be drawn by the tip of the stylus 501 is the pencil-type brush 903; Fig. 9 As shown in (b) in FIG. 5 , the user moves the brush type control 902 by slowly sliding rightward 901 , and moves the crayon-type brush 904 in the brush type control 902 to the projection area 502 . At this time, the tip of the stylus 501 is the crayon-type brush 904 .

[0202] Fig.10 A scenario of calling out a color suction control and selecting an element in the color suction control through two different gesture operations is shown; Fig.10 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call up the color palette control by a three-finger single-click operation 1001 in the drawing interface 303. At this time, the white color 1002 in the color palette control is displayed in the projection area 502 by default, that is, the color of the tip of the stylus 501 is white 1002; Fig.10 As shown in (b) in FIG. 1 , the user uses the three-finger pinch operation 1003 to absorb brown 1004 from the eye position of the portrait where the tip of the stylus pen 501 is located, that is, the three-finger pinch operation 1003 moves the brown 1004 to the projection area 502. At this time, the color absorbed by the tip of the stylus pen 501 is brown 1004. When the three fingers are pinched to a certain distance, the stylus pen 501 will gradually become a color picking control as the distance of the finger pinching increases. At this time, the color picking point of the color picking control is the color of the corresponding position of the stylus pen 501. When the stylus pen 501 moves, the color on the color picking control also changes accordingly. When the three fingers continue to pinch inward, the color picking control gradually shrinks to the color of the current position of the stylus pen 501. At this time, if the stylus pen 501 is lifted, the color attribute of the stylus pen 501 becomes the color attribute corresponding to the current position (that is, the color of the tip of the stylus pen 501 is switched to the color corresponding to the current position).

[0203] Fig.11 A scenario of calling out a control and selecting an element in the control through two different types of operations (eg, a gesture operation combined with a voice operation) is shown; Fig.11 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus pen 501, the user can call up the color palette control 802 through the voice input method 1101 (for example, voice input of the voice command "color palette control") in the drawing interface 303. At this time, the black color 803 in the color palette control 802 is displayed in the projection area 502 by default, that is, the color of the tip of the stylus pen 501 is black 803; Fig.11 As shown in (b), the user rotates the color wheel control 802 by quickly sliding upward 1102 to move the green 1103 in the color wheel control 802 to the projection area 502. Optionally, at this time, the green 1103 corresponding to the projection area can be enlarged, and the color of the stylus 501 when the tip of the stylus touches the drawing board is green 1103.

[0204] Fig.12 and Fig.13 A scenario of calling out different elements in a control through different gesture operations is shown; for example, Fig.12 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can quickly slide left with four fingers in the drawing interface 303 to adjust the purple color 1202 in the color palette control. At this time, the purple color 1202 in the color palette control will be displayed in the projection area 502, that is, the color of the tip of the stylus 501 is purple 1202; Fig.12 As shown in (b), the user adjusts the pink color 1204 in the color palette control by a three-finger single-click operation 1203. At this time, the pink color 1204 in the color palette control will be displayed in the projection area 502. Optionally, at this time, the pink color 1204 corresponding to the projection area can be enlarged, and the color of the stylus 501 when the tip of the stylus touches the drawing board is pink 1204.

[0205] For example, Fig.13 As shown in (a) in FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call out the liquid brush 1302 in the stylus control by a two-finger pinch operation 1301 in the drawing interface 303. At this time, the liquid brush 1302 in the stylus control will be displayed in the projection area 502, that is, the stylus type of the tip of the stylus 501 is the liquid brush 1302; Fig.13 As shown in (b) in FIG. 1 , the user double-clicks the index finger to operate 1303 to call out the eraser 1304 in the erase control. At this time, the eraser 1304 in the erase control will be displayed in the projection area 502. At this time, the tip of the stylus 501 is the eraser.

[0206] Fig.14 and Fig.15 A scenario of calling out a control and selecting an element in the control through two different gesture operations is shown; for example, Fig.14 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can use a two-finger pinch operation 1401 in the drawing interface 303 to bring up the color palette control 802. At this time, the black color 803 in the color palette control 802 is displayed in the projection area 502 by default, that is, the color of the tip of the stylus 501 is black 803; Fig.14 As shown in (b) in FIG. 5 , the user rotates the color wheel control 802 by quickly sliding downward 1402 to move the yellow 1403 in the color wheel control 802 to the projection area 502 . At this time, the color of the tip of the stylus 501 is yellow 1403 .

[0207] For example, Fig.15As shown in (a) in FIG. 1 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call out the brush type control 902 by slowly sliding leftward 1501 in the drawing interface 303. At this time, the pencil-type brush 903 in the brush type control 902 is displayed in the projection area 502 by default, that is, the brush that can be drawn by the tip of the stylus 501 is the pencil-type brush 903; Fig.15 As shown in (b) in FIG. 5 , the user moves the brush type control 902 by quickly sliding rightward 1502 , and moves the crayon-type brush 904 in the brush type control 902 to the projection area 502 . At this time, the tip of the stylus 501 is the crayon-type brush 904 .

[0208] Fig.16 and Fig.17 The following diagram shows the scenes of fast sliding operation and slow sliding operation to call out the control and select the elements in the control; for example, Fig.16 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can quickly slide up (or quickly slide down) in the drawing interface 303 to bring up the color palette control 802. At this time, the black color 803 in the color palette control 802 is displayed in the projection area 502 by default, that is, the color of the tip of the stylus 501 is black 803; Fig.16 As shown in (b), the user continues to quickly swipe upward (or downward) to move the next yellow 1603 (or previous purple) in the color wheel control 802 to the projection area 502. At this time, the yellow 1603 (or purple) in the projection area 502 is in an enlarged selected state; when the stylus 501 is lifted (for example, the stylus is in an off-screen state), the color of the tip of the stylus 501 is yellow 1603 (or purple).

[0209] For example, Fig.17 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call up the color palette control 802 by slowly sliding upward (or slowly sliding downward) in the drawing interface 303. At this time, the color palette control 802 will be displayed on the drawing interface 303, and the black color 803 in the color palette control 802 is displayed in the projection area 502 by default, that is, the color of the tip of the stylus 501 is black 803; Fig.17As shown in (b), the user continues to rotate the color wheel control 802 by slowly swiping upward (or slowly swiping downward) operation 1702. During the rotation of the color wheel control 802, each color in the color wheel control 802 has the opportunity to move to the projection area 502; optionally, when the slow swiping upward (or slowly swiping downward) operation is stopped or the stylus is lifted up by the user and is in an off-screen state, the color in the color wheel control 802 that moves to the projection area 502 is the selected color, for example, purple 1703 moves to the projection area 502, at this time, purple 1703 in the projection area 502 is in an enlarged selected state; when the stylus 501 is lifted up (for example, the stylus is in an off-screen state), the color of the tip of the stylus 501 is purple 1703.

[0210] It should be noted that when the upward slow slide (or downward slow slide) operation is stopped or the stylus is lifted up by the user and is off-screen, and when the color wheel control 802 is about to stop rotating, no element may just move to the projection area 502. At this time, the terminal device will determine which color in the color wheel control 802 is closest to the projection area 502, and move the closest color to the projection area 502. For example, Fig.18 As shown in (a) in FIG. 1 , when the upward slow slide (or downward slow slide) operation is stopped or the stylus is lifted by the user and is off-screen, the color wheel control 802 stops rotating. When the color wheel control 802 is about to stop, if the yellow 1603 and the green 1802 move to the vicinity of the projection area 502, the terminal device calculates the distance d1 of the yellow 1603 from the center of the projection area 502 and the distance d2 of the green 1802 from the center of the projection area 502, and compares the sizes of d1 and d2; when d1 is greater than d2, the terminal device moves the yellow 1603 to the center of the projection area 502. When d1 moves to the projection area, the yellow 1603 in the projection area 502 is in an enlarged selected state; when the stylus 501 is lifted up (for example, the stylus is in an off-screen state), the color of the tip of the stylus 501 is yellow 1603; or, when d1 is less than or equal to d2, the terminal device moves the green 1802 into the projection area; at this time, the yellow 1603 in the projection area 502 is in an enlarged selected state, and when the stylus 501 is lifted up (for example, the stylus is in an off-screen state), the color of the tip of the stylus 501 is green 1802.

[0211] For example, Fig.18As shown in (b) in FIG. 1 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can quickly slide up (or quickly slide down) in the drawing interface 303 to bring up the color wheel control 802. At this time, the complete color wheel control 802 will appear implicitly on the drawing interface 303 in the form of a certain transparency (for example, 90% transparency), and the black 803 in the color wheel control 802 is displayed in the projection area 502 by default. Note that the black 803 displayed in the projection area 502 at this time can be seen by the user; the user continues to rotate the color wheel control 802 by slowly sliding up (or slowly sliding down) in the operation 1803; Since the color wheel control is displayed implicitly, the user may not see the rotation, but the color wheel control 802 is in a rotating state at this time; while the color wheel control 802 is rotating, each color in the color wheel control 802 has the opportunity to move to the projection area 502; optionally, when the upward swipe (or downward swipe) operation is stopped or the stylus is lifted up by the user and is in an off-screen state, the color in the color wheel control 802 that moves to the projection area 502 is the selected color, for example, blue 1804 moves to the projection area 502 and is in an enlarged selected state; when the stylus 501 is lifted up (for example, the stylus is in an off-screen state), the color of the tip of the stylus 501 is blue 1804.

[0212] Fig.19 Another scenario of calling out a control and selecting an element in the control is shown; Fig.19 As shown in (a) of FIG. 3 , when the terminal device generates a projection area 502 near the position of the stylus 501, the user can call out the brush type control 902 by slow sliding (for example, slow sliding to the left or slow sliding to the right) operation 1901 in the drawing interface 303. At this time, the rigid pen type brush 903 in the brush type control 902 is displayed in the projection area 502 by default; Fig.19 As shown in (b), the user can directly select the desired brush type on the brush type control 902 with the tip of the stylus 501. Of course, the user can also continue to move the brush type control 902 by slow sliding (for example, slow sliding to the left or slow sliding to the right) operation 1901 to move the desired brush type to the projection area 502.

[0213] For example, the user directly selects the rigid pencil type brush 1902 on the brush type control 902 with the tip of the stylus 501, and the rigid pencil type brush 1902 appears in the projection area 502; when the stylus 501 is lifted (for example, the stylus is in an off-screen state) or the stylus 501 is directly put down to paint from a suspended state, the brush type of the stylus 501 tip is the rigid pencil type brush 2002. It can be seen that, compared with multiple slow sliding operations 1901 to select the desired brush type, in this embodiment, the user directly selects the desired brush type on the brush type control 902 with the tip of the stylus 501, which is more efficient and provides a better user experience.

[0214] Fig. 20 A scene showing a projection area identification is shown; Fig. 20 As shown in (a) in the figure, when the terminal device generates a projection area 502 near the position of the stylus 501, at this time, the entire projection area 502 is not displayed as shown in the above embodiment, but an arrow mark 2001 that can indicate the position of the projection area 502 is displayed. After the arrow mark 2001 appears, the user can call out the brush type control to be called by slowly sliding 2002 to the left in the drawing interface 303. At this time, the projection area 502 can be fully displayed, and the rigid pen type brush 2003 in the brush type control is displayed in the projection area 502 by default, so that the user can select the required brush type, such as Fig. 20 As shown in (b) in .

[0215] Fig.21 A scene of dynamically displaying a projection area is shown; Fig.21 As shown in (a) of FIG. 2 , when the terminal device generates a projection area 502 near the position of the stylus 501, the projection area 502 can be dynamically displayed by fading in and out at a certain period or frequency; optionally, when the user triggers the control through gesture operation 2102, the fading in and out effect of the projection area 502 disappears, and the projection area 502 is directly displayed in a preset color, so that the user can select an element in the control in the projection area 502. For example, when the user triggers the control through gesture operation, the projection area 502 can be dynamically displayed by fading in and out at a certain period or frequency; Fig.21 As shown in (a), the terminal device dynamically displays the projection area 502 with a period T; at the initial moment T0=0, the stylus is in a suspended state, and the projection area 502 is displayed by a preset color 2101. As time goes by, the projection area 502 gradually becomes lighter, as shown in FIG. Fig.21 As shown in (b), the color of the projection area 502 gradually becomes lighter from T0=0 to T2=T, that is, a gradual disappearance and appearance effect occurs within a period T, and the cycle repeats until a gesture operation of the user triggering the control is detected, and the effect disappears.

[0216] It should be noted that although the article uses fast or slow sliding operation examples to call the color palette control and select elements in the color palette control, it should be understood that in actual applications, designers can use other gesture operations (such as pinching, tapping, rotating or pressing operations, etc.) to realize the calling of the color palette control and element selection according to actual conditions. Due to limited space, this application only uses sliding operations to illustrate the calling of the color palette control and element selection. Similarly, the calling of the brush type control, brush thickness control, color suction control or erase control and element selection are not limited to the operations given in the previous embodiments, and other gesture operations (such as selecting sliding operations, tapping operations, rotating operations or pressing operations, etc.) can also be used to realize the calling of the color palette control and element selection. This application does not limit this.

[0217] It should also be noted that when a touch device (for example, a stylus) is in a suspended state, if an element in the target control moves into the projection area, the element is selected, and the element is selected regardless of whether the touch device is lifted up; for example, when the stylus is lifted up (for example, the stylus is in an off-screen state) or when the stylus is directly used to draw from a suspended state, the tip of the stylus has been switched to the target element.

[0218] It should be noted that the method of displaying elements provided in this application is not limited to painting scenes, but can also be applied to scenes such as electronic note-taking software, memos, and e-books. This application only introduces the method of displaying elements using painting scenes as an example, and should not be understood as a limitation on the application scenarios of this application.

[0219] The above describes in detail an example of a method for displaying elements provided by the present application. It is understandable that, in order to implement the above functions, the terminal device includes a hardware structure and / or software module corresponding to each function. It should be easily realized by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application. The present application can divide the method of displaying elements into functional units according to the above method examples, such as dividing each function into each functional unit, or integrating two or more functions into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0220] Fig. 22A schematic diagram of the structure of a terminal device provided by the present application is shown. Fig. 22 The dashed line indicates that the unit or the module is optional. The terminal device 2200 can be used to implement the method described in the above method embodiment. The terminal device 2200 can be a server or a chip (system).

[0221] The terminal device 2200 includes one or more processors 2201, which can support the terminal device 2200 to implement Figure 4 The method in the corresponding method embodiment. The processor 2201 can be a general-purpose processor or a special-purpose processor. For example, the processor 2201 can be a central processing unit (CPU). The CPU can be used to control the terminal device 2200, execute software programs, and process data of software programs. The terminal device 2200 can also include a communication unit 2205 to implement signal input (reception) and output (transmission).

[0222] The terminal device 2200 may be a chip (system) including a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory to implement the methods shown in the above embodiments.

[0223] The communication unit 2205 may be an input and / or output circuit of the chip (system), or the communication unit 2205 may be a communication interface of the chip (system), and the chip (system) may be a component of the terminal device 2200 .

[0224] For another example, the communication unit 2205 may be a transceiver of the terminal device 2200, or the communication unit 2205 may be a transceiver circuit of the terminal device 2200. The terminal device 2200 may include one or more memories 2202, on which a program 2204 is stored. The program 2204 can be run by the processor 2201 to generate instructions 2203, so that the processor 2201 performs the method described in the above method embodiment according to the instructions 2203. Optionally, data may also be stored in the memory 2202. Optionally, the processor 2201 may also read the data stored in the memory 2202, and the data may be stored at the same storage address as the program 2204, or the data may be stored at a different storage address from the program 2204.

[0225] The processor 2201 and the memory 2202 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of a terminal device. The specific manner in which the processor 2201 executes the method for displaying elements may refer to the relevant description in the method embodiment.

[0226] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or a software-based instruction in the processor 2201. The processor 2201 can be a CPU, a digital signal processor (DSP), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0227] The present application also provides a computer program product, which implements the method of any method embodiment of the present application when executed by the processor 2201. The computer program product can be stored in the memory 2202, for example, a program 2204, which is finally converted into an executable target file that can be executed by the processor 2201 after preprocessing, compilation, assembly, and linking.

[0228] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method of any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0229] The computer-readable storage medium is, for example, a memory 2202. The memory 2202 may be a volatile memory or a nonvolatile memory, or the memory 2202 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SynchLinkDRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM).

[0230] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0231] In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not executed. The device embodiments described above are merely schematic, and the splitting of the units is only a logical function splitting. There may be other splitting methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0232] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

[0233] Finally, the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for displaying an element, characterized in that: Applied to a terminal device including a display screen, the method comprises: Detect the state of the touch device; When the state is a suspended state, displaying a projection area of ​​the touch device on the display screen; receiving a first operation; In response to the first operation, a first element of a control is displayed in the projection area.

2. The method according to claim 1, characterized in that The method further comprises: In response to the first operation, a second element of the control is displayed outside the projection area.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a second operation; In response to the second operation, a second element of the control is displayed in the projection area.

4. The method according to any one of claims 1 to 3, characterized in that The controls include: Color wheel controls, brush type controls, brush thickness controls, color pickup controls, or erasing controls.

5. The method according to claim 4, characterized in that When the control is the color wheel control, and when the first operation is a gesture operation, the displaying the first element of the control in the projection area in response to the first operation includes: In response to the gesture operation, the first element of the color wheel control is displayed in the projection area, wherein the first element is a color in the color wheel control, and the color wheel control is in a ring shape.

6. The method according to claim 4, characterized in that When the control is the brush type control, and when the first operation is a gesture operation, displaying the first element of the control in the projection area in response to the first operation includes: In response to the gesture operation, the first element of the brush type control is displayed in the projection area, wherein the first element is a brush type in the brush type control, and the brush type control is in a bar shape.

7. The method according to claim 4, characterized in that When the control is the brush thickness control, and when the first operation is a gesture operation, displaying the first element of the control in the projection area in response to the first operation includes: In response to the gesture operation, the first element of the brush thickness control is displayed in the projection area, wherein the first element is a brush of a certain thickness in the brush thickness control, and the brush thickness control is in a strip shape.

8. The method according to claim 4, characterized in that When the control is the color absorption control, and when the first operation is a gesture operation, displaying the first element of the control in the projection area in response to the first operation includes: In response to the gesture operation, a first element of the color absorption control is displayed in the projection area, wherein the first element is a color corresponding to a target position on the display screen.

9. The method according to claim 4, characterized in that When the control is the erase control, and when the first operation is a gesture operation, displaying the first element of the control in the projection area in response to the first operation includes: In response to the gesture operation, a first element of the erasing control is displayed in the projection area, wherein the first element is an erasing tool in the erasing control, and the erasing control is in a strip shape.

10. The method according to any one of claims 5 to 9, characterized in that The gesture operation is one of a pinch operation, a slide operation, a tap operation, a rotation operation or a press operation.

11. The method according to any one of claims 1 to 10, characterized in that The first operation and the second operation are operations of the same type.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Detecting the state of the touch device; When the state is a contact state, an operation is performed according to the element in the projection area.

13. The method according to any one of claims 1 to 12, characterized in that The touch device is a stylus pen, and the projection area is an area where the intersection of the axis of the stylus pen in an inclined state and the display interface of the terminal device is located.

14. A terminal device, characterized in that: The terminal device comprises a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the terminal device executes the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 13.

16. A chip system, characterized in that: The chip system includes a memory and a processor, and the processor is configured to execute a computer program stored in the memory to implement the method according to any one of claims 1 to 13.

Images