Color display method and electronic equipment

Abstract

The invention provides a color display method and electronic equipment. In the method, the electronic equipment responds to a received user operation, moves a color picker control into an area outside an interface of a first application, and performs color picking on the position where the color picker control is located, so that global color picking of a display interface in a screen is realized, and the richness and practicability of screen color picking are improved.

Description

Technical Field

[0001] The present application relates to the field of terminal devices, and in particular to a method for displaying colors and an electronic device. Background Art

[0002] With the development of terminal device technology, the types of terminal device applications are increasing, and the functions provided are becoming more and more abundant. Current note applications and the like can provide drawing functions, so that users can draw images in the application. During the drawing process, users can use the color picking function provided by the note application to pick the color of a specified position in the interface, so as to attach the required color to the brush. Summary of the invention

[0003] The present application provides a method and electronic device for displaying colors. In the method, the electronic device can select a color at any position in the interface to meet user needs.

[0004] In the first aspect, the present application provides a method for displaying colors. The method includes: an electronic device displays a first interface at a first time, and the first interface includes a second interface of a first application and a third interface of a second application. Wherein, the second interface is a part of the first interface, and the second interface includes a first icon, and the color of the first icon is a first color. At a second time, the electronic device responds to the first operation received on the second interface, and the color picker control is displayed at a first position of the first interface. Then, the electronic device responds to the second operation received on the color picker control, and the color picker control is displayed at a second position of the first interface. Wherein, the second position is outside the display area of ​​the second interface, and the color picker control displays a second color, wherein the second color is the color of the pixel corresponding to the second position in the first interface. At a third time, the electronic device responds to the third operation received, does not display the color picker control, and the color of the first icon is changed to the second color. In this way, the present application can realize cross-process interaction, and the user can place the color picker control at any position outside the interface display area of ​​the first application to pick colors, thereby meeting the different needs of users for colors when drawing.

[0005] For example, the user may operate the electronic device according to the color picking requirement, so that different color picking objects are displayed on the display interface of the electronic device, thereby realizing color picking for different color picking objects.

[0006] Exemplarily, the first time and the third time are different times, and it can be understood that the content displayed on the first interface at the first time is different from the content displayed on the first interface at the third time. For example, at the first time, the first interface includes the second interface and the third interface, wherein the first icon in the second interface is the first color. At the second event, the first interface still includes the second interface and the third interface, and the first icon in the second interface is the second color.

[0007] Exemplarily, the first position may be the position where the color picker control last stopped when the color was picked last time. The first position may also be a preset position, such as the center of the first interface. The first position may be included in the second interface or in the third interface, which is not limited in this application.

[0008] Exemplarily, the size of the second interface is smaller than the size of the first interface, and the size of the third interface is smaller than the size of the first interface.

[0009] Exemplarily, the second interface and the first interface can be displayed overlappingly, in split screens, or in the form of floating windows, which is not limited in this application.

[0010] Exemplarily, the first operation is Figure 4b Click operation on the color picking option.

[0011] Exemplarily, the second operation is a dragging operation, and the hand is not lifted during the dragging process.

[0012] Exemplarily, the third operation is a hand-raising operation.

[0013] Exemplarily, the color picker control may also be called a color picker or a color picker icon, etc., which is not limited in this application.

[0014] Exemplarily, the user may perform various user operations in the present application, such as the first operation, the second operation, and the third operation, by using a finger or a stylus.

[0015] In one possible implementation, the method further includes: at a fourth time, the electronic device responds to the received first operation on the second interface, and the color picker control is displayed at the third position of the first interface. In response to the received pressing operation on the second icon in the first interface, the electronic device displays the color picker control at the fourth position of the first interface, and the color picker displays a third color, which is the color of the pixel corresponding to the fourth position in the first interface. At the fifth time, the electronic device responds to the received fourth operation by not displaying the color picker control, and the color of the first icon is changed to the third color. In this way, when the electronic device executes the global color picking process, the touch events of the electronic device are locked, and all touch events only act on the global color picking.

[0016] Exemplarily, the fourth operation is a hand-raising operation.

[0017] Exemplarily, when the color picker control is in the third position, the lower half of the outer ring of the color picker control displays the second color, that is, the color picked last time.

[0018] In a possible implementation, the method further includes: at a sixth time, the electronic device displays a fourth interface in response to a received click operation on the second icon, the fourth interface includes the second interface, and the fourth interface does not include a color picker control. In this way, before the electronic device completes the global color picking process or does not execute the global color picking process, the touch event of the electronic device is executed according to the normal process, that is, the touch event is not locked. The electronic device can display a corresponding interface in response to the user's operation. For example, the user clicks on the calendar application icon, and the displayed fourth interface includes the calendar application interface and the second interface.

[0019] In one possible implementation, the second application is a video application, the third interface is a video playback interface, the video playback interface includes a video playback window, and the video image is played in the video playback window. The electronic device responds to the second operation received on the color picker control, and the color picker control is displayed at the second position of the first interface, including: the electronic device responds to the second operation received on the color picker control, the color picker control is displayed at the second position of the first interface, the second position is in the video playback window, and the color displayed by the color picker control is the color of the pixel corresponding to the second position in the video playback interface; the color picker control remains in the second position, and the color displayed by the color picker control changes with the color of the pixel corresponding to the second position in the played video image. In this way, the color picking object corresponding to the global color picking in the present application can be a video stream, that is, the user can select a color in the video stream through the color picker.

[0020] In a possible implementation, the second application is a camera application, and the third interface is a photo preview interface, in which a preview image captured by the camera is displayed; the electronic device responds to the second operation received on the color picker control, and the color picker control is displayed at the second position of the first interface, including: the electronic device responds to the second operation received on the color picker control, and the color picker control is displayed at the second position of the first interface, the second position is in the photo preview interface, and the color displayed by the color picker control is the color of the pixel corresponding to the second position in the photo preview interface. The color picker control remains in the second position, and the color displayed by the color picker control changes with the color of the pixel corresponding to the second position in the preview image. In this way, the color picking object corresponding to the global color picking in the present application can be an image captured by the camera, that is, the user can select the color in the preview image through the color picker to obtain the color in the real world.

[0021] In a possible implementation, the color picker control includes an inner ring area and an outer ring area, wherein the inner ring area includes a first color picking area and at least one second color picking area, and the outer ring area includes a first semi-ring and a second semi-ring. When the color picker control is displayed at a second position on the first interface, the first color picking area displays the second color, the at least one second color picking area displays the color corresponding to at least one pixel around the second position, the first semi-ring displays the second color, and the second semi-ring displays the preset color.

[0022] Exemplarily, the preset color may be a color obtained by a previous color extraction.

[0023] Exemplarily, the first color picking area may be located at the center of at least one second color picking area.

[0024] Exemplarily, the first color picking area and the second color picking area have the same size.

[0025] Exemplarily, the shape of the color picker control is original or square.

[0026] In a possible implementation, in response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, including: receiving a second operation on a fifth position in the color picker control, and displaying the color picker control at a second position of the first interface; wherein the color picking center of the color picker control is located at the second position, and the fifth position deviates from the color picking center. In this way, to avoid occlusion, the user can drag any position in the color picker control except the first color picking area to move the color picker control, thereby picking colors at the designated position.

[0027] In one possible implementation, the method further includes: at the seventh time, the electronic device displays ink in the second interface in response to the received fifth operation, and the color of the ink is the second color. In this way, the user can use the global color picking scheme in this application to pick colors for any position in the interface. For example, the color picking object can be any point in the video stream or a point on the interface of other applications. The electronic device can add color to the brush based on the color picking result. The user can use the brush tool to draw or write in the second interface of the first application, and the color of the corresponding ink is the color picked by the global color picking.

[0028] In a possible implementation, in response to a received first operation on a second interface, a color picker control is displayed at a first position of a first interface, including: the electronic device calls a stylus tool module to display the color picker control, and subscribes to a touch event from an input module through the stylus tool module. In this way, each module in the electronic device implements a cross-process call, and the stylus tool module can subscribe to a touch event to obtain the touch event obtained by the input module. Moreover, after the stylus tool module subscribes to the touch event, the touch event of the electronic device is locked.

[0029] In one possible implementation, in response to a received second operation on a color picker control, the color picker control is displayed at a second position on the first interface, including: the electronic device triggers the global writing module through the stylus tool module to obtain the first bitmap corresponding to the screenshot of the first interface. The electronic device outputs the touch event obtained by the input module to the global writing module through the stylus tool module, and obtains the regional color value obtained by the global writing module based on the touch event and the first bitmap, wherein the regional color value includes the color of the first pixel corresponding to the second position in the first interface, and the color corresponding to at least one pixel around the first pixel. The electronic device updates the color displayed in the color picker control through the stylus tool, and the regional color value is displayed in the color picker control. In this way, the stylus tool interacts with the global writing module across processes, that is, across modules, to achieve global color picking of the screen display interface.

[0030] In a possible implementation, in response to the received third operation, the color picker control is not displayed, including: the stylus tool module unsubscribes from the touch event. In this way, the stylus tool module unsubscribes from the touch event so that the touch event of the electronic device can act on other applications.

[0031] In a second aspect, the present application provides an electronic device. The electronic device includes: one or more processors, a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: at a first time, displaying a first interface, the first interface includes a second interface of a first application and a third interface of a second application, the second interface is a part of the first interface, the second interface includes a first icon, and the color of the first icon is a first color; at a second time, in response to a first operation received on the second interface, a color picker control is displayed at a first position of the first interface; in response to a second operation received on the color picker control, the color picker control is displayed at a second position of the first interface, the second position is outside the display area of ​​the second interface, and the color picker control displays a second color, the second color is the color of the pixel corresponding to the second position in the first interface; at a third time, in response to a third operation received, the color picker control is not displayed, and the color of the first icon is changed to the second color.

[0032] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: at a fourth time, in response to a first operation received on the second interface, a color picker control is displayed at a third position on the first interface; in response to a press operation received on a second icon in the first interface, the color picker control is displayed at a fourth position on the first interface, and a third color is displayed in the color picker, where the third color is the color of a pixel corresponding to the fourth position in the first interface; at a fifth time, in response to a fourth operation received, the color picker control is not displayed, and the color of the first icon is changed to the third color.

[0033] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: at a sixth time, in response to receiving a click operation on a second icon, a fourth interface is displayed, where the fourth interface includes the second interface and does not include a color picker control.

[0034] In one possible implementation, the second application is a video application, the third interface is a video playback interface, the video playback interface includes a video playback window, and the video image is played in the video playback window; when the computer program is executed by one or more processors, the electronic device performs the following steps: in response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, the second position is in the video playback window, and the color displayed by the color picker control is the color of the pixel corresponding to the second position in the video playback interface; the color picker control remains in the second position, and the color displayed by the color picker control changes with the color of the pixel corresponding to the second position in the played video image.

[0035] In one possible implementation, the second application is a camera application, and the third interface is a photo preview interface, which displays a preview image captured by the camera; when the computer program is executed by one or more processors, the electronic device performs the following steps: in response to a second operation received on a color picker control, the color picker control is displayed at a second position of the first interface, the second position is in the photo preview interface, and the color displayed by the color picker control is the color of a pixel corresponding to the second position in the photo preview interface; the color picker control remains in the second position, and the color displayed by the color picker control changes with the color of the pixel corresponding to the second position in the preview image.

[0036] In one possible implementation, the color picker control includes an inner ring area and an outer ring area, wherein the inner ring area includes a first color picking area and at least one second color picking area, and the outer ring area includes a first semi-ring and a second semi-ring; when the color picker control is displayed at a second position on the first interface, the first color picking area displays the second color, at least one second color picking area displays the color corresponding to at least one pixel around the second position, the first semi-ring displays the second color, and the second semi-ring displays a preset color.

[0037] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: receiving a second operation on the fifth position within the color picker control, and the color picker control is displayed at the second position of the first interface; wherein the color picking center of the color picker control is located at the second position, and the fifth position deviates from the color picking center.

[0038] In one possible implementation, when the computer program is executed by one or more processors, the electronic device executes the following steps: at the seventh time, in response to receiving the fifth operation, ink is displayed in the second interface, and the color of the ink is the second color.

[0039] In one possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device calls a stylus tool module to display a color picker control, and subscribes to a touch event from an input module through the stylus tool module.

[0040] In a possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the electronic device triggers the global writing module through the stylus tool module to obtain the first bitmap corresponding to the screenshot of the first interface. The electronic device outputs the touch event obtained by the input module to the global writing module through the stylus tool module, and obtains the regional color value obtained by the global writing module based on the touch event and the first bitmap, wherein the regional color value includes the color of the first pixel corresponding to the second position in the first interface, and the color corresponding to at least one pixel around the first pixel.

[0041] In a possible implementation, when the computer program is executed by one or more processors, the electronic device performs the following steps: the stylus tool module unsubscribes from the touch event. In this way, the stylus tool module unsubscribes from the touch event so that the touch event of the electronic device can act on other applications.

[0042] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0043] In a third aspect, the present application provides a computer-readable medium for storing a computer program, wherein the computer program includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0044] In a fourth aspect, the present application provides a computer program, comprising instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0045] In a fifth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive a signal and control the sending pin to send a signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0047] Figure 2 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0048] Figure 3 Schematic diagram of a global color picking process shown as an example;

[0049] Figure 4a to Figure 4c A schematic diagram of an exemplary user interface;

[0050] Figure 5a to Figure 5b A schematic diagram of an exemplary user interface;

[0051] Figure 6 A schematic diagram of a screen shot is shown for exemplary purposes;

[0052] Figure 7a to Figure 7b A schematic diagram of an exemplary user interface;

[0053] Figure 8a to Figure 8b is a schematic diagram of position coordinates shown as an example;

[0054] Fig. 9 is a schematic diagram of color selection shown as an example;

[0055] Fig.10 The figure is a schematic diagram of updating the color picker shown as an example;

[0056] Fig.11 is a schematic diagram of a color picker shown as an example;

[0057] Figure 12a to Figure 12b A schematic diagram of an exemplary user interface;

[0058] Fig.13 A schematic diagram of an exemplary user interface;

[0059] Fig.14 A schematic diagram of an exemplary user interface;

[0060] Figure 15a to Figure 15c is a schematic diagram showing an exemplary color palette, color ring, and color value;

[0061] Fig.16 A schematic diagram of an exemplary user interface;

[0062] Figure 17a to Figure 17d A schematic diagram of an exemplary user interface;

[0063] Figure 18a to Figure 18b A schematic diagram of an exemplary user interface;

[0064] Fig.19 Schematic diagram of the device structure shown as an example. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0066] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0067] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0068] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0069] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0070] Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. It should be understood that Figure 1 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application-specific integrated circuits. In the embodiment of the present application, the electronic device 100 is taken as an example of a tablet for illustration. In other embodiments, the color picking method in the embodiment of the present application can also be applied to electronic devices with display screens such as mobile phones, wearable devices, vehicle-mounted devices, smart home devices, and televisions, and the present application does not limit it.

[0071] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0072] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0073] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0074] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0075] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0076] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0077] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0078] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0079] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0080] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0081] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0082] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0083] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0084] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0085] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0086] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0087] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0088] In an embodiment of the present application, the relevant components can obtain the image captured by the camera by calling the camera, and extract colors from the image captured by the camera to obtain the real color in reality.

[0089] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0090] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0091] The sensor module includes but is not limited to: a pressure sensor, a gyroscope sensor, a touch sensor, etc. The sensor involved in this application is mainly a touch sensor.

[0092] A touch sensor is also called a "touch panel". The touch sensor can be arranged on the display screen 194. The touch sensor and the display screen 194 form a touch screen, also called a touch screen (Touch Panel, abbreviated as: TP). The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor can also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0093] The touch panel integrated circuit (TP IC) and the mobile phone application processor (AP) are generally connected through the Inter-Integrated Circuit (I2C) or Serial Peripheral Interface (SPI) bus. When the user touches the touch screen with his finger, the touch sensor detects the change in the capacitance value caused by the finger on the touch screen, and the TP IC can obtain the position of the touch point (i.e., coordinates). The AP obtains the position of the touch point in plain text through the I2C or SPI bus, thereby completing the touch report process.

[0094] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0095] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0096] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into an application layer, an application framework layer, and a kernel layer. In other embodiments, more or fewer layers may be included, which is not limited in this application.

[0097] The application layer can include a series of application packages.

[0098] like Figure 2 As shown, the application package may include drawing (also called painting), memo, document, PenKit (handwriting tool), Globalwrite (global writing) and other applications. Among them, PenKit and Globalwrite can also be called modules or components. In some instances, PenKit and Globalwrite may be located in the application framework layer and provide an interface for upper-layer application services to respond to calls from upper-layer applications, which is not limited in this application.

[0099] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0100] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a resource manager, a ScreenService (screen service), an Input (input) module, a view system, and the like.

[0101] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0102] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0103] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0104] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0105] ScreenService is used in the embodiment of the present application to provide services such as screenshots (referred to as screenshots). Related modules or services can obtain screenshots by calling ScreenService. Optionally, the screenshots can include full-screen screenshots or partial screenshots. The screenshots involved in the embodiment of the present application all refer to full-screen screenshots, that is, the treasure trove in the full-screen screenshots is not limited to: all images displayed on the screen (including pictures, text, controls, etc.).

[0106] The Input module can be used to obtain touch events of the touch sensor from the sensor driver. Among them, the touch event includes but is not limited to: touch position (also known as touch coordinates), wherein the touch position is usually the corresponding position coordinate in the coordinate system established with the upper left corner of the screen as the origin. The data interaction process between the touch sensor, the sensor driver and the Input module can refer to the existing technology, and this application will not repeat it.

[0107] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, sensor driver, etc.

[0108] Understandably, Figure 2 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0109] Figure 3 For an exemplary global color picking process diagram, please refer to Figure 3 , including but not limited to the following steps:

[0110] S301, the user triggers color picking.

[0111] For example, when using an application, the user may operate an option corresponding to the color picking function provided by the application to instruct the application to start executing the color picking process. Figure 4a to Figure 4b This is a user interface diagram for an exemplary user-triggered color picking function. Figure 4a For example, the user triggers the color picking function in the memo application as an example. The user can start the memo application by clicking the memo application icon on the desktop or through the sidebar. In response to the received user operation, the tablet displays the memo application interface in the first interface 400.

[0112] The memo application interface includes, but is not limited to: an input area 410 (which may also be referred to as a display area or an input window, etc., which is not limited in this application) and an input control 420, etc.

[0113] The input area 410 includes, but is not limited to, options or controls such as a title, a return option, and a confirmation option. The user can draw an image or type text in the input area 410 by finger touch, mouse control, or stylus. Figure 4a The input content (ie, image) in the input area 410 is only a schematic example.

[0114] The input tool box 420 includes but is not limited to input tools and function controls (also referred to as function options or function icons, etc., which are not limited in this application). The input tools include but are not limited to some different types of brushes (or paintbrushes) or coloring tools, such as crayon brushes 421 (also referred to as crayon coloring tools or simply crayons, which are not limited in this application), pens, pencils, markers, erasers, etc.

[0115] Exemplarily, the functional controls include but are not limited to: typing option 423, color option (also known as color picking option or coloring option, etc., which is not limited in this application), color picking option 422 (also known as color wheel option, which is not limited in this application) and more options.

[0116] The typing option 423 is used to trigger displaying a keyboard. When the user clicks the typing option 423, the tablet (specifically, the memo application) displays the keyboard in response to the received user operation.

[0117] The color options correspond to colors of different color values, such as red, green, white, black, etc. After selecting a corresponding brush, such as crayon 421, the user can select the corresponding color by clicking on the corresponding color option. For example, if the user chooses to click on the color option corresponding to green, the tip of crayon 421 can display the corresponding color (i.e., green). When the user operates (e.g., slides) on the input area 420 with a stylus (or finger), the tablet displays the corresponding ink on the input area 420 in response to the received user operation, and the color of the ink is green.

[0118] The color picking option 422 is used to trigger the display of the color picking box. When the colors provided by the color options described above do not meet the user's expectations, that is, the user needs more color options, the user can click the color picking option 422 to obtain more color options and select the desired color.

[0119] Still refer to Figure 4a , the user clicks on the color selection option 422 to obtain more color selection options. Figure 4b In response to receiving the user's click on the color picking option 422, the tablet displays a color picking box 430 in the memo application interface of the second interface 401. The color picking box 430 includes but is not limited to: a color picker option 431, a color palette option 432, a color ring option 433, and a color value option 434.

[0120] The color picker option 431 is used to trigger the display of the color picker. When the user clicks the color picker option 431, the tablet displays the color picker in response to the received user operation.

[0121] The color palette option 432 is used to trigger the display of the color palette. In the embodiment of the present application, when the user clicks the color selection option 422, the tablet displays the color selection box 430 in response to the received user operation, and by default displays the color palette and other controls corresponding to the color palette option 432 in the color selection box 430. The options of the color ring and the color value will be described in detail in the following embodiments. Figure 4b The corresponding contents of the color palette in are explained.

[0122] Illustratively, the color selection box 430 includes a color palette 435 , a color selection box 436 , a color bar 437 , and color options 438 .

[0123] The color plate 435 includes a plurality of blocks (or other graphics, which are not limited in this application), each row of blocks (or each column of blocks) belongs to the same color system, and the color value of the blocks in each row or column gradually decreases or increases, which is not limited in this application. In other words, the color (i.e., color value) corresponding to each block is different.

[0124] The color selection box 436 is used to select a corresponding color in the color palette 435. The user can select a color in the color palette 435 by moving the color selection box 436. Exemplarily, after the user selects the corresponding color through the color selection box 436, the tablet fills the color selection option 422 with the color selected by the user in response to the received user operation, for example, the inner ring in the color selection option 422 is filled with yellow. Correspondingly, the color of the ink input by the user through the stylus or finger is also yellow.

[0125] The color bar 437 is used to adjust the transparency. That is, after selecting the corresponding color, the user can further adjust the transparency of the color through the color bar 437. The larger the value, the higher the transparency, and the smaller the value, the lower the transparency.

[0126] The color options 438 include but are not limited to options corresponding to different colors, such as a green color option, a red color option, etc. The user can click the green color option to cause the tablet to move the color selection box 436 to a position corresponding to the green color in the color palette options 435 based on the user operation.

[0127] Still refer to Figure 4b In this example, the user can click on the color picker option 431 to trigger the display of the color picker, so that the user can use the color picker to select the color of the current display interface (such as the second interface 400).

[0128] It should be noted that in this example, only the memo application is used as an example. In other embodiments, the solution in the embodiment of the present application can also be applied to native applications or third-party applications such as drawing applications, note applications, document applications, etc., and this application does not limit it.

[0129] It should be further noted that in this example, only the user operates by finger touch (such as clicking) to trigger the corresponding function. In other embodiments, the user can also perform the corresponding operation by mouse, keyboard, stylus, etc., and this application does not limit it.

[0130] S302, the memo application sends a showColorPicker (show color picker) message to PenKit.

[0131] Exemplarily, the memo application receives the user's operation of clicking the color picker option 431, and determines that the global color picker function needs to be called. The memo application sends a showColorPicker message to PenKit to instruct the color picker to be displayed. Specifically, PenKit provides a showColorPicker interface function, and the memo application calls the showColorPicker interface function to trigger PenKit to display the color picker.

[0132] S303, PenKit executes the startColorPickerHandler (triggering color picker processing) method function.

[0133] Exemplarily, PenKit determines that the color picker needs to be displayed in response to the received showClorPicker message, which can be understood as the need to start the global color picking function (or global color picking service). Accordingly, PenKit runs PenKit's internal startColorPickerHandler command (method function) to perform self-startup, that is, trigger the global color picking function (or global color picking service). Among them, the startColorPickerHandler method function encapsulates multiple sub-functions (or sub-commands), and PenKit implements self-startup by executing multiple sub-functions in the startColorPickerHandler method function. The method functions below are similar and will not be repeated below.

[0134] S304, PenKit displays a color picker.

[0135] Exemplarily, after PenKit completes the startup execution (ie, runs the startColorPickerHandler method function), a color picker is displayed on the screen to pick colors for pixels on the screen through the color picker. Figure 4cFor an exemplary user interface diagram, please refer to Figure 4c After the user clicks the color picker option 431 , the tablet cancels the display of the color picker box 430 in response to the received user operation, and displays the color picker 440 in the third interface 402 .

[0136] In the embodiment of the present application, after the application is started, when the global color picking service is executed for the first time, the color picker 440 is displayed at the initial position, for example, the center position of the display interface 400. Specifically, PenKit can establish a coordinate system with the upper left corner of the tablet display interface 400 as the origin, and obtain the center position coordinates of the display interface 400, and PenKit displays the color picker 440 at the position coordinates, wherein the center of the color picker 440 is located at the center position coordinates, and the description of the position coordinates can be referred to below. Figure 8a The relevant description of the application is not repeated here. Optionally, the application startup includes but is not limited to: the first startup of the application after the tablet is turned on for the first time or each time it is restarted; or, after the application is closed, it is restarted. For example, after the user closes the memo application (that is, ends the memo application process), restarts the memo application, and follows the operation method described above (that is, Figure 4a to Figure 4b ) triggers the global color picking function, the tablet responds to the received user operation and displays the color picker 440 at the initial position (ie, the center position of the display interface 400).

[0137] Optionally, when the user triggers the global color picker function again within the application (i.e., the application is not closed), the tablet responds to the received user operation and displays the color picker 440 at the position where the color picker 440 last stopped. The specific example will be described in detail in the following embodiments.

[0138] In a possible implementation, each time the application is called to the foreground (i.e., the application is switched from the background to the foreground when the application is not closed), when the global color picking service is executed for the first time, the color picker 440 is displayed at the initial position, such as the center of the display interface 400. For example, after the user triggers the color picker 440 in the memo application and selects a color, the user calls other applications (such as video applications) to the foreground and switches the memo application to the background (i.e., only the video application is currently in the foreground). When the user switches the memo application to the foreground again and triggers the global color picking function, the color picker 440 is displayed at the initial position (i.e., the center of the third interface 400).

[0139] In another possible implementation, each time the application is called to the foreground (i.e., the application switches from the background to the foreground when the application is not closed), when the user triggers the global color picking function again within the application (i.e., the application is not closed) (i.e., the application has triggered the global color picking function at least once before switching to the background), the tablet responds to the received user operation and displays the color picker 440 at the position where the color picker 440 last stopped.

[0140] Still refer to Figure 4c , the color picker 440 is displayed on the top layer of the display interface 440, which is above all images or layers. The color picker 440 includes an outer ring 441 (also referred to as an outer ring) and an inner ring 442 (also referred to as an inner ring). Among them, the radius of the inner ring 442 is smaller than the radius of the outer ring 441. Among them, the outer ring includes an upper half ring 4411 and a lower half ring 4412. Exemplarily, the upper half ring 4411 can be understood as the upper half between the outer ring 441 and the inner ring 442, and the lower half ring 4412 can be understood as the lower half between the outer ring 441 and the inner ring 442.

[0141] In an embodiment of the present application, the upper half ring 4411 and the lower half ring 4412 are equal in size and are divided by the diameter in the lateral direction of the color picker device 440. The lateral direction can optionally be parallel to the X-axis (i.e., the long side of the flat plate). In other embodiments, the upper half ring 4411 and the lower half ring 4412 may be unequal in size, and / or the upper half ring 4411 and the lower half ring 4412 may be divided by any other diameter of the color picker 440 (for example, the diameter in the longitudinal direction, which is parallel to the Y-axis (i.e., the wide side of the flat plate)), which is not limited in the present application.

[0142] In the embodiment of the present application, the upper half ring 4411 is used to display the color currently obtained by the color center point 443. Accordingly, the color displayed in the upper half ring 4411 changes with the color obtained by the color center point 443. Optionally, the color in the upper half ring 4411 fills the upper half ring 4411. Specific examples will be described in detail below.

[0143] Exemplarily, the lower half ring 4412 is used to display the color that the color picker last picked up. The last picked up color refers to the color that the color picker picked up last time, that is, the color that the color picker picked up most recently. It can also be understood as the color picked up by the color center point 443 when the user raised his hand last time (specific examples will be described below). Specific examples will be described in detail below.

[0144] In a possible implementation, if the global color picking function is triggered for the first time after the memo application is started (it should be noted that the "triggering the global color picking function" in the embodiment of the present application can be understood as triggering the execution Figure 3), the lower half ring 4412 can display the default color (for example, white), or the color corresponding to the current brush, or the same color as the upper half ring 4411. In other words, the last color taken by the lower half ring refers to the same application triggered during the startup to shutdown process. Optionally, if the application is switched to the background after startup and switched to the foreground again, the tablet displays the color picker 440 in response to the received user operation. The lower half ring 4412 of the color picker 440 displays the color selected by the color picker for the latest time (i.e., the most recent time) before the application switched to the background.

[0145] In another possible implementation, after the device (e.g., tablet) is initially started or restarted, when the color picker 440 is first displayed, its lower half ring 4412 displays a default color, or displays the color corresponding to the current brush, or can display the same color as the upper half ring 4411. During subsequent use, the lower half ring 4412 of the color picker 440 displays the last color. The last color may be obtained in other applications or in this application, which is not limited in this application.

[0146] In another possible implementation, the color displayed in the lower half ring 4412 of the color picker 440 can be differentiated by application. That is, when the lower half ring 4412 is displayed in the application, except for the initial call to display the default color, each call of the color picker 440 in the same application displays the color selected in the last call in the application. For example, if the color selected by the memo application this time is black, then after the memo application is closed and restarted, the tablet responds to the received user operation, calls the global color picking function and displays the color picker 440, and the color displayed in the lower half ring of the color picker 440 is black. The user closes the memo application, starts the drawing application, and calls the global color picking function in the drawing application. In response to the received user operation, the tablet calls the global color picking function and displays the color picker 440. In this application, the lower half ring of the color picker 440 may be green, which is the color obtained by the global color picking function in the drawing application last time (i.e., the most recent time).

[0147] In this example, the last color picked by the color picker is black. Accordingly, the lower half ring 4412 of the color picker 440 is filled with black.

[0148] In the embodiment of the present application, as described above, the filling color of the upper half ring 4411 is consistent with the color of the center point of the color picker 440, but in S304, since the color picker 440 has not started to pick colors (it can also be understood as not obtaining the color of the specified pixel point), accordingly, in this example, the filling color of the upper half ring 4411 is the default color, such as white (it can also be other colors, which are not limited in this application). In addition, the color in the inner ring 442 of the color picker 440 is also the default color (for example, white). After executing S312, the colors in the upper half ring 4411 and the inner ring 442 are updated synchronously again.

[0149] Still refer to Figure 4c , the inner ring 442 includes at least one square, and in other embodiments, it can also be other shapes, which is not limited in this application. In the embodiment of the present application, each square in the inner ring 442 corresponds to a pixel in the screen, and the color of the square (which can also be understood as the filling color) is the color of the corresponding pixel. It can be understood that since the pixel is a smaller unit, at least one square in the inner ring 442 will correspond to the color of the pixel, so that the user can intuitively obtain the color result of the current position (that is, the color displayed by the center point 443) and the colors corresponding to other pixels around the current position.

[0150] Optionally, adjacent blocks in the inner ring 442 may be tightly connected or may have gaps between them, which is not limited in the present application.

[0151] Figure 5a For an exemplary user interface diagram, please refer to Figure 5a In an embodiment of the present application, if the user needs to perform global color picking, that is, to pick the color of objects other than the present application (such as the memo application), the user can operate the memo application to shrink the memo application for display, for example, in the form of a floating window in the embodiment of the present application. In response to the received user operation, the tablet displays a window 510 in the fourth interface 500, and the current interface of the memo application is displayed in the window 510. In addition, the desktop 520 is displayed in the fourth interface 500 (the size of the desktop 520 is equal to the size of the fourth interface 500). The user can operate the memo application in window 510 to trigger the global color picking function. The specific triggering method can be referred to. Figure 4a to Figure 4b The description of the color picker 530 is not repeated here. Accordingly, the tablet displays the color picker 530 at the center of the fourth interface 500 in response to the received user operation. The specific description of the color picker 530 can be found in Figure 4c The relevant description in will not be repeated here.

[0152] It should be noted that Figure 5aThe size and position of the window 510 in FIG. 5 are only illustrative examples and can be set according to actual needs, and this application does not limit them. Optionally, the user can manually adjust the position and size of the window 510.

[0153] It is further necessary to explain that Figure 5a In the fourth interface 500, only the desktop 520 is shown as an example. In other embodiments, the fourth interface 500 may display any object that the user needs to pick a color. The user may select the object that needs to be picked at any time before clicking the color picker option. For example, the user may select a memo application (the memo application may be displayed in full screen, for example, Figure 4a ) is switched to the background, a corresponding operation is performed in the fourth interface 500 (for example, clicking the icon of the object to be colored) to call the object to be colored to the foreground, that is, to display it in the fourth interface 500. Then, the user can operate the memo application to indicate that the memo application is displayed in the window 510. The specific operation steps can be set according to actual needs, and this application is not limited. For another example, after the user sets the memo application to be displayed in the window 510, the current memo application and the desktop are both foreground applications. The user can click the application icon on the desktop to call the object to be colored to the foreground. For example, the user can click the gallery application icon to open the gallery application. During the user operation, the window 510 still maintains the current position and the display content unchanged. The interface of the gallery application is correspondingly displayed in the fourth interface 500. The gallery application interface may include but is not limited to at least one picture, and the gallery application interface is the object to be colored. Among them, the window 510 is always located in the upper layer of the fourth interface 500, for example, it is displayed in the upper layer of the gallery application interface. For another example, the user can operate the tablet to display the object that needs to be colored in the fourth interface 500, and display the memo application in the fourth interface 500 in a floating window through a sidebar or the like. Figure 5a There are many ways to control the interface status, which can be operated according to actual needs. The embodiments of this application are all illustrative examples and are not limited in this application.

[0154] Still refer to Figure 5a In this example, the initial position of the color picker 530 is located at the center of the fourth interface 500. This position is outside the memo application (i.e., window 510) and is no longer limited to the memo application. In other words, the color picker 530 in the embodiment of the present application can pick colors for objects outside the memo application.

[0155] It should be further explained that, in this embodiment, only the desktop 520 is used as an example for description. In other embodiments, the object to be picked up can also be a preview image (i.e., a preview image stream) captured by a camera, a video image (i.e., a video stream), a picture (e.g., a picture in a gallery), an application interface (which can be an interface of any application, such as a calculator application interface, a music application interface, etc., which is not limited in this application), etc., which is not limited in this application.

[0156] It should be further explained that the interface of the memo application can be understood as a part of the object to be color-picked, that is, global color picking is to pick colors within the screen, that is, the color-picking object includes the interface of the memo application. However, the object to be color-picked described in the embodiment of the present application can be understood as the object that the user actually wants to pick colors from, that is, the desktop 520. That is, the user expects to pick colors from the desktop 520 and use the obtained colors to color the objects in the memo application.

[0157] It should be further noted that the embodiment of the present application only takes the display of the memo application (i.e., the application using the global color picking function) in the floating window (i.e., window 510) as an example. In other embodiments, the memo application and the object to be color picked may also be displayed in a split screen or other manner, which is not limited in the present application.

[0158] S305, PenKit registers touch monitoring with the Input module.

[0159] Exemplarily, PentKit sends a touch monitoring request (also referred to as a touch event monitoring request, which is not limited in this application) to the Input module to request monitoring of touch events. It can be understood that PenKit requests the Input module to feed back the touch event to PenKit after detecting the touch event.

[0160] In the embodiment of the present application, as described above, the Input module can be used to detect the result of the touch sensor and obtain the corresponding touch event. The Input module can report the obtained touch event to the designated module or application registered and monitored with it.

[0161] Exemplarily, after Penkit registers touch monitoring with the Input module, the Input module feeds back the acquired touch events to Penkit in response to the received touch monitoring request.

[0162] Exemplarily, the touch event fed back by Input to PenKet includes but is not limited to: the touch event type and the touch event occurrence location (also referred to as the touch event occurrence coordinates, or simply referred to as the touch location or touch coordinates, etc., which are not limited in this application). The description of the touch location can be found above and will not be repeated here.

[0163] Optionally, the types of touch events include but are not limited to: press (Down) type, move (Move) type and raise (Up) type event types. Corresponding to different event types, PenKit will perform different operations, and the specific processes will be described one by one below.

[0164] In an embodiment of the present application, after PenKit registers a monitor with the Input module, the touch events obtained by the Input module are no longer fed back to other upper-level applications or modules. It can be understood that after PenKit registers the touch monitor, the touch events of the Input module are locked (or blocked) and will no longer be called by other upper-level applications or modules. In other words, from the user interface perspective, when the tablet executes the color picking process, that is, after the tablet displays the color picker 440, before the color picking is completed, the touch events on the screen only act on the color picker, and other touch events are invalid, that is, the touch events are locked by PenKit and are only used for the global color picking process. For example, Figure 5b As shown in (1), after the tablet displays the color picker, the user touches the location of the calendar icon, such as Figure 5b As shown in (2), the tablet responds to the received user operation and displays the color picker 530 at the position touched (ie pressed) by the user. Optionally, the color picker 530 is displayed from Figure 5b Move to the position shown in (1) Figure 5b The position shown in (2) can be displayed after disappearing, that is, canceling Figure 5b The (1) position is displayed and appears in Figure 5b (2) position (i.e., the position touched by the user). The color picker 530 will pick colors for the current position (i.e., the center of the color picker 530 and the pixels near it). The specific color picking method can be found below. Optionally, to avoid occlusion, the center of the color picker 530 does not coincide with the user touch position, that is, any point in the color picker 530 except the point in the lower half (preferably the point in the lower half) coincides with the user touch position.

[0165] For example, after the user raises his hand, the color picker 530 picks a color and cancels the display (for details, refer to the above), or before the color picker 530 is not displayed, that is, when PenKit does not lock the touch event, the interface displayed on the tablet is as follows: Figure 6 As shown in the example, if the user clicks the calendar option, the tablet displays the calendar application interface in the first display interface 500 in response to the received user operation. Optionally, the window 510 is still displayed above the calendar application interface. It can be understood that the current desktop 520 will be replaced by the calendar application interface.

[0166] It should be noted that after PenKit registers the touch monitor, only the touch event is locked, and other events are still executed normally. For example, when the user clicks the power button, the tablet responds to the received user operation and turns off the display interface (i.e., turns off the screen). For another example, when the user slides down from the upper edge of the screen to call the drop-down menu, the tablet responds to the received user operation and displays the drop-down menu. For another example, when the screen slides from the side edge of the screen to the center of the screen to call the sidebar, the tablet responds to the received user operation and displays the sidebar. For another example, when the user taps the screen with a knuckle to indicate a screenshot, the tablet responds to the received user operation and takes a screenshot. In other words, after PenKit registers the touch monitor, other events or gestures other than touch events, such as edge gestures, will not be blocked. Exemplarily, in the process of the tablet responding to the received operation and performing the above-mentioned operations such as displaying the menu bar or taking a screenshot, the color picker 440 is still displayed at the current position.

[0167] Optionally, while the touch event is locked, other input methods can still be performed normally, for example, the user can input through keyboard or voice input, etc., which is not limited in this application.

[0168] It should be noted that after the user clicks the color picker option 431, until the user touches the color picker option 431 (such as Figure 8a The time interval between the global color picking (as shown in FIG. 4 and will be described in detail below) and the tablet continuing to execute S306 to S309 is about 100ms. That is, after PenKit requests touch monitoring, before receiving a touch event (i.e., the user touches the color picker option 431), PenKit has completed S306 to S309.

[0169] S306, Penkit is bound to Globalwrite.

[0170] Exemplarily, PenKit sends a bindService message to Globalwrite to request service binding with Globalwrite. Specifically, Globalwrite provides a bindService interface function for service binding with other services or modules. PenKit calls the bindService interface function of Globalwrite to perform service binding with Globalwrite.

[0171] In the embodiment of the present application, the binding service can also be understood as establishing a data channel with Globalwrite so that data can be exchanged between the two modules (or components) to achieve cross-process data transmission. The binding service can also be understood as PenKit requesting Globalwrite to provide corresponding services for it (i.e., Penkit).

[0172] S307, Globalwrite starts ScreenShotService (screen shot service, referred to as screen shot service).

[0173] Exemplarily, Globalwrite responds to the call of PenKit, that is, responds to the received bindService request, and Globalwrite starts ScreenShotService. Specifically, Globalwrite runs the relevant program (or command) of ScreenShotService to trigger ScreenShotService, thereby providing screenshot service for Globalwrite.

[0174] In the embodiment of the present application, Globalwrite starts ScreenShotService only after receiving a trigger indication (such as a bindService request) and feeds back the screenshot result to Penkit. Before being triggered, Globalwrite does not start ScreenShotService to avoid increasing system power consumption.

[0175] S308, Globalwrite sends a getScreenShot (get screenshot) message to ScreenService (screen service).

[0176] Exemplarily, after Globalwrite starts ScreenShotService, Globalwrite sends a getScreenShot request to ScreenService to request to obtain a screenshot. Specifically, Globalwrite calls the getScreenShot interface function of ScreenService to trigger ScreenService to obtain a screenshot and feed it back to Globalwrite.

[0177] As described above, ScreenService can be used for screenshots. In the embodiment of the present application, screenshots refer to capturing the image stream in the screen. The captured images include but are not limited to the images and options included in the image stream at the current moment (i.e., the screenshot moment). It should be noted that the color picker 530 is not included in the captured controls.

[0178] Exemplarily, ScreenService responds to the getScreenShot request of Globalwrite, takes a screenshot of the screen, and feeds the image (ie, the screenshot) back to Globalwrite.

[0179] For example, Figure 5a Taking the user interface shown in as an example, after PenKit displays the color picker 530 in the fourth interface 500, it starts to execute S305-S308, Globalwrite sends a getScreenShot request to ScreenService, and ScreenService responds to the received getScreenShot request to the image stream (i.e., as shown in FIG. 5 ) displayed in the current screen (i.e., the fourth interface 500). Figure 5a to take a screenshot. Figure 6 For an exemplary screenshot diagram, please refer to Figure 6 The image 600 includes but is not limited to all images and controls in the fourth interface 500 except the color picker 530, such as but not limited to: the image corresponding to the window 510 and the image corresponding to the desktop. Optionally, the image 600 may also include the battery icon, network icon, Bluetooth icon and time (not shown) in the fourth interface 500. Figure 5a The size (i.e., resolution) of the image 600 can be set according to actual needs, which is not limited in this application.

[0180] S309, Globalwrite executes the getColorLocation (get color location) method function.

[0181] Exemplarily, after Globalwrite obtains the image fed back by ScreenService (i.e., a screenshot, such as image 600), it executes the getColorLocation method function to perform image processing (also referred to as color positioning processing) on ​​image 600 to obtain the bitmap (bitmap), also referred to as a dot map, corresponding to image 600 to obtain the color of each pixel in image 600. That is, Globalwrite calculates the color value of each pixel in the image by calling the getColorLocation method function. The specific calculation method can refer to the existing technical embodiments, and this application does not limit it.

[0182] In the embodiment of the present application, S308-S309 may be set with an execution cycle, and the cycle duration may be set according to actual needs, which is not limited in the present application. For example, in the embodiment of the present application, the execution cycle of S308-S309 may be 200ms, that is, Globalwrite sends a getScreenShot message to ScreenService every 200ms (that is, calls the getScreenShot interface function once) to obtain a screenshot, and executes the getColorLocation method function on the obtained screenshot to obtain the corresponding dot map.

[0183] Optionally, the execution cycle of S308, that is, the cycle duration of Globalwrite obtaining the screenshot can be the same as the touch sensor acquisition cycle, which can also be understood as the cycle of the Input module reporting the touch event, for example, both are 200ms, which can be set according to actual needs and is not limited in this application.

[0184] S310, Penkit sends a setOffsetLocation message to Globalwrite.

[0185] For example, from the user's perspective, Figure 5a As shown, after receiving an operation in which the user clicks on the color picker option, the tablet displays a color picker 530 in the user interface 500 . Figure 7a For an exemplary user interface diagram, please refer to Figure 7a After the color picker 530 is displayed on the fourth interface 500, the user can drag the color picker 530 to pick a color at a designated position.

[0186] In the embodiment of the present application, the user can touch any position (specifically, any position) of the color picker 530 with a finger (or a stylus, a mouse, etc., which is not limited in the present application). Figure 4c The color picker 530 can be dragged to any position within the outer ring 441 shown in the figure. Before the user raises his hand, the color picker 530 continues to pick colors for the current fourth interface 500. In this way, in the embodiment of the present application, the user can drag the color picker 530 to pick colors by clicking anywhere, so that the user can observe the center position of the color picker 530 and the picked color, thereby avoiding finger occlusion to cause color picking errors.

[0187] Combine the following Figure 8a The position coordinate diagram shown in the figure provides a detailed description of the color picking method. Figure 8aWhen Penkit displays the color picker 530 (i.e., when executing S304), it establishes a coordinate system with the upper left corner of the fourth interface 500 as the origin, wherein the horizontal direction is the X-axis direction and the vertical direction is the Y-axis direction. In the embodiment of the present application, PenKit can obtain the coordinates of the center point of the fourth interface 500, that is, the coordinates of the center point 801 of the color picker 530, which is recorded as (X0, Y0). PenKit displays the color picker 530 at this position.

[0188] Furthermore, PenKit can obtain the position coordinates of the upper left corner (ie, point 802 ) of the square 803 (or rectangle) tangent to the color picker 530 in the coordinate system of the display interface 800 , which are recorded as (X1, Y1).

[0189] Still refer to Figure 8a For example, PenKit can obtain the coordinates of the center point 801 of the color picker 530 in the coordinate system with the point 802 as the origin, which is recorded as (x0, y0).

[0190] like Figure 7a As shown in , the user touches the color picker 530, Figure 7a The touch position in is only an illustrative example and is not limited in this application. Input sends a touch event to PenKit, and the touch event includes a touch event type and a touch position coordinate. Among them, the touch event type is a touch type (also called a Down type), and the touch position coordinate is (X2, Y2), that is, the coordinates in the coordinate system of the fourth interface 500 are (X2, Y2).

[0191] Exemplarily, PenKit sends a setOffsetLocation message to Globalwrite, wherein the setOffsetLocation message includes the position coordinates of the center point 801 of the color picker 530, namely (X0, Y0). Specifically, PenKit calls the setOffsetLocation interface function of Globalwrite to transmit the coordinate information to Globalwrite.

[0192] S311, Globlawrite returns the 9*9 area color value to PenKit.

[0193] Exemplarily, Globlawrite, in response to the received setOffsetLocation message, obtains the coordinates of the center point of the color picker 530 in the coordinate system of the fourth interface 500 as (X0, Y0). As described above, Globlawrite has obtained the bitmap corresponding to the screenshot 600 corresponding to the fourth interface 500 in S309, that is, the color value of each pixel in the screenshot 600 has been obtained. Figure 8aAs shown, Globlawrite can expand around to obtain the color values ​​of 9*9 pixels in a square shaped with the coordinates (X0, Y0) as the center point of the color picker 530 in the coordinate system of the fourth interface 500 based on the coordinates (X0, Y0) of the center point of the color picker 530. In other words, the pixel corresponding to the center point 801 is the center point of the 9*9 pixel point. In the color values ​​of the 9*9 pixels in the square, each row includes 9 pixels and each column includes 9 pixels. PenKit can assign the color values ​​of the 9*9 pixels to the color values ​​of the 9*9 pixels. Figure 8a In the square grid array shown in FIG. 8 , each square corresponds to a pixel point, and its position corresponds to the arrangement of the pixel points in the image. For example, the color (i.e., color value) of the center square of the square grid is the color value of the pixel corresponding to the center point 801, and the colors of the remaining squares correspond in sequence, which will not be described one by one here.

[0194] Combination Figure 8a , Fig. 9 For an exemplary color selection diagram, please refer to Fig. 9 , Figure 8a The position of the center point in the coordinate system is mapped to the image 600 (i.e., the screenshot 600), and the center point 801 of the color picker 530 is at Fig. 9 At the position shown, PenKit can obtain the color values ​​of 9*9 pixels centered on this point, thereby completing the color selection of point 801 in the image 600 (i.e., the position where the center of the color picker 530 is aligned) and its nearby pixels, a total of 9*9 pixels.

[0195] S312, PenKit updates the color picker.

[0196] Fig.10 For an exemplary diagram of the color picker update, please refer to Fig.10 (1), for example, after PenKit obtains the color value of the 9*9 area centered at point 801, as shown in FIG. Fig.10 As shown in (2) and (3), PenKit can cut the 9*9 square array based on the preset shape and size of the color picker (i.e., the shape and size of the inner ring of the color picker) to obtain squares that conform to the shape (e.g., circle) and size (which can be set according to actual needs and is not limited in this application) of the color picker. In other words, some of the squares on the edge of the 9*9 square may be cut off, and some of the squares may only be partially displayed.

[0197] For example, Figure 4c, the fill color in the upper half ring of the color picker displayed by PenKit is the same as the color value of the center point 801. PenKit can fill the upper half ring of the color picker based on the color value of the pixel at the center point 801, so that the color of the upper half ring is consistent with the color of the center point, so that the user can more intuitively obtain the color currently obtained by the color picker. In the embodiment of the present application, the color obtained by the color picker is the color obtained by the center point 801 of the color picker, that is, the color value of the pixel corresponding to the center point 801.

[0198] In the embodiment of the present application, the center point 801 and the pixels around it are enlarged and displayed, that is, displayed as a 9*9 array (that is, area), which can also be understood as Fig.10 The squares in the inner ring of (3) enable the user to intuitively obtain the color values ​​of other pixels near the center point 801, so that the user can drag the color picker as needed to more accurately select colors through the color picker.

[0199] In the embodiment of the present application, only a circular color picker is used as an example for illustration. In other embodiments, the color picker can also be any shape such as a rectangle, a square, a diamond, etc., which is not limited in the present application. Fig.11 For an exemplary diagram of the color picker, please refer to Fig.11 In this example, the color picker is in the shape of a square. Accordingly, PenKit can be Fig.10 In the same way, a 9*9 square (also called a region) is assigned to the inner frame of the color picker. The outer frame of the color picker can also include an upper frame and a lower half frame. The fill color of the upper half frame is the same as the color of the center of the color picker, and the lower half frame is the color obtained by the color picker last time.

[0200] Still refer to Figure 3 Before PenKit obtains the hand-raising event, PenKit executes S308 to S312 in a loop. That is to say, since the user drags the color picker 530 and does not raise his hand, Input can continuously report trigger events to PenKit (i.e., every preset period, such as 200ms, as described above, it can be set according to actual needs and is not limited in this application). Every time PenKit receives a trigger event, it can execute S310, that is, calculate the coordinates of the center point of the color picker 530 and send the coordinates of the center point to Globalwrite.

[0201] For an example, please refer to Figure 7b , for example, the user drags the color picker 530 from Figure 7a Move to the position Figure 7bWhen the user drags the color picker 530 , PenKit and Globalwrite execute S308 to S312 in a loop to update the colors of the blocks and the upper half of the ring in the color picker 530 according to the different positions of the center point of the color picker 530 .

[0202] In this example, the user moves the color picker 530 from the position 7a to Figure 7b The time interval between the positions shown is used as an example to illustrate the touch event reporting cycle. That is, at the first moment, the touch event obtained by PenKit is used to describe Figure 7a The touch position shown in the figure is used to describe the touch event obtained by PenKit at the second moment. Figure 7b The touch locations are shown.

[0203] In this example, the user always drags the color picker 530 on the desktop to pick colors. That is, when executing S308-S309, the image and its corresponding bitmap obtained by Globlawrite are the same as the image 600 and its corresponding bitmap. In other embodiments, as described above, the color picking object can also be a preview image captured by a video or camera in real time, that is, the color picking object is an image stream, and the image of each frame is the same or different, that is, the screenshots obtained by Globawrite from ScreenService in each trigger cycle (refer to the description above) may be the same or different, and this application does not limit it.

[0204] Combination Figure 7b The user interface diagram shown is Figure 8b For an exemplary schematic diagram of the position coordinates, please refer to Figure 8b , PenKit obtains the touch event reported by the Input module at the current moment (for example, the second moment), and the touch event includes but is not limited to: touch type and touch position. Among them, the touch type is the drag type, and the touch position is the coordinate corresponding to the point 804 in the coordinate system with the upper left corner of the fourth interface 500 as the origin, recorded as (X5, Y5). Based on the touch type, PenKit can determine that the current user has not raised his hand, that is, the hand-raising event has not occurred (that is, the touch type is a touch event of raising the hand), and PenKit continues to execute S310. It should be noted that the touch event can be received at any time between S308 and S309, and this application does not limit it.

[0205] Please refer to Figure 8bFor example, PenKit can obtain the displacement offset of the current touch point 804 as (△X, △Y) based on the coordinates of the touch position (X5, Y5) and the position coordinates corresponding to the touch event obtained at the last moment (i.e., the first moment, which can also be understood as the reporting cycle of the last touch event), i.e., (X2, Y2). In other words, the displacement offset of the current touch point 804 and the last touch event (i.e., Figure 8a The offset between the touch points in the touch event shown in FIG. 1 is △X on the X-axis, and the offset on the Y-axis is △Y. It can be understood that when the user drags the color picker 530 between two moments (i.e., the reporting moment of the touch event, which can also be understood as within a cycle), the dragging distance on the X-axis is △X, and the dragging distance on the Y-axis is △Y.

[0206] For example, PenKit can be based on the displacement offset of the touch point 804 being (△X, △Y) and the upper left corner (ie, Figure 8a The coordinates of the upper left corner point 802 of the square tangent to the color picker 530 in the fourth interface 500 are obtained to obtain the coordinates of point 802 at the current moment (i.e., the second moment) (i.e., the coordinate system with the upper left corner of the fourth interface 500 as the origin). In other words, when the user drags the color picker 530, the upper left corner point 802 moves with the touch point dragged by the user, and its offset on the X-axis and Y-axis is the same. Therefore, PenKit can calculate the coordinates of point 802 at the current moment based on the coordinates of point 802 obtained last time and the offsets of touch point 801 on the X-axis and Y-axis, recorded as (X4, Y4).

[0207] Still refer to Figure 7b As mentioned above, PenKit acquires the trigger event in the last cycle (i.e., the last time, such as Figure 7a As shown in the figure, the coordinates of the center point 801 of the color picker 530 in the coordinate system with the point 802 as the origin are obtained, which are recorded as (x0, y0). Since the coordinates of the center point 801 are the coordinates in the coordinate system with the point 802 as the origin, the center point 801 moves arbitrarily on the fourth interface 500, and its coordinates in the coordinate system with the upper left corner of the fourth interface 500 as the origin are transformed, but its coordinates in the coordinate system with the point 802 as the origin remain unchanged, that is, they are still (x0, y0). Accordingly, PenKit can calculate the coordinates of the center point 801 in the coordinate system with the upper left corner of the fourth interface 500 as the origin (also called relative position coordinates), recorded as (X3, Y3), based on the current coordinates of the point 802 (i.e. (X4, Y4)) and the coordinates of the center point 801 in the coordinate system with the point 802 as the origin (i.e. (x0, y0)).

[0208] Accordingly, PenKit sends setOffsetLocation information to Globalwrite, wherein the setOffsetLocation information includes the position coordinates of the center point 801 of the color picker 530 (ie Figure 7b The position coordinates in the image are (X3, Y3).

[0209] Exemplarily, Globlawrite responds to the received setOffsetLocation information and obtains the coordinates of the center point 801 of the color picker 530 in the coordinate system of the fourth interface 500 as (X5, Y5). As described above, Globlawrite has obtained the bitmap corresponding to the screenshot 600 corresponding to the fourth interface 500 in S309, that is, the color value of each pixel in the screenshot 600 has been obtained. Figure 8b As shown, Globlawrite can expand around to obtain the color value of 9*9 pixels in a square shaped with the coordinates (X5, Y5) as the center point of the color picker 530 in the coordinate system of the fourth interface 500 based on the coordinates (X5, Y5) of the center point of the color picker 530. In other words, the pixel corresponding to the center point 801 is the center point of the 9*9 pixel point. In the color value of the 9*9 pixel point of the square, each row includes 9 pixels and each column includes 9 pixels. PenKit can assign the color value of the acquired 9*9 pixel point to Figure 8b In the square grid array shown in FIG. 8 , each square corresponds to a pixel point, and its position corresponds to the arrangement of the pixel points in the image. For example, the color (i.e., color value) of the center square of the square grid is the color value of the pixel corresponding to the center point 801, and the colors of the remaining squares correspond in sequence, which will not be described one by one here.

[0210] Next, PenKit obtains the color value of the 9*9 area centered at point 801, such as Fig.10 As shown in (2) and (3), PenKit can cut the 9*9 square array based on the preset shape and size of the color picker (i.e., the shape and size of the inner ring of the color picker) to obtain squares that conform to the shape (e.g., circle) and size (which can be set according to actual needs and is not limited in this application) of the color picker. In other words, some of the squares on the edge of the 9*9 square may be cut off, and some of the squares may only be partially displayed.

[0211] For example, Figure 4c, the fill color in the upper half ring of the color picker displayed by PenKit is the same as the color value of the center point 801. PenKit can fill the upper half ring of the color picker based on the color value of the pixel at the center point 801, so that the color of the upper half ring is consistent with the color of the center point, so that the user can more intuitively obtain the color currently obtained by the color picker. In the embodiment of the present application, the color obtained by the color picker is the color obtained by the center point 801 of the color picker, that is, the color value of the pixel corresponding to the center point 801.

[0212] In the embodiment of the present application, the center point 801 and the pixels around it are enlarged and displayed, that is, displayed as a 9*9 array (that is, area), which can also be understood as Fig.10 The squares in the inner ring of (3) enable the user to intuitively obtain the color values ​​of other pixels near the center point 801, so that the user can drag the color picker as needed to more accurately select colors through the color picker.

[0213] The user can continue to select colors in the fourth interface 500 by dragging the color picker 530 , and PenKit and Globalwrite repeat S308 - S312 .

[0214] In one possible implementation, if the user's touch operation acts on a point outside the color picker 530, the tablet responds to the received user operation and displays the color picker 530 at the position clicked by the user. Specifically, Penkit can obtain the corresponding touch position in response to the touch event reported by the Input module. Penkit can make the color picker 530 disappear from the current position and display it at the touch position, and continue to perform subsequent operations, that is, the user can continue to drag the color picker 530 from this position to pick colors, or the user can also raise his hand to complete the color picking, which is not limited in this application. Optionally, PenKit can also add animation effects, such as jumping or fading in and out, in the process of moving the color picker 530 from the current position (such as the center of the screen) to the user's touch position, which is not limited in this application.

[0215] S313, PenKit obtains the hand raising event and cancels the display of the color picker.

[0216] Exemplarily, during the process of PenKit and Globalwrite repeatedly executing S308 to S312, if PenKit detects a hand-raising event, PenKit cancels display of the color picker and returns to selecting a color (ie, executing S314).

[0217] Fig.12a For an exemplary user interface diagram, please refer to Fig.12a , users from Figure 7a Drag the color picker 530 to the position shown Fig.12a The color values ​​of the blocks in the color picker 530 and the upper half ring change as the user drags the color picker 530. The specific implementation can be referred to above and will not be repeated here. Fig.12a After reaching the position shown, raise your hand to instruct the color picker 530 to pick a color at the current position and attach the selected color to the color wheel (or drawing tool, such as a brush).

[0218] For example, after the user raises his hand, the touch sensor detects that the user has raised his hand, and the input module can receive the touch event reported by the touch sensor. The touch event includes the touch event type and the touch position. The touch event type is a hand raising event (i.e., an up event), and the touch position is as follows: Fig.12a As shown. The input module PenKit reports the touch event. Based on the touch event, PenKit can determine that a hand-raising event has occurred. It should be noted that PenKit may monitor the hand-raising event at any time during the execution of S308 to S312. After PenKit monitors the hand-raising event, it can stop the currently executed step. Figure 12b As shown, PenKit cancels the display of the color picker 530 at the current position (i.e., point 1201). Point 1201 is the position corresponding to the center point of the color picker 530. Accordingly, PenKit obtains the color (i.e., color value) selected by the center point of the color picker 530 last obtained before the hand-raising event occurs, for example, the color (i.e., color value) of the pixel corresponding to the position of point 1201. Figure 12b As shown, before the user raises his hand, the center point of the color picker 503 moves to point 1201, and PenKit picks the color of this point. The specific color picking process can be referred to the relevant content above, and will not be repeated here. For example, when the color picker 530 is at this point (i.e., point 1201), its color display is as follows Figure 12b As shown, the color of the upper half ring 1221 in the outer ring 1210 of the color picker 530 is the same as the color value of the pixel corresponding to the center point 1220, that is, the color value of the pixel corresponding to the point 1201. The lower half ring 1222 is filled with the default color or the color selected in the last color picker. The squares in the inner ring 1220 show the colors of the pixels around the pixel corresponding to the point 1201. At the current moment, that is, after the user raises his hand, the color selected by PenKit is Figure 12b The color selected by the center point 1230 is the color value of the pixel corresponding to the point 1201. In addition, PenKit cancels the display of the color picker 530.

[0219] It should be noted that the various touch events described in the embodiments of the present application are merely illustrative examples and are not limited in this application.

[0220] S314, PenKit releases touch monitoring.

[0221] Exemplarily, PenKit sends a touch monitoring release instruction to the Input module to release the touch monitoring. In response to the received touch monitoring release instruction, the Input module no longer reports the touch event to PenKit. It should be noted that the Input module still obtains the touch event reported by the touch sensor, but it no longer reports it to PenKit.

[0222] And, as mentioned above, after PenKit registers touch monitoring with the Input module, the touch event is locked, that is, the Input module only reports touch events to Penkit. After PenKit releases touch monitoring, the Input module can report the received touch events to other applications or services that registered touch monitoring, thereby unlocking the touch event.

[0223] S315, PenKit returns the selected color to the memo application.

[0224] For example, PenKit selects a color (e.g. Figure 12b After that, PenKit sends color value indication information to the application. The color value indication information includes the color value, that is, the color value of the pixel corresponding to the point 1201, which can also be understood as Figure 12b The color value selected (ie displayed) by the center point 1230 of the color picker 530 .

[0225] The memo application obtains the corresponding color value in response to the received color value indication information. Fig.13 For an exemplary user interface diagram, please refer to Fig.13 After the memo application obtains the color value, it fills the color option 1301 (corresponding to the color option 422 in FIG. 4 ) with the selected color, that is, the obtained color value. Correspondingly, the colors of the current brushes and dyeing tools are synchronously updated to the color value. Fig.14 As shown, the user uses a stylus to use a crayon to color the image, and the color of the color is the color obtained by the memo application from PenKit.

[0226] S315, Penkit and Globalwrite are untied.

[0227] S316, Globalwrite closes ScreenShotService (screen shot service, referred to as screen shot service).

[0228] Exemplarily, PenKit sends an unbindService message to Globalwrite to request to unbind the service from Globalwrite. Specifically, PenKit calls the unbindService interface function of Globalwrite to unbind the service from Globalwrite.

[0229] In the embodiment of the present application, the unbinding service can also be understood as deleting (or releasing) the data channel with Globalwrite so that the two modules (or components) no longer interact with each other. The unbinding service can also be understood as Globalwrite no longer providing corresponding services for Penkit.

[0230] Exemplarily, in response to the received unbindService message, Globalwrite closes ScreenShotService. Specifically, Globalwrite stops running the ScreenShotService method function to close ScreenShotService, which no longer triggers ScreenService to perform screen capture, thereby avoiding power consumption waste.

[0231] In a possible implementation, the color palette, color ring, and color value in the memo application are synchronously updated based on the color selected by the color picker 530 . Figure 15a to Figure 15c Schematic diagram showing the display of color palette, color ring and color value. Fig.15a , after the memo application displays the color picking option 422 as the color selected by the color picker 530, if the user clicks on the color picking option 422, the tablet responds to the received user operation and displays the color picking box 430 in the memo application interface in the fifth interface 404. The color picking box 430 displays the color palette and other controls corresponding to the color palette option 432. The color palette 435 includes a plurality of blocks (it can also be other graphics, which is not limited in this application), and each row of blocks (or each column of blocks) belongs to the same color system, and the color value of the blocks in each row or column gradually decreases or increases, which is not limited in this application. In other words, the color (i.e., color value) corresponding to each block is different. Among them, the color in the block framed by the color selection box 435 is the color selected by the color picker 530.

[0232] like Fig.15b As shown, the user can click on the color wheel option 433, and the tablet responds to the received user operation and displays the color wheel 1501 in the color selection box 430 in the sixth display interface 405. The color wheel 1501 includes multiple color blocks, each color block corresponds to a color value, and the color value of each color block is different. The color in the block framed by the color selection box 435 is the color selected by the color picker 530.

[0233] like Fig.15c As shown, the user can click on the color ring option 433, and the tablet responds to the received user operation by displaying the color value 1503 in the color picking box 430 in the seventh display interface 406, wherein the color value 1503 includes RGB (red, green, blue) color values, and the RGB color values ​​represent the color selected by the color picker 530.

[0234] In another possible implementation, as described above, the user triggers the global color picking function again, and the color picker is displayed at the position where the last color picking ended. Fig.16 As shown, the eighth interface 1600 includes but is not limited to a desktop 1601 (it may also be other objects, which is not limited in this application) and a window 1602, in which a memo application interface is displayed. The user can trigger the global color picking function again. The specific triggering method can refer to the above embodiment, which is not limited in this application. The tablet responds to the received user operation and obtains the last (for example, Fig.12a In the embodiment of the present application, after executing S313, the tablet can record the stop position of the color picker, that is, Fig.12a The position shown, for example, the coordinates of point 1201. Fig.16 In the scene shown, the tablet detects that the global color picker function is called again in the memo application, and the tablet obtains the last recorded position, which is the coordinate of point 1201. The tablet displays a color picker 1603 at this position. The center point 1630 of the color picker 1603 is at the coordinate corresponding to point 1201.

[0235] Optionally, the upper half ring 1611 of the outer ring 1610 of the color picker 1603 displays a default color, such as white, and the lower half ring 1612 displays the color selected by the color picker last time, that is, Fig.12a or Figure 12b Each square in the inner ring 1620 (including the center 1630) is a default color, such as white.

[0236] In another possible implementation, as described above, the color picking object in the embodiment of the present application can be any application or video stream, etc., and can also be a preview image captured by a camera. Fig.17a For an exemplary user interface diagram, please refer to Fig.17a, the user can turn on the camera in advance, and the ninth interface 1700 can display a preview interface 1710, in which the preview image captured by the camera in real time is displayed. Exemplarily, the preview interface 1710 also includes but is not limited to a shooting button 1701. The user can trigger the global color picking function in the memo application in the manner described above, and the tablet responds to the received user operation and displays a color picker 1730 on the ninth interface 1700. Among them, when the color picker 1730 is in the current position (for example, the first position), the color picked by the center 1733 of the color picker 1730 is white (that is, the color of the pixel covered by the center 1733 is white, which is only an illustrative example and not limited to), and accordingly, the color displayed by the upper half ring 1731 of the color picker 1730 is consistent with the color picked by the center 1733, that is, white. In addition, the lower half ring 1732 displays the color picked last time, for example, black (also an illustrative example).

[0237] Exemplarily, the user can drag the color picker 1730 in the ninth interface 1700 to pick colors in the ninth interface 1700. During the color picking process, that is, when executing S308, Globalwrtie sends a getScreenShot request to ScreenService, and ScreenService takes a screenshot of the currently displayed fourth interface 500, which includes a preview image captured by the camera (which can also be understood as the current frame in the image stream) and the image corresponding to window 1720. PenKit can pick colors in the acquired screenshot based on the position of the color picker, thereby realizing color picking at any position in the image stream, allowing users to obtain the colors they need in reality through the camera, and color the images in the memo application based on the selected colors.

[0238] For example, when the color picker stops at a position, the color of the center point 1733 of the color picker 1730 and other squares in its inner ring and the upper half ring 1731 will change as the image content of the object to be picked changes. Fig.17b As shown, the user touches the color picker 1730 and stops it at the current position, which is the same as Fig.17aThe position in is the same (for example, the first position). Correspondingly, as the preview image captured by the camera changes, the color of the pixels covered by the center point 1733 in the color picker 1730 changes accordingly, for example, displayed as blue (shown as gray in the figure). In addition, the colors of other squares in the inner ring also change with the color changes of the corresponding pixels. And, the upper half ring 1731 is also displayed as blue with the center point 1733. The specific color picking process can be referred to above and will not be repeated here. In other words, when the color picker 1730 stops at the current position, PenKit and Globalwrite repeatedly execute S307 to S311. During the execution process, the content of the screenshot obtained by Globalwrite changes with the preview image, while the coordinate position corresponding to the touch event remains unchanged.

[0239] Optionally, as described above, after PenKit registers the touch listener with the Input module, the touch events of the screen are locked. Fig.17a and Fig.17b In the scene shown, the shooting button 1501 is disabled (for example, it can be displayed in gray to indicate that the current button is unavailable), that is, it is impossible to take a photo by clicking it until the global color picking function ends, that is, PenKit releases the touch monitoring.

[0240] Fig.17c and Fig.17d For scenes where the color objects include video images, Fig.17c For an exemplary user interface diagram, please refer to Fig.17c , the tenth interface 1701 may include a video application interface 1740, and the video application interface 1710 displays a video image (i.e., a video stream) and other controls (e.g., a play control, a fast-forward control, etc.). The user may trigger the global color picking function in the memo application in the manner described above, and the tablet may display a color picker 1750 on the tenth interface 1701 in response to the received user operation. Among them, when the color picker 1750 is in the current position (e.g., the second position), the color picked by the center of the color picker 1750 is white (i.e., the color of the pixel covered by the center is white, which is only an illustrative example and is not limited thereto). For other descriptions, please refer to Fig.17a , I will not go into details here.

[0241] For example, when the user holds down the color picker 1750 (i.e., does not lift the hand) and stops at the second position, the center point of the color picker 1750 and the other squares in its inner ring and the color of the upper half ring will change as the image content of the object to be picked changes. Fig.17d As shown, the user touches the color picker 1750 and stops it at the current position, which is the same as Fig.17cThe position in is the same (for example, the second position). Correspondingly, as the video image played in the video application interface changes, the color of the pixel covered by the center point in the color picker 1750 changes accordingly, for example, it is displayed as blue (shown as gray in the figure). And, the colors of other squares in the inner ring also change with the color changes of the corresponding pixels. And, the upper half ring is also displayed as blue along with the center point. The specific color picking process can be referred to above and will not be repeated here. That is to say, when the color picker 1750 stops at the current position (ie, the second position), PenKit and Globalwrite repeatedly execute S307 to S311. During the execution process, the content of the screenshot obtained by Globalwrite changes with the video image, while the coordinate position corresponding to the touch event remains unchanged.

[0242] Optionally, as described above, after PenKit registers the touch listener with the Input module, the touch events of the screen are locked. Fig.17a and Fig.17b In the scene shown, the shooting button 1501 is disabled (for example, it can be displayed in gray to indicate that the current button is unavailable), that is, it is impossible to take a photo by clicking it until the global color picking function ends, that is, PenKit releases the touch monitoring.

[0243] In another possible implementation, as described above, the method in the embodiment of the present application can also be applied to other third-party applications. Figure 18a to Figure 18b For an exemplary user interface diagram, please refer to Fig.18a , the eleventh interface 1800 includes but is not limited to: desktop 1801 and window 1802. Among them, the document application (which can be a third-party application) interface is displayed in window 1802. The document application interface includes but is not limited to: start option, insert option 1810, view option and review option, etc. The user can click on the insert option 1810, and the tablet responds to the received user operation and displays the insert object option box 1820 in the document application interface. The insert object option box 1820 includes but is not limited to: annotation option 1821, blank page option, etc. The user can click on the annotation option 1821, and the tablet responds to the received user operation and displays the insert annotation option box 1830. The insert annotation option box 1830 includes but is not limited to: input box and handwriting insertion option 1831. As Fig.18bAs shown, the user clicks on the handwriting insert option 1831, and the tablet responds to the received user operation and displays an add ink comment box 1840 in the document application interface. Specifically, in this example, the application layer of the tablet can set an add ink comment module (or service), which can provide a calling interface for other applications (such as document applications). The document application responds to the received user click on the handwriting insert option 1831 operation, and can call the add ink comment module. The add ink comment module responds to the call of the document application and provides it with the add ink comment function. That is, the add ink comment box 1840 is displayed in the document application interface. The add ink comment box 1840 includes but is not limited to: an input tool box 1850. The input tool box 1850 includes but is not limited to: input tools and function controls (or function options). Among them, the input tools include but are not limited to some different types of brushes (or brushes) or coloring tools, such as crayon brushes (also known as crayon coloring tools or simply crayons, which are not limited in this application), pens, pencils, markers, erasers, etc. Exemplarily, the function controls (also referred to as function options) include but are not limited to: typing options, color picking options 1860 (also referred to as color palette options, which are not limited in this application) and more options. The user can click on the color picking option 1860, and its specific implementation method can refer to the process in the above embodiment, which will not be repeated here. It should be noted that in this example, the call to PenKit is implemented by adding the ink annotation module, that is, Figure 3 The process executed by the application in is executed by the ink annotation module.

[0244] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0245] In one example, Fig.19 A schematic block diagram of a device 1900 according to an embodiment of the present application is shown. The device 1900 may include: a processor 1901 and a transceiver / transceiver pin 1902 , and optionally, a memory 1903 .

[0246] The various components of the device 1900 are coupled together via a bus 1904, wherein the bus 1904 includes a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, various buses are referred to as bus 1904 in the figure.

[0247] Optionally, the memory 1903 may be used for the instructions in the aforementioned method embodiment. The processor 1901 may be used to execute the instructions in the memory 1903, and control the receiving pin to receive a signal, and control the sending pin to send a signal.

[0248] The device 1900 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0249] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0250] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0251] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0252] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0253] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0254] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for displaying color, characterized in that: Applied to electronic equipment, the method comprises: At a first time, a first interface is displayed, the first interface includes a second interface of a first application and a third interface of a second application, the second interface is a part of the first interface, the second interface includes a first icon, and the color of the first icon is a first color; At a second time, in response to receiving a first operation on the second interface, the color picker control is displayed at a first position of the first interface; In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, the second position is outside the display area of ​​the second interface, and a second color is displayed in the color picker control, the second color being the color of a pixel corresponding to the second position in the first interface; At a third time, in response to a received third operation, the color picker control is not displayed, and the color of the first icon is changed to the second color.

2. The method according to claim 1, characterized in that The method further comprises: At a fourth time, in response to receiving the first operation on the second interface, a color picker control is displayed at a third position of the first interface; In response to receiving a pressing operation on a second icon in the first interface, the color picker control is displayed at a fourth position of the first interface, and a third color is displayed in the color picker, where the third color is the color of a pixel corresponding to the fourth position in the first interface; At a fifth time, in response to the received fourth operation, the color picker control is not displayed, and the color of the first icon is changed to the third color.

3. The method according to claim 2, characterized in that The method further comprises: At the sixth time, in response to receiving a click operation on the second icon, a fourth interface is displayed, where the fourth interface includes the second interface and does not include the color picker control.

4. The method according to claim 1, characterized in that The second application is a video application, the third interface is a video playback interface, the video playback interface includes a video playback window, and the video image is played in the video playback window; In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, including: In response to receiving a second operation on the color picker control, the color picker control is displayed at the second position of the first interface, the second position is within the video playback window, and the color displayed by the color picker control is the color of a pixel corresponding to the second position within the video playback interface; The color picker control remains at the second position, and the color displayed by the color picker control changes as the color of the pixel corresponding to the second position in the played video image changes.

5. The method according to claim 1, characterized in that The second application is a camera application, and the third interface is a photo preview interface, in which a preview image captured by the camera is displayed; In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, including: In response to receiving a second operation on the color picker control, the color picker control is displayed at the second position of the first interface, the second position is within the photo preview interface, and the color displayed by the color picker control is the color of a pixel corresponding to the second position within the photo preview interface; The color picker control remains at the second position, and the color displayed by the color picker control changes as the color of the pixel corresponding to the second position in the preview image changes.

6. The method according to claim 1, characterized in that The color picker control includes an inner ring area and an outer ring area, wherein the inner ring area includes a first color picking area and at least one second color picking area, and the outer ring area includes a first semi-ring and a second semi-ring; When the color picker control is displayed at the second position of the first interface, the first color picking area displays the second color, the at least one second color picking area displays the color corresponding to at least one pixel around the second position, the first semi-ring displays the second color, and the second semi-ring displays a preset color.

7. The method according to claim 6, characterized in that In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, including: A second operation on a fifth position within the color picker control is received, and the color picker control is displayed at the second position of the first interface; wherein a color picking center of the color picker control is located at the second position, and the fifth position deviates from the color picking center.

8. The method according to any one of claims 1 to 7, characterized in that: In response to receiving a first operation on the second interface, the color picker control is displayed at a first position of the first interface, including: The stylus tool module is called to display the color picker control, and the touch event is subscribed to the input module through the stylus tool module.

9. The method according to claim 8, characterized in that In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, including: Triggering a global writing module through the stylus tool module to obtain a first bitmap corresponding to the screenshot of the first interface; Outputting the touch event acquired by the input module to the global writing module through the stylus tool module, and acquiring a regional color value obtained by the global writing module based on the touch event and the first bitmap, wherein the regional color value includes a color of a first pixel corresponding to the second position in the first interface, and a color corresponding to at least one pixel around the first pixel; The color displayed in the color picker control is updated by the stylus tool, and the color value of the area is displayed in the color picker control.

10. The method according to claim 8, characterized in that In response to the received third operation, not displaying the color picker control includes: The stylus tool module unsubscribes from the touch event.

11. An electronic device, characterized in that: include: One or more processors, memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the following steps: At a first time, a first interface is displayed, the first interface includes a second interface of a first application and a third interface of a second application, the second interface is a part of the first interface, the second interface includes a first icon, and the color of the first icon is a first color; At a second time, in response to receiving a first operation on the second interface, the color picker control is displayed at a first position of the first interface; In response to receiving a second operation on the color picker control, the color picker control is displayed at a second position of the first interface, the second position is outside the display area of ​​the second interface, and a second color is displayed in the color picker control, the second color being the color of a pixel corresponding to the second position in the first interface; At a third time, in response to a received third operation, the color picker control is not displayed, and the color of the first icon is changed to the second color.

12. The electronic device according to claim 11, characterized in that: When the computer program is executed by the one or more processors, the electronic device performs the following steps: At a fourth time, in response to receiving the first operation on the second interface, a color picker control is displayed at a third position of the first interface; In response to receiving a pressing operation on a second icon in the first interface, the color picker control is displayed at a fourth position of the first interface, and a third color is displayed in the color picker, where the third color is the color of a pixel corresponding to the fourth position in the first interface; At a fifth time, in response to the received fourth operation, the color picker control is not displayed, and the color of the first icon is changed to the third color.

13. The electronic device according to claim 12, characterized in that: When the computer program is executed by the one or more processors, the electronic device performs the following steps: At the sixth time, in response to receiving a click operation on the second icon, a fourth interface is displayed, where the fourth interface includes the second interface and does not include the color picker control.

14. The electronic device according to claim 11, characterized in that: The second application is a video application, the third interface is a video playback interface, the video playback interface includes a video playback window, and the video image is played in the video playback window; when the computer program is executed by the one or more processors, the electronic device performs the following steps: In response to receiving a second operation on the color picker control, the color picker control is displayed at the second position of the first interface, the second position is within the video playback window, and the color displayed by the color picker control is the color of a pixel corresponding to the second position within the video playback interface; The color picker control remains at the second position, and the color displayed by the color picker control changes as the color of the pixel corresponding to the second position in the played video image changes.

15. A computer storage medium, characterized in that The method comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

16. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 10.

Images