Stylus input method, electronic device, and system

Abstract

The embodiments of the present application provide a stylus pen input method, an electronic device, and a system, which relate to the field of terminal technology. A first electronic device receives a projection interface of a first application from a second electronic device, and displays the projection interface. The first electronic device receives a first input and a second input of a stylus pen used by a user in the projection interface, and displays an updated projection interface. The updated projection interface includes a first layer and a second layer. The first layer includes a first image, the first image includes a first hand-drawn trajectory and is generated by the second electronic device based on the first input, and the second layer includes a second hand-drawn trajectory locally drawn by the first electronic device based on the second input. Since the latency for the first electronic device to locally draw the second hand-drawn trajectory is shorter than the latency for the first electronic device to obtain the first image, the second hand-drawn trajectory is superimposed and displayed on the first image including the hand-drawn trajectory, and the distance and line drawing latency between the first hand-drawn trajectory and the stylus pen can be reduced, and the writing immediacy of the stylus pen can be improved.

Description

Technical Field

[0001] Cross - reference to related applicationsThis application claims priority to Chinese Patent Application No. 202110592512.1, titled "STYLUS PEN INPUT METHOD, ELECTRONIC DEVICE, AND SYSTEM", filed with the China National Intellectual Property Administration on May 28, 2021, the entire content of which is incorporated herein by reference.

[0002] Technical Field The present invention relates to the field of terminal technologies, and in particular, to a stylus pen input method, an electronic device, and a system.

Background Art

[0003] A stylus pen, also called a stylus, is a pen - shaped tool used to input commands into an electronic device having a touch screen such as a computer screen or computer display, a mobile device, and a drawing board. A user can use the stylus pen to tap on the touch screen of the electronic device to select files, write, draw, etc., so that the user's interaction experience of writing, drawing, etc. on the electronic device is better.

[0004] Writing immediateness is an important indicator affecting the user experience of using a stylus pen. Usually, it is represented by the latency (line - drawing latency) from the moment when the stylus pen executes an input on the touch screen to the moment when the hand - drawn trajectory is displayed on the touch screen. There are multiple factors affecting the line - drawing latency. How to reduce the line - drawing latency and improve the writing immediateness of the stylus pen is a problem that needs to be solved.

Summary of the Invention

[0005] The present invention provides a stylus pen input method, an electronic device, and a system, as a result of which the line drawing latency of the stylus pen is reduced and the writing immediacy is improved.

[0006] To achieve the above object, the following technical solutions are used in the present application.

[0007] According to a first aspect, a stylus pen input method is provided. The method is that a first electronic device receives a projection interface of a first application. The first application is executed on a second electronic device. The first electronic device receives the input of the user's stylus pen through the projection interface and displays an updated projection interface. The input of the stylus pen includes a first input and a second input. The updated projection interface includes a first layer and a second layer. The first layer includes a first image. The first image includes a first hand-drawn trajectory and is generated by the second electronic device based on the first input. The second layer includes a second hand-drawn trajectory drawn by the first electronic device based on the second input.

[0008] In this method, the second hand-drawn trajectory locally drawn by the first electronic device is superimposed and displayed on the projection image including the first hand-drawn trajectory generated by the second electronic device. Since the latency of the locally drawn second hand-drawn trajectory is shorter than the latency for obtaining the projection image including the first hand-drawn trajectory, the distance between the first hand-drawn trajectory and the stylus pen can be reduced, the line drawing latency can be reduced, and the writing immediacy of the stylus pen can be improved.

[0009] According to a first aspect, the second layer is superimposed on the first layer.

[0010] According to the first aspect or any embodiment of the first aspect, the method further includes: the first electronic device obtains a second image. The second image includes a third hand-drawn trajectory generated by the second electronic device based on the second input; the first electronic device displays the second image on the first layer.

[0011] The third hand-drawn trajectory on the first layer covers the second hand-drawn trajectory on the second layer.

[0012] That is, the hand-drawn trajectory locally drawn by the first electronic device is covered by the hand-drawn trajectory drawn by the second electronic device based on the same input after a specific duration. In one embodiment, after the second hand-drawn trajectory is displayed for a preset duration, the display of the second hand-drawn trajectory stops. In another embodiment, after the third hand-drawn trajectory covers the second hand-drawn trajectory, the display of the second hand-drawn trajectory stops.

[0013] In one embodiment, the second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory. In another embodiment, the second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory.

[0014] According to the first aspect or any embodiment of the first aspect, the input period of the first input and the input period of the second input do not overlap at all or partially overlap. The length of the input period of the second input is based on the first latency and / or the second latency. The first latency includes the latency for the first electronic device to acquire an image from the second electronic device, and the second latency includes the latency for the first electronic device to generate the second hand-drawn trajectory. For example, the input period of the second input = (the first latency - the second latency), that is, the input period of the first input and the input period of the second input do not overlap at all. In this case, the second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory. As another example, the input period of the second input = the first latency, that is, the input period of the first input and the input period of the second input partially overlap. In this case, the second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory.

[0015] According to the first aspect or any embodiment of the first aspect, the method further includes the following: The first electronic device obtains drawing parameters transmitted by the second electronic device, and the first electronic device draws a second hand-drawn trajectory by using the drawing parameters. The drawing parameters include at least one of a line color parameter, a line thickness parameter, and a line texture parameter. The drawing parameters include the drawing parameters used by the second electronic device to draw the first hand-drawn trajectory. Since the drawing parameters used by the first electronic device to draw the second hand-drawn trajectory are the same as the drawing parameters used by the second electronic device to draw the first hand-drawn trajectory, the first image including the second hand-drawn trajectory and the first hand-drawn trajectory can be well fused. This brings a better visual effect.

[0016] According to the first aspect or any embodiment of the first aspect, when performing local drawing, the first electronic device predicts segments of the hand-drawn trajectory by using a stroke estimation algorithm based on the drawn hand-drawn trajectory, and combines the actually drawn hand-drawn trajectory based on the hand-drawn information with the predicted hand-drawn trajectory. The predicted hand-drawn trajectory is pre-displayed to form a visual effect of a fast line drawing display speed. In this way, the latency for the first electronic device to draw the hand-drawn trajectory can be reduced.

[0017] According to a second aspect, a stylus pen input method is provided. The method includes the following: A first electronic device displays a projection interface of a first application and receives a first input from a stylus pen. The first application is executed on a second electronic device. The first electronic device transmits information about the first input to the second electronic device and receives a first image from the second electronic device. The first image includes a first hand-drawn trajectory drawn based on the information about the first input. The first electronic device displays the first image. The first electronic device receives a second input from the stylus pen, draws a second hand-drawn trajectory based on the information about the second input, and displays the second hand-drawn trajectory. The second hand-drawn trajectory is displayed as a continuous trajectory of the first hand-drawn trajectory.

[0018] In this method, the first electronic device uses the computing power of the second electronic device to generate a projection image including the first hand-drawn trajectory, locally draw the second hand-drawn trajectory, and superimpose the locally drawn hand-drawn trajectory on the projection image generated by the second electronic device. Since the local drawing latency of the first electronic device is small, the locally drawn hand-drawn trajectory reduces the distance between the stylus pen on the screen of the first electronic device and the hand-drawn trajectory on the projection image, reduces the line drawing latency of the input of the stylus pen on the first electronic device, and as a result, improves the writing immediacy of the stylus pen.

[0019] The entire second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory, or the second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory.

[0020] In one embodiment, the first image is displayed on a first layer, the second hand-drawn trajectory is displayed on a second layer, and the second layer superimposes on the first layer.

[0021] According to the second aspect, the method further includes the following: The first electronic device transmits information regarding the second input to the second electronic device. The first electronic device receives a second image from the second electronic device. The second image includes a third hand-drawn trajectory drawn based on the information regarding the second input. The first electronic device displays the second image. The third hand-drawn trajectory of the second image covers the second hand-drawn trajectory.

[0022] That is, the hand-drawn trajectory locally drawn by the first electronic device is covered by the hand-drawn trajectory drawn by the second electronic device based on the same input after a specific duration. In one embodiment, after the second hand-drawn trajectory is displayed for a preset duration, the display of the second hand-drawn trajectory is stopped. In another embodiment, after the third hand-drawn trajectory covers the second hand-drawn trajectory, the display of the second hand-drawn trajectory is stopped.

[0023] According to the second aspect or any implementation of the second aspect, the length of the input period of the second input is based on the first latency and / or the second latency. The first latency includes the latency for the first electronic device to acquire an image from the second electronic device. The second latency includes the latency for the first electronic device to generate the second hand-drawn trajectory. For example, the input period of the second input = (the first latency - the second latency). In this case, the second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory. For example, the input period of the second input = the first latency. In this case, the second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory.

[0024] According to the second aspect or any embodiment of the second aspect, the method further includes the following: The first electronic device receives the drawing parameters of the second electronic device. The first electronic device draws a second freehand trajectory by using the drawing parameters based on the information regarding the second input. The drawing parameters include at least one of a line color parameter, a line thickness parameter, and a line texture parameter. Since the drawing parameters used by the first electronic device to draw the second freehand trajectory are the same as those used by the second electronic device to draw the first freehand trajectory, the first image including the second freehand trajectory and the first freehand trajectory can be well integrated. This brings about a better visual effect.

[0025] According to the second aspect or any embodiment of the second aspect, when performing local drawing, the first electronic device predicts segments of the freehand trajectory by using a stroke estimation algorithm based on the drawn freehand trajectory, and combines the actually drawn freehand trajectory based on the freehand information with the predicted freehand trajectory. The predicted freehand trajectory is pre-displayed to form a visual effect of a fast line drawing display speed. In this way, the latency for the first electronic device to draw the freehand trajectory can be reduced.

[0026] According to a second aspect, a stylus input method is provided. The method includes the following: A first electronic device displays a projection interface of a first application. The first application is executed on a second electronic device. The first electronic device receives a first input of a stylus pen and transmits information regarding the first input to the second electronic device. The second electronic device receives the information regarding the first input, draws a first hand-drawn trajectory based on the information regarding the first input, and generates a first image including the first hand-drawn trajectory. The second electronic device sends the first image to the first electronic device. The first electronic device receives the first image. The first electronic device displays the first image. The first electronic device receives a second input of the stylus pen and draws a second hand-drawn trajectory based on the information regarding the second input. The first electronic device displays the second hand-drawn trajectory. The second hand-drawn trajectory is displayed as a continuous trajectory of the first hand-drawn trajectory.

[0027] In this method, the first electronic device uses the computing power of the second electronic device to generate a projection image including a hand-drawn trajectory, locally draw the hand-drawn trajectory, and superimpose the hand-drawn trajectory locally drawn on the projection image generated by the second electronic device. Since the local drawing latency of the first electronic device is small, the hand-drawn trajectory locally drawn reduces the distance between the stylus pen on the screen of the first electronic device and the hand-drawn trajectory on the projection image, reduces the line drawing latency of the input of the stylus pen on the first electronic device, and as a result, improves the writing immediacy of the stylus pen.

[0028] The whole of the second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory, or the second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory.

[0029] According to a third aspect, the method further includes the following: The first electronic device transmits information regarding a second input to the second electronic device. The second electronic device receives information regarding the first input, draws a third freehand trajectory based on the information regarding the second input, and generates a second image including the third freehand trajectory. The second electronic device sends the second image to the first electronic device. The first electronic device receives the second image. The first electronic device displays the second image, and the third freehand trajectory covers the second freehand trajectory.

[0030] That is, the freehand trajectory locally drawn by the first electronic device is covered by the freehand trajectory drawn by the second electronic device based on the same input after a specific duration. In one embodiment, after the second freehand trajectory is displayed for a preset duration, the display of the second freehand trajectory is stopped. In another embodiment, after the third freehand trajectory covers the second freehand trajectory, the display of the second freehand trajectory is stopped.

[0031] According to the third aspect or any implementation of the third aspect, the method further includes the following: The second electronic device transmits drawing parameters for drawing the first freehand trajectory to the first electronic device. The first electronic device receives the drawing parameters and draws a second freehand trajectory by using the drawing parameters based on the information regarding the second input. The drawing parameters include at least one of a line color parameter, a line thickness parameter, and a line texture parameter. Since the drawing parameters used by the first electronic device for drawing the second freehand trajectory are the same as the drawing parameters used by the second electronic device for drawing the first freehand trajectory, the second freehand trajectory and the first image including the first freehand trajectory can be well integrated. This brings a better visual effect.

[0032] According to the third aspect or any embodiment of the third aspect, the method further includes the following: When any drawing parameter changes, the second electronic device determines to send the changed drawing parameter to the first electronic device. The first electronic device draws a second hand-drawn trajectory by using the changed drawing parameter based on the information regarding the second input. In this implementation, when the drawing parameter changes, the second electronic device sends the drawing parameter to the first electronic device. Since the drawing parameter used by the first electronic device to draw the second hand-drawn trajectory is the same as the drawing parameter used by the second electronic device to draw the first hand-drawn trajectory, the first image including the second hand-drawn trajectory and the first hand-drawn trajectory can be well integrated. This brings a better visual effect.

[0033] According to the second aspect, a stylus pen input method is provided. The method includes the following: The first electronic device displays a projection interface of the first application. The first application is executed on the second electronic device. The first electronic device receives a first input of the user's stylus pen in the projection interface at time t0. The first electronic device displays a first hand-drawn trajectory corresponding to the first input in the projection interface at time t1. The first electronic device refreshes the projection interface at time t2. The refreshed projection interface includes a second hand-drawn trajectory corresponding to the first input, and time t1 is earlier than time t2.

[0034] In this method, the first electronic device and the second electronic device each draw a freehand trajectory based on a first input. The latency of the freehand trajectory locally drawn by the first electronic device is smaller than the latency for acquiring a projected image from the second electronic device, and time point t1 is earlier than time point t2. Displaying the freehand trajectory locally drawn by the first electronic device on the screen of the first electronic device is faster than displaying the freehand trajectory drawn by the second electronic device. As a result, the freehand trajectory input by the stylus pen can be displayed to the user more quickly. Thereby, the line drawing latency of the stylus pen is reduced, and the writing immediacy of the stylus pen is improved.

[0035] Since both the first freehand trajectory and the second freehand trajectory are drawn based on information regarding the first input, the first freehand trajectory and the second freehand trajectory are the same line trajectory.

[0036] The first freehand trajectory locally drawn by the first electronic device and the second freehand trajectory drawn by the second electronic device are the same line trajectory, and the second freehand trajectory covers the first freehand trajectory. From time point t2, the display of the first freehand trajectory may stop. In one embodiment, the display of the first freehand trajectory stops at time point t3. For example, time point t3 is the same as time point t2. In another example, time point t3 is later than time point t2.

[0037] According to the fourth aspect or any embodiment of the fourth aspect, the first freehand trajectory is drawn by the first electronic device, and the second freehand trajectory is drawn by the second electronic device.

[0038] According to the fourth aspect or any embodiment of the fourth aspect, the first freehand trajectory is on the first layer, and the second freehand trajectory is on the second layer. In one embodiment, the first layer overlaps the second layer.

[0039] According to the fourth aspect or any embodiment of the fourth aspect, any parameter of the color, thickness, and texture of the line of the first hand-drawn trajectory is the same as or different from the corresponding parameter of the second hand-drawn trajectory.

[0040] According to the fifth aspect, an electronic device is provided that includes a display, one or more processors, a memory, and one or more computer programs. The processor is coupled to both the display and the memory, the one or more computer programs are stored in the memory, and when the electronic device operates, the processor can execute the one or more computer programs stored in the memory, and the electronic device can execute the method in any one of the implementation forms of the foregoing aspects.

[0041] Regarding the technical effects corresponding to any one of the fifth aspect or the implementation forms of the fifth aspect, refer to the technical effects of the corresponding implementation forms in the foregoing aspects. Details are not described again in this specification.

[0042] According to the sixth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (which may also be referred to as instructions or code), and when the computer program is executed by an electronic device, the electronic device can execute the method in any one of the implementation forms of the foregoing aspects.

[0043] Regarding the technical effects corresponding to any one of the sixth aspect or the implementation forms of the sixth aspect, refer to the technical effects of the corresponding implementation forms in the foregoing aspects. Details are not described again in this specification.

[0044] According to the seventh aspect, a computer program product is provided. When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method in any one of the implementation forms of the foregoing aspects.

[0045] Regarding the technical effect corresponding to any one of the seventh aspect or the implementation form of the seventh aspect, please refer to the technical effect of the corresponding implementation form in the foregoing aspect. Details will not be described again in this specification.

[0046] According to the eighth aspect, a circuit system is provided. The circuit system includes a processing circuit, and the processing circuit is configured to execute the method in any one of the implementation forms of the foregoing aspect.

[0047] Regarding the technical effect corresponding to any one of the eighth aspect or the implementation form of the eighth aspect, please refer to the technical effect of the corresponding implementation form in the foregoing aspect. Details will not be described again in this specification.

[0048] According to the ninth aspect, a chip system is provided. The chip system includes a processor and an interface circuit. The interface circuit executes a transceiver function and is configured to send instructions to the processor. When the instructions are executed by the processor, the processor can execute the method in any one of the implementation forms of the foregoing aspect.

[0049] Regarding the technical effect corresponding to any one of the ninth aspect or the implementation form of the ninth aspect, please refer to the technical effect of the corresponding implementation form in the foregoing aspect. Details will not be described again in this specification.

Brief Description of the Drawings

[0050]

Figure 1

[0051]

Figure 2

[0052]

Figure 3A

[0053]

Figure 3B

[0054]

Figure 4

[0055]

Figure 5

[0056]

Figure 6

[0057]

Figure 7A

[0058]

Figure 7B

[0059]

Figure 7C

[0060]

Figure 8A

[0061]

Figure 8B

[0062]

Figure 9

[0063]

Figure 10

[0064]

Figure 11

[0065]

Figure 12

[0066]

Figure 13

[0067] The terms used in the following embodiments are for the sole purpose of describing specific embodiments and are not intended to limit the present invention. Unless specifically defined in the context, the singular forms "one", "a", "the", "the foregoing", "this", and "the one" as used in this specification and the appended claims of this application are also intended to include forms such as "more than one". In the following embodiments of this application, it should be further understood that "at least one" and "one or more" mean one or at least two (including two). The term "and / or" is used to describe the relationship between related objects and represents that three relationships can exist. For example, A and / or B can represent the following cases: A and B can each be singular or plural, and the case where only A exists, the case where both A and B exist, and the case where only B exists. The character " / " generally indicates an "or" relationship between related objects.

[0068] References to "embodiments", "some embodiments", etc. described in this specification indicate that one or more embodiments of the present invention include the specific features, structures, or characteristics described with reference to the embodiments. Therefore, descriptions such as "in one embodiment", "in some embodiments", "in some other embodiments", and "in other embodiments" that appear in different places in this specification do not necessarily mean references to the same embodiment. Rather, unless otherwise particularly emphasized, it means "one or more embodiments, but not all embodiments". The terms "include", "contain", "have", and their variants all mean "include but are not limited to" unless otherwise particularly emphasized. The term "connection" includes direct connection and indirect connection unless otherwise specified.

[0069] In the following description, the terms "first" and "second" are for illustrative purposes only and should not be construed as indicating or implying relative importance or as implying any quantity of the technical features shown. Thus, features defined by "first" or "second" may explicitly or implicitly include one or more features.

[0070] In embodiments of this application, words such as "example" and "for example" are used to present examples, illustrations, or explanations. Any embodiment or design method described as an "example" or "for example" in the embodiments of this application should not be described as being more preferred or having more advantages than another embodiment or design method. Strictly speaking, the use of words such as "example" and "for example" is intended to present relative concepts in a specific way.

[0071] Writing immediateness is usually represented by stroke latency. The smaller the stroke latency, the better the writing immediateness, and the larger the stroke latency, the worse the writing immediateness. For example, as shown in FIG. 1, a stylus pen is used to draw a line on the screen of an electronic device. There is a latency (stroke latency) from the time when the electronic device receives the input of the stylus pen on the screen until the hand-drawn trajectory is displayed, so there is a specific distance between the position of the stylus pen on the screen and the hand-drawn trajectory. The longer the stroke latency, the longer the distance between the position of the stylus pen on the screen and the hand-drawn trajectory, and thus the poorer the writing immediateness.

[0072] There are multiple factors that affect the write immediacy. For example, in some products, the electronic device receives the input of the stylus pen on the screen of the electronic device, draws a hand-drawn trajectory based on the input information of the stylus pen, synthesizes a hand-drawn trajectory image, and displays the hand-drawn trajectory image on the screen. All factors such as the drawing latency, the synthesis latency, and the latency for transmitting the generated hand-drawn trajectory image from the processor to the screen for display affect the write immediacy. As another example, in some products, after receiving the input of the stylus pen on the screen of the electronic device, the electronic device transmits the input information of the stylus pen to the processing device for drawing via a wired or wireless connection and synthesizes an image including the hand-drawn trajectory. The processing device transmits the generated image to the electronic device, and the image including the hand-drawn trajectory is displayed on the screen of the electronic device. All factors such as the latency for transmitting the input information of the stylus pen from the electronic device to the processing device, the latency for the processing device to generate an image including the hand-drawn trajectory, and the latency for transmitting the image including the hand-drawn trajectory from the processing device to the electronic device affect the write immediacy.

[0073] For example, the stylus pen input method provided in the embodiments of the present invention can be applied to the system shown in FIG. 2. The system includes an electronic device 100 having a touch screen, a stylus pen 200, and a processing device 300. An application 1 is executed on the processing device 300. The electronic device 100 and the processing device 300 are connected by wire or wirelessly. The processing device 300 projects and displays each interface of the application 1 on the electronic device 100. Projection is a technology in which interfaces of different electronic devices are synchronously displayed in a wired (e.g., high definition multimedia interface (HDMI (registered trademark))) or wireless (e.g., Wi-Fi-based Miracast) transmission method. The electronic device 100 displays the interface of the application 1, that is, the projection interface. The electronic device 100 can further receive the input of the stylus pen 200 on the screen (touch screen) of the electronic device 100. For example, it can receive the input of the user's stylus pen 200 in the projection interface. The electronic device 100 acquires hand-drawn information such as the input coordinates, pressure sensing, and tilt angle of the stylus pen 200 on the screen, and transmits it to the processing device 300 by wire or wirelessly. The processing device 300 draws a hand-drawn trajectory on the display interface of the application 1 based on the hand-drawn information, and performs rendering and composition to generate an image including the hand-drawn trajectory. In one embodiment, the processing device 300 compresses the generated image into a video stream, and transmits the video stream to the electronic device 100. The electronic device 100 decompresses the video stream, acquires the image including the hand-drawn trajectory, and displays the image on the screen, that is, updates the projection interface.

[0074] The electronic device 100 can include a tablet computer, a graphics tablet, a portable mobile device, a handheld computer (such as a mobile phone), a notebook computer, a netbook, a personal computer (PC), a smart home device (such as a smart TV, a smart screen, or a large screen), a personal digital assistant (PDA), a wearable device (such as a smartwatch or a smart band), an in-vehicle computer, etc. This is not limited in the embodiments of the present invention.

[0075] The stylus pen 200 can include a stylus, a capacitive stylus, etc. This is not limited in the embodiments of the present invention.

[0076] The processing device 300 can include a personal computer (PC), a portable mobile device (such as a mobile phone), a handheld computer, a notebook computer, a netbook, a tablet computer, a smart TV, a smart screen, a large screen, etc. This is not limited in the embodiments of the present invention.

[0077] An example is used where the tablet computer is the electronic device 100 and the PC is the processing device 300. FIG. 3A shows an example of a system to which the stylus pen input method according to an embodiment of the present invention is applicable. The tablet computer is wirelessly connected to the PC. For example, the tablet computer and the PC are connected via wireless fidelity (Wi-Fi). The application 1 is executed on the PC, and the tablet computer is set as an input device of the PC. The PC further projects the interface of the application 1 onto the tablet computer for display. Each time after receiving the input of the user's stylus pen in the projected interface, the tablet computer transmits the hand-drawn information input this time to the PC for processing. The PC draws a hand-drawn trajectory based on the hand-drawn information, renders and synthesizes the hand-drawn trajectory and the current display interface of the application 1, and generates and displays an image including the hand-drawn trajectory. In this way, the user can use the stylus pen to draw a line on the tablet computer, change the application 1 on the PC, and input (for example, write or draw) a hand-drawn line in the display interface of the application 1.

[0078] The PC further projects an image including the hand-drawn trajectory onto the tablet computer for display. In one embodiment, the PC compresses the image including the hand-drawn trajectory into a video stream and transmits the video stream to the tablet computer. The tablet computer decompresses the video stream to obtain an image including the hand-drawn trajectory. The tablet computer displays the image including the hand-drawn trajectory. Specifically, the PC can project the image including the hand-drawn trajectory onto the tablet computer. This image including the hand-drawn trajectory is referred to as a projected image. In this way, both the tablet computer and the PC display the display interface of the application 1, and the display interface includes the hand-drawn trajectory of the stylus pen.

[0079] In this way, in one aspect, the tablet computer does not need to install and execute Application 1, but only needs to display the interface of Application 1; the tablet computer can draw an image on Application 1 by using the powerful computing ability of the PC, and the image generation speed is improved. In another aspect, the PC can receive the input of the stylus pen by using the tablet computer, so that the user can change the document, input words into the document, or draw an image in the drawing software by using the stylus pen. The user can input words, graphs, etc. by using the stylus pen on a device without a touch screen such as a PC.

[0080] It should be noted that in some embodiments, the processing device 300 may be a server instead of a terminal device. For example, as shown in FIG. 3B, the processing device 300 is a cloud server or a remote server. After receiving the input of the stylus pen, the tablet computer sends the hand-drawn information to the server. The server generates an image including the hand-drawn trajectory based on the hand-drawn information and sends the generated image to the tablet computer. The tablet computer acquires and displays the image including the hand-drawn trajectory. The tablet computer implements high-performance stylus pen writing by using the powerful computing ability of the server. This provides a better writing experience for the user.

[0081] In the above process, factors such as the latency for sending the hand-drawn information from the electronic device 100 to the processing device 300, the latency for the processing device 300 to generate an image including the hand-drawn trajectory, and the latency for sending the image including the hand-drawn trajectory from the processing device 300 to the electronic device 100 cause line drawing latency. As a result, there is a specific distance between the stylus pen and the hand-drawn trajectory on the screen of the electronic device 100. As a result, this results in an insufficient user experience with poor writing immediacy.

[0082] Embodiments of the present invention provide a stylus input method. After receiving the input of the user's stylus in the projection interface, the electronic device transmits the hand-drawn information of the stylus input to the processing device for processing. The processing device generates a projection image including a first hand-drawn trajectory based on the hand-drawn information. The electronic device further performs local drawing based on the input of the stylus to generate a second hand-drawn trajectory. After receiving the projection image from the processing device, the electronic device superimposes the second hand-drawn trajectory locally drawn on the projection image for display (i.e., displays an updated projection interface). Since the latency of the hand-drawn trajectory locally drawn by the electronic device is smaller than the latency for the electronic device to acquire the projection image, the hand-drawn trajectory locally drawn and the hand-drawn trajectory drawn by the processing device are displayed in a superimposed form, and as a result, the distance between the stylus and the hand-drawn trajectory on the screen of the electronic device can be reduced, the line drawing latency of the stylus input on the electronic device is reduced, and the writing immediacy of the stylus is improved.

[0083] For example, as shown in FIG. 4, the latency for the electronic device to acquire the projection image is 100 milliseconds (ms), and the latency of the hand-drawn trajectory locally drawn by the electronic device is 20 ms. At a certain point in time T, the hand-drawn trajectory in the projection image acquired by the electronic device from the processing device is drawn based on the input of the stylus at (T - 100 ms), and the hand-drawn trajectory locally drawn is drawn based on the input of the stylus at (T - 20 ms). The hand-drawn trajectory locally drawn reduces the distance between the hand-drawn trajectory drawn by the processing device and the stylus.

[0084] For example, FIG. 5 is a schematic diagram of the structure of an electronic device 100 according to an embodiment of the present invention.

[0085] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a display 150, an antenna, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, buttons 190, a camera 191, etc. The sensor module 180 may include a pressure sensor, a magnetic sensor, a distance sensor, a fingerprint sensor, a touch sensor, etc.

[0086] It can be understood that the structure shown in this embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In some other embodiments of this application, the electronic device 100 may include more or fewer components than those shown in the figure, or some components may be combined, or some components may be divided, or it may have a different component arrangement. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.

[0087] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. Different processing units may be independent components or may be integrated into one or more processors.

[0088] The controller can be the center and command center of the electronic device 100. The controller can generate an operation control signal based on an instruction operation code and a time series signal to control instruction fetching and instruction execution.

[0089] Memory may be further disposed in the processor 110 and is configured to store instructions and data. In some embodiments, the memory within the processor 110 is a cache. The memory can store instructions or data that have been recently used or periodically used by the processor 110. When the processor 110 needs to reuse an instruction or data, the processor can directly call the instruction or data from the memory. This avoids repeated access, reduces the latency of the processor 110, and improves system efficiency.

[0090] In some embodiments, the processor 110 may include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) interface, and / or the like.

[0091] The I2C interface is a bidirectional synchronous serial bus and includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be separately coupled to a touch sensor, a charger, a flash, a camera 191, etc. via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor via the I2C interface, whereby the processor 110 communicates with the touch sensor via the I2C bus interface to implement the touch function of the electronic device 100.

[0092] The I2S interface may be configured to perform audio communication. In some embodiments, the processor 110 may include multiple groups of I2S buses. The processor 110 may be coupled to the audio module 170 via the I2S bus and may perform communication between the processor 110 and the audio module 170.

[0093] The PCM interface may also be configured to perform audio communication, sample, quantize, and encode analog signals. In some embodiments, the audio module 170 may be coupled to the wireless communication module 160 via the PCM bus interface. Both the I2S interface and the PCM interface may be configured to perform audio communication.

[0094] The UART interface is a universal serial data bus and is configured to perform asynchronous communication. The bus can be a bidirectional communication bus. The bus converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically configured to connect the processor 110 to the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth (registered trademark) module in the wireless communication module 160 via the UART interface to implement Bluetooth (registered trademark) functions. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the UART interface to implement the function of playing music by using a Bluetooth (registered trademark) headset.

[0095] The MIPI interface can be configured to connect the processor 110 to peripheral components such as the display 150 or the camera 191. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 communicates with the camera 191 via the CSI to implement the photographing function of the electronic device 100. The processor 110 communicates with the display 150 via the DSI to implement the display function of the electronic device 100.

[0096] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be configured to connect the processor 110 to the camera 191, the display 150, the wireless communication module 160, the audio module 170, the sensor module 180, etc. Alternatively, the GPIO interface can be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0097] The USB interface 130 is an interface compliant with the USB standard specification. Specifically, it can be a mini-USB interface, a micro-USB interface, a USB type-C interface, etc. The USB interface 130 can be configured to connect a charger to charge the electronic device 100, or can be configured to perform data transmission between the electronic device 100 and a peripheral device, or can be configured to connect a headset to play audio by using the headset. Alternatively, the interface can be configured to connect to another electronic device, such as an AR device.

[0098] It can be understood that the interface relationship between the modules shown in the embodiments of the present invention is only an example for explanation and does not constitute a limitation on the structure of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can be alternatively interfaced in a manner different from that in the foregoing embodiments or by combining multiple interface methods.

[0099] The charging management module 140 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive a charging input from a wired charger via the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive a wireless charging input via a wireless charging coil of the electronic device 100. The charging management module 140 can further supply power to the electronic device by using the power management module 141 while charging the battery 142.

[0100] The power management module 141 is configured to connect the battery 142, the charge management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charge management module 140 and supplies power to the processor 110, the internal memory 121, the external memory, the display 150, the camera 191, the wireless communication module 160, and the like. The power management module 141 may be further configured to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage or impedance). In some other embodiments, the power management module 141 may alternatively be disposed within the processor 110. In some other embodiments, the power management module 141 and the charge management module 140 may alternatively be disposed within the same device.

[0101] The wireless communication function of the electronic device 100 may be implemented by using an antenna, the wireless communication module 160, a modem processor, a baseband processor, and the like.

[0102] The antenna is configured to transmit and receive electromagnetic wave signals. Each antenna within the electronic device 100 may be configured to cover one or more communication frequency bands. Different antennas may be multiplexed to improve antenna utilization. For example, the antennas may be multiplexed as diversity antennas in a wireless local area network. In some other embodiments, the antenna may be used in combination with a tuning switch.

[0103] The modem processor may include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. Thereafter, the demodulator transmits the low-frequency baseband signal obtained through demodulation to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then transmitted to the application processor. The application processor outputs an audio signal using an audio device (not limited to, for example, speaker 170A and receiver 170B), or displays an image or video using the display 150. In some embodiments, the modem processor may be an independent device. In some other embodiments, the modem processor may be independent of the processor 110 and disposed in the same device as another functional module.

[0104] The wireless communication module 160 can provide a wireless communication solution, and the wireless communication solution is applicable to the electronic device 100 and includes a wireless local area network (WLAN) (for example, a wireless fidelity (Wi-Fi) network), Bluetooth (registered trademark) (Bluetooth, BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC) technology, infrared (IR) technology, and the like. The wireless communication module 160 can be one or more components that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can further receive signals to be transmitted from the processor 110, perform frequency modulation and amplification on the signals, and convert the signals into electromagnetic waves for radiation via the antenna.

[0105] The electronic device 100 implements a display function by using a GPU, a display 150, an application processor, and the like. The GPU is a microprocessor for image processing and is connected to the display 150 and the application processor. The GPU is configured to execute mathematical and geometric calculations and render images. The processor 110 can include one or more GPUs that execute program instructions for generating or changing display information.

[0106] The display 150 is configured to display images, videos, and the like. The display 150 includes a display panel. The display panel may use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini LED, a micro LED, a micro OLED, a quantum dot light emitting diode (QLED), or the like. In some embodiments, the electronic device 100 may include one or N displays 150, where N is a positive integer greater than 1.

[0107] The electronic device 100 can implement a photographing function using an ISP, a camera 191, a video codec, a GPU, a display 150, an application processor, and the like.

[0108] The ISP is configured to process the data fed back by the camera 191. For example, during photographing, when the shutter is pressed, light is sent through the lens to the photosensitive element of the camera. The optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing to convert the electrical signal into a visible image. The ISP may further perform algorithm optimization on the noise, brightness, and complexion of the image. The ISP may further optimize parameters such as the exposure and color temperature of the photographing scenario. In some embodiments, the ISP may be disposed within the camera 191.

[0109] Camera 191 is configured to capture still images or videos. The optical image of the subject is generated through the lens and projected onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal and then transmits the electrical signal to the ISP in order to convert the electrical signal into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 100 may include one or N cameras 191, where N is a positive integer greater than 1.

[0110] The digital signal processing unit is configured to process digital signals and can process other digital signals in addition to digital image signals. For example, when the electronic device 100 selects a frequency, the digital signal processing unit is configured to perform a Fourier transform or the like on the frequency energy.

[0111] The video codec is configured to compress or decompress digital videos. The electronic device 100 may support one or more types of video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-3, and MPEG-4.

[0112] The NPU is a neural-network (NN) computing processor. By referring to the structure of a biological neural network, for example, by referring to the transmission mode between human brain neurons, it can quickly process input information and can continuously perform self-learning. Applications such as intelligent recognition of the electronic device 100, for example, image recognition, face recognition, speech recognition, and text understanding, can be implemented via the NPU.

[0113] The external memory interface 120 is configured to connect to an external memory card, such as a micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external storage card.

[0114] The internal memory 121 can be configured to store computer-executable program code. The executable program code includes instructions. The processor 110 executes the instructions stored in the internal memory 121 to perform various functional applications and data processing of the electronic device 100. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, applications required by at least one function (such as a voice playback function and an image playback function), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data and a phone book), etc. In addition, the internal memory 121 can include a high-speed random access memory and can further include a non-volatile memory, such as at least one magnetic disk storage device, a flash storage device, and a universal flash storage (UFS).

[0115] The electronic device 100 can implement audio functions, such as music playback and recording, using an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, an application processor, and the like.

[0116] The audio module 170 is configured to convert digital audio information into an analog audio signal and output it, and is also configured to convert an analog audio input into a digital audio signal. The audio module 170 may be configured to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed within the processor 110, or some functional modules of the audio module 170 may be disposed within the processor 110.

[0117] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into an audio signal. The electronic device 100 can be used to listen to music or respond to a hands-free call via the speaker 170A.

[0118] The receiver 170B, also referred to as an "earpiece", is configured to convert an electrical audio signal into an audio signal. When making a call or receiving voice information via the electronic device 100, the receiver 170B can be brought close to a person's ear to listen to the voice.

[0119] The microphone 170C, also referred to as a mike or mic, converts an audio signal into an electrical signal. When making a call or sending a voice message, the user can input an audio signal into the microphone 170C by making a sound in the vicinity of the microphone 170C through the user's mouth. The electronic device 100 may include at least one microphone 170C. In some other embodiments, the electronic device 100 may include two microphones 170C to implement a noise reduction function in addition to collecting audio signals. In some other embodiments, the electronic device 100 may alternatively be provided with three, four, or more microphones 170C for collecting audio signals, implementing noise reduction, identifying sound sources, implementing a directional recording function, and the like.

[0120] The headset jack 170D is configured to connect to a wired headset. The headset jack 170D may be a USB interface 130, or may be a 3.5 mm open terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0121] The magnetic sensor includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor. The distance sensor is configured to measure distance. The electronic device 100 can measure distance in an infrared or laser manner. In some embodiments, in a photo-taking scenario, the electronic device 100 can measure distance by using the distance sensor and perform quick focusing. The fingerprint sensor is configured to collect fingerprints. The electronic device 100 can use the characteristics of the collected fingerprints to implement fingerprint-based unlocking, application lock access, fingerprint-based photo-taking, fingerprint-based incoming call answering, and the like.

[0122] The touch sensor is also referred to as a touch panel. The touch sensor may be disposed on the display 150, and the touch sensor and the display 150 form a touch screen, which is also referred to as a "touch screen". The touch sensor is configured to detect a touch operation performed on or near the touch sensor. The touch sensor transmits the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation may be provided on the display 150. In some other embodiments, the touch sensor may alternatively be disposed on the surface of the electronic device 100 and at a position different from the position of the display 150.

[0123] The pressure sensor is configured to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor may be disposed on the display 150. There are multiple types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates made of a conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is performed on the display 150, the electronic device 100 uses the pressure sensor to detect the intensity of the touch operation. The electronic device 100 can further calculate the touch position based on the detection signal of the pressure sensor. In some embodiments, the pressure sensor is configured to detect the input position of a stylus pen, pressure sensing, etc.

[0124] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button or a touch button. The electronic device 100 can receive a button input and generate a button signal input related to user settings and function control of the electronic device 100.

[0125] Hereinafter, an example in which the tablet computer is the electronic device 100 and the PC is the electronic device 300 will be used to explain in detail the stylus pen input method according to the embodiments of the present invention.

[0126] As shown in FIG. 6, the application 1 operates on the PC. For example, the application 1 is a drawing application. The tablet computer is set as an input device of the PC. The PC further projects the interface of the application 1 onto the tablet computer for display. The user can use the stylus pen to draw a line on the tablet computer and perform freehand input on the PC. For example, the user uses the stylus pen to draw a line on the screen (projection interface) of the tablet computer and performs freehand drawing input on the display interface of the drawing application.

[0127] In one embodiment, the touch control module of the tablet computer receives the input of the stylus pen, obtains corresponding freehand information based on each input, and the freehand information includes any one or an arbitrary combination of input coordinates, pressure sensing, tilt angle, etc. The input of the stylus pen at a point (for example, a pixel point) on the screen of the tablet computer is used as the input.

[0128] For example, as shown in FIG. 7A, the user uses the stylus pen 200 to draw a line on the tablet computer 100. The input of the stylus pen 200 to the point a on the screen of the tablet computer 100 is defined as input a (the first input). The freehand information corresponding to the input a is called freehand information a (the first freehand information).

[0129] Refer to FIG. 6. The touch control module of the tablet computer transmits the hand-drawn information a to the PC via wireless communication (e.g., Bluetooth (registered trademark)). The application 1 executed on the PC receives the hand-drawn information a and draws a hand-drawn trajectory based on the hand-drawn information a. For example, the application 1 draws a hand-drawn trajectory based on the hand-drawn information a. Determine the position of the drawing point based on the input coordinates of the hand-drawn information, determine the grayscale of the drawing point based on the pressure sensing of the hand-drawn information, and determine the size of the drawing point based on the inclination angle of the hand-drawn information. The points drawn based on the hand-drawn information a are connected to the drawing points on the PC to form a first hand-drawn trajectory. For example, the first hand-drawn trajectory includes the points drawn by the PC based on the input during the process of the stylus pen moving from the point o shown in FIG. 7A to the point a. The application 1 further performs rendering and composition based on the current display interface and the first hand-drawn trajectory, and generates a first projected image including the first hand-drawn trajectory.

[0130] In one embodiment, based on the process in which the hand-drawn information draws a hand-drawn trajectory, the application 1 predicts segments of the hand-drawn trajectory by using a stroke estimation algorithm based on the drawn hand-drawn trajectory, and combines the actually drawn hand-drawn trajectory and the predicted hand-drawn trajectory. Since the predicted hand-drawn trajectory is displayed in advance, a visual effect of fast line drawing display speed is formed. In this way, the latency for drawing a hand-drawn trajectory on the PC can be reduced, that is, the latency for the PC to generate the first image can be reduced. The stroke estimation algorithm pre-estimates and pre-calculates the stroke position based on the known movement rules of the stroke. Specifically, a conventional stroke estimation algorithm in the prior art (e.g., Gaussian distribution) may be used. This is not limited in the embodiments of the present invention.

[0131] Optionally, the PC compresses the first projection image for video streaming. The PC transmits the video streaming to the tablet computer via wireless communication (e.g., Wi-Fi). The tablet computer receives the video streaming, decompresses the video streaming, and obtains the first projection image. The tablet computer displays the first projection image (the first projection image includes the first hand-drawn trajectory).

[0132] There is latency for the hand-drawn information to be transmitted from the tablet computer to the PC, the first projection image to be transmitted from the PC to the tablet computer, the PC to generate the first projection image, and the tablet computer to obtain the first projection image. Within the latency, the user continues to perform input on the display of the tablet computer by using the stylus pen. There is a specific distance between the first hand-drawn trajectory displayed on the tablet computer and the stylus pen.

[0133] According to the stylus pen input method provided in this embodiment of the present invention, the touch control module of the tablet computer receives the input of the stylus pen, and the application 2 running on the tablet computer further locally draws a hand-drawn trajectory based on the hand-drawn information input this time. In one embodiment, the application 2 draws points based on the input hand-drawn information. For example, the application 2 determines the position of the drawing point based on the input coordinates of the hand-drawn information, determines the grayscale of the drawing point based on the pressure sensing of the hand-drawn information, and determines the size of the drawing point based on the tilt angle of the hand-drawn information. For the specific method of drawing the hand-drawn trajectory by the tablet computer based on the hand-drawn information, reference may be made to the means in the prior art. This is not limited in the embodiments of the present invention.

[0134] The points drawn by the application 2 based on the current input are connected to the points drawn by the application 2 to form a second hand-drawn trajectory.

[0135] In one embodiment, the application 2 predicts segments of the hand-drawn trajectory by using a stroke estimation algorithm based on the drawn hand-drawn trajectory, where the hand-drawn information is based on the process of drawing the hand-drawn trajectory, and combines the actually drawn hand-drawn trajectory with the predicted hand-drawn trajectory. For example, as shown in FIG. 7B, by using the stroke estimation algorithm, predicted points are calculated based on the drawn points, and the predicted hand-drawn trajectory (the predicted points are connected to form the predicted hand-drawn trajectory) is pre-displayed to form a visual effect of fast line drawing display speed. In this way, the latency for the tablet computer to draw the hand-drawn trajectory can be reduced. In some embodiments, the application 2 may further reduce the latency for drawing the hand-drawn trajectory by using methods such as forced single-frame drawing and synthesis.

[0136] It can be understood that the application 2 on the tablet computer only draws the hand-drawn trajectory and does not need to generate a complete image of the display interface of the application 1. Therefore, the requirements for the computing ability of the tablet computer are low, and the tablet computer can be applicable to an electronic device having a simple system-on-a-chip (SOC).

[0137] The tablet computer superimposes the locally drawn second hand-drawn trajectory on the first projection image for display (i.e., displays the updated projection interface). In one embodiment, the display image (updated projection interface) of the tablet computer includes two layers. The lower layer is the projection layer, and the upper layer is the local layer. The local layer is suspended within the projection layer. The tablet computer uses the received first projection image as the projection layer and the locally drawn hand-drawn trajectory as the local layer. The tablet computer displays the image synthesized by the projection layer and the local layer.

[0138] The tablet computer has a first latency for acquiring a projected image and a second latency for locally drawing a hand-drawn trajectory. In one embodiment, the display image of the tablet computer at a certain time point T is obtained by synthesizing a first projected image generated by the PC based on a first input at a certain time point (T - the first latency) and a second hand-drawn trajectory drawn by the tablet computer based on a second input at a certain time point (T - the second latency).

[0139] For example, the first latency is 100 ms and the second latency is 20 ms. As shown in FIG. 7C, the tablet PC receives a first input (input a) of the stylus pen at time point (T - 100 ms), transmits hand-drawn information a corresponding to input a to the PC, and acquires the first projected image from the PC. The first projected image includes a first hand-drawn trajectory, and the first hand-drawn trajectory includes points drawn by the PC based on the input during the process of the stylus pen moving from point o to point a. Within the first latency (100 ms) for the tablet computer to acquire the first projected image, the stylus pen moves from point a to point b, and at time T, the stylus pen moves to point b.

[0140] The tablet computer receives a second input (input c) of the stylus pen at a certain time point (T - 20 ms) and locally draws a second hand-drawn trajectory based on input c. The second hand-drawn trajectory includes points drawn by the tablet computer based on the input during the process of the stylus pen moving from point a to point c. It will be understood that the second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory.

[0141] The projection layer of the display image of the tablet computer at time point T is the first projected image (including the first hand-drawn trajectory), and the local layer is the second hand-drawn trajectory. The second hand-drawn trajectory drawn locally reduces the distance between the first hand-drawn trajectory and the stylus pen, reduces the line drawing latency, and improves the writing immediacy of the stylus pen.

[0142] In one embodiment, application 2 determines the length of the second freehand trajectory based on the first latency and the second latency. For example, a tablet computer receives a second input of a stylus pen at time (T - second latency), and connects points drawn based on the inputs during the duration of (first latency - second latency) before the second input to form a second freehand trajectory.

[0143] For example, as shown in FIG. 7C, a tablet computer receives an input c of a stylus pen at a certain time (T - 20 ms), and connects the points drawn based on the inputs within 80 ms (100 ms - 20 ms) before the input c (i.e., the inputs between point a and point c) as the second freehand trajectory. The length of the second freehand trajectory is the length of the trajectory between point a and point c. In this implementation, the second freehand trajectory is a continuous trajectory of the first freehand trajectory, and the second freehand trajectory does not coincide with the first freehand trajectory.

[0144] It will be understood that in some other embodiments, the length of the second freehand trajectory may be longer than the length of the trajectory between point a and point c. For example, the points drawn based on the inputs within 150 ms before the input c (the inputs between point d and point c in FIG. 7C) are connected as the second freehand trajectory. In another example, the points drawn based on the inputs between input o and input c are connected as the second freehand trajectory. This is not limited in the embodiments of the present invention. In this embodiment, the second freehand trajectory coincides with a part of the first freehand trajectory, and the second freehand trajectory includes a continuous part of the first freehand trajectory. As a continuation of the first freehand trajectory, it can be understood that the second freehand trajectory may alternatively coincide completely with the first freehand trajectory.

[0145] Note that the correspondence between the projected image acquired from the PC and the hand-drawn trajectory locally drawn when the tablet computer composes the display image is not limited in the embodiments of the present invention. For example, in some other embodiments, the tablet computer acquires the projected image at time point T, acquires the hand-drawn trajectory locally drawn at time point (T + t) (t > 0), and composes the projected image acquired at time point T and the hand-drawn trajectory locally drawn at time point (T + t) on the display image for display. In some other embodiments, the tablet computer acquires the hand-drawn trajectory locally drawn at time point T, acquires the projected image at time point (T + t), where t > 0, and composes the locally drawn hand-drawn trajectory acquired at time point T and the projected image acquired at time point (T + t) on the display image for display.

[0146] For example, FIG. 8A shows a specific example of the stylus pen input method according to an embodiment of the present invention. As shown in FIG. 8A, the tablet computer 100 displays the display interface of the drawing application, and at the first time point, receives the input of point a on the screen by the stylus pen 200 to acquire the hand-drawn information a. The tablet computer 100 transmits the hand-drawn information a to the PC 300. The PC 300 draws on the display interface 710 of the drawing application based on the hand-drawn information a, and the current drawing point and the drawn point are connected to form the hand-drawn trajectory 720. The PC 300 performs rendering and composition to generate the first projected image 730 including the hand-drawn trajectory 720. The PC 300 compresses the first projected image 730 into a video stream and transmits the video stream to the tablet computer 100. The tablet computer 100 receives the video stream and acquires the first projected image 730.

[0147] While the tablet computer 100 transmits the hand-drawn information a to the PC 300 and the tablet computer 100 receives the first projection image 730, the user continues to perform input by using the stylus pen 200, and the tablet computer 100 continues to receive the input of the stylus pen 200. For example, as shown in FIG. 8A, the stylus pen 200 moves from point a to point b. At the second time point, the tablet computer receives the input of the stylus pen 200 at point c on the screen, obtains the corresponding hand-drawn information c, and performs drawing based on the hand-drawn information c. The tablet computer 100 connects the points drawn based on the input between the input a and the input c to form a hand-drawn trajectory 740.

[0148] The tablet computer 100 displays an updated display interface of the drawing application, that is, it displays the image 750 (the hand-drawn trajectory 740 is superimposed on the first projection image 730 for display). The projection layer of the image 750 is the first projection image 730, and the local layer is the hand-drawn trajectory 740.

[0149] As shown in FIG. 8B, when the hand-drawn trajectory 740 (the second hand-drawn trajectory) is displayed on the image 750, the distance between the hand-drawn trajectory 720 (the first hand-drawn trajectory) and the stylus pen 200 becomes smaller, and the line drawing latency of the tablet computer 100 becomes smaller.

[0150] Note that the interface within the dashed box in FIG. 8A is not displayed separately and is used only as an intermediate step for illustrative explanation to facilitate understanding.

[0151] Note that the PC 300 in FIG. 8A is displaying the interface 710 and the first projection image 730. It can be understood that in some other embodiments, the PC 300 may not display the interface 710 and the first projection image 730 and only provides computing capabilities for the tablet computer 100.

[0152] It can be understood that the tablet computer receives the second input (input c) of the stylus pen at time point (T - 20 ms), and further transmits the hand-drawn information c corresponding to the second input to the PC. The application 1 executed on the PC receives the hand-drawn information c, executes drawing based on the hand-drawn information c, and executes rendering and composition based on the current display interface and the drawing points of the application 1 to generate a second projection image including the hand-drawn trajectory 3. The PC compresses the second projection image into a video stream. The PC transmits the video stream to the tablet computer via wireless communication (e.g., Wi-Fi). The tablet computer receives the video stream, decompresses the video stream to obtain the second projection image. Further, the tablet computer locally overlays the hand-drawn trajectory 4 on the second projection image and displays it.

[0153] The hand-drawn trajectory 3 includes points drawn by the PC based on the input during the process of the stylus pen moving from point o to point c. The second hand-drawn trajectory includes points drawn by the tablet computer based on the input during the process of the stylus pen moving from point a to point c. After the tablet computer displays the second projection image, the hand-drawn trajectory 3 can cover the second hand-drawn trajectory.

[0154] That is, the points locally drawn by the tablet computer are covered by the points drawn by the PC based on the same input after a specific duration. In one embodiment, after displaying the hand-drawn trajectory locally drawn by the tablet computer for a preset duration, the tablet computer stops displaying the hand-drawn trajectory.

[0155] Note that in this embodiment of the present invention, an example is used for explanation in which the PC and the tablet computer perform drawing once each time an input is received. In some other embodiments, after receiving a plurality of inputs, the PC and the tablet computer may also perform drawing once based on the plurality of inputs. For example, based on the input of the stylus pen received within 10 ms, one drawing is performed. This is not limited in the embodiments of the present invention.

[0156] For example, as shown in FIG. 9, at time T0, the tablet computer 100 displays the projection interface of the PC, and the projection interface does not include a hand-drawn part. The user uses a stylus pen to draw a line on the tablet computer 100. The tablet computer 100 receives the input of the stylus pen. For example, the tablet computer uses the input of the stylus pen received within the first duration as an input, and transmits hand-drawn information corresponding to the input to the PC to draw a hand-drawn trajectory. For example, in FIG. 9, the input of the stylus pen from time T0 to time T1b is the first input. The tablet computer 100 transmits the hand-drawn information corresponding to the first input to the PC 300. The PC 300 generates a projection image including the first hand-drawn trajectory based on the first input. The tablet computer 100 acquires and displays the projection image including the first hand-drawn trajectory.

[0157] The tablet computer 100 further performs local drawing based on a second input. In one embodiment, the input period of the second input is after the input period of the first input and does not overlap with the input period of the first input at all. For example, in FIG. 9, the input of the stylus pen between time point T1b and time point T2 is the second input. The tablet computer 100 executes local drawing based on the input of the stylus pen between time point T1b and time point T2 to generate a second hand-drawn trajectory. The second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory, and the second hand-drawn trajectory does not coincide with the first hand-drawn trajectory. In another embodiment, the start time point of the input period of the second input is after the start time point of the input period of the first input, and the input period of the second input partially overlaps with the input period of the first input. For example, in FIG. 9, the input of the stylus pen between time point T1a and time point T2 is the second input, time point T1a is after time point T0 and before time point T1b. The tablet computer 100 executes local drawing based on the input of the stylus pen between time point T1a and time point T2 to generate a second hand-drawn trajectory. The second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory, and the second hand-drawn trajectory includes a continuous part of the first hand-drawn trajectory.

[0158] The tablet computer 100 displays an updated projection interface. A projection image including the first hand-drawn trajectory is displayed on the projection layer of the updated projection interface, and the second hand-drawn trajectory (local layer) is superimposed and displayed on the projection image. The second hand-drawn trajectory includes a continuous part of the first hand-drawn trajectory, as a result, the distance between the first hand-drawn trajectory and the stylus pen is reduced, and the line drawing latency of the tablet computer 100 is reduced.

[0159] In one embodiment, the tablet computer 100 determines the length of the input period of the second input based on the first latency and / or the second latency. The first latency includes the latency for the tablet computer 100 to acquire the projected image from the PC, and the second latency includes the latency for the tablet computer 100 to locally draw the second hand-drawn trajectory. In one example, the length of the input period of the second input = (the first latency - the second latency). Thus, the second hand-drawn trajectory drawn based on the second input is a continuous trajectory of the first hand-drawn trajectory drawn based on the first input, and the second hand-drawn trajectory does not coincide with the first hand-drawn trajectory. In another example, the length of the input period of the second input = the first latency. Thus, the second hand-drawn trajectory drawn based on the second input coincides with a part of the first hand-drawn trajectory drawn based on the first input, and the second hand-drawn trajectory includes a continuous part of the first hand-drawn trajectory. For example, refer to FIG. 9. The first latency = (T2 - T1a), and the second latency = (T1b - T1a). In one embodiment, the length of the input period of the second input = (the first latency - the second latency) = ((T2 - T1a) - (T1b - T1a)) = (T2 - T1b), that is, the second input is the input of the stylus pen between time point T1b and time point T2. In another embodiment, the length of the input period of the second input = the first latency = (T2 - T1a), that is, the second input is the input of the stylus pen between time point T1a and time point T2.

[0160] FIG. 10 shows an exemplary scenario of a stylus pen input method according to an embodiment of the present invention.

[0161] As shown in FIG. 10, application 1 operates on PC 300. For example, application 1 is a drawing application. Tablet computer 100 is set as an input device of PC 300. PC 300 projects the interface of application 1 onto tablet computer 100 for display. Tablet computer 100 displays the interface 810 of the drawing application. The user uses the stylus pen 200 to draw a line on the interface 810. For example, at time t0, tablet computer 100 receives an input (first input) that the stylus pen 200 moves from point a to point b within the interface 810. Tablet computer 100 transmits the information of the input that the stylus pen 200 moves from point a to point b to PC 300, and performs local drawing based on the information of the input. For example, tablet computer 100 locally draws the hand-drawn trajectory 820. At time t1, tablet computer 100 displays the hand-drawn trajectory 820 within the interface 810. Also, PC 300 receives the information of the input that the stylus pen 200 moves from point a to point b, and generates a projection image including the hand-drawn trajectory 830 based on the information. PC 300 transmits the projection image including the hand-drawn trajectory 830 to tablet computer 100. Tablet computer 100 acquires the projection image including the hand-drawn trajectory 830, refreshes the projection interface at time t2, and displays the projection image including the hand-drawn trajectory 830 in the projection layer.

[0162] It can be understood that the local layer of the refreshed projection interface includes the hand-drawn trajectory 820, the projection layer includes the hand-drawn trajectory 830, and the local layer is superimposed on the projection layer. Both the hand-drawn trajectory 820 and the hand-drawn trajectory 830 are drawn based on the input that moves the stylus pen 200 from point a to point b, and the hand-drawn trajectory 820 and the hand-drawn trajectory 830 are the same line trajectory.

[0163] Since the latency for the tablet computer 100 to perform local rendering is shorter than the latency for obtaining the projected image from the PC 300, time point t1 is earlier than time point t2. In other words, displaying the hand-drawn trajectory locally drawn by the tablet computer 100 on the screen of the tablet computer 100 is faster than displaying the hand-drawn trajectory by the PC 300. As a result, the hand-drawn trajectory input by the stylus pen can be displayed to the user more quickly. Thereby, the line drawing latency of the stylus pen is reduced, and the writing immediacy of the stylus pen is improved.

[0164] At time point t2, the tablet computer 100 obtains a projected image including the hand-drawn trajectory 830 from the PC 300 and displays the projected image in the projection layer. The hand-drawn trajectory 830 and the hand-drawn trajectory 820 are the same line trajectory, and the hand-drawn trajectory 830 covers the hand-drawn trajectory 820. From time point t2, the display of the hand-drawn trajectory 820 may stop. In some embodiments, after displaying the hand-drawn trajectory 820 locally drawn by the tablet computer 100 for a period of time, the tablet computer 100 stops displaying the hand-drawn trajectory 820 at time point t3. For example, time point t3 is the same as time point t2, or time point t3 is after time point t2.

[0165] In one embodiment, between time point t1 and time point t2, the user continues to use the stylus pen 200 to draw a line on the tablet computer 100, and the stylus pen 200 moves from point b to point c. It can be understood that the tablet computer 100 performs local rendering based on the information input by the stylus pen when the stylus pen moves from point b to point c, and displays the locally generated hand-drawn trajectory (the line segment between point b and point c in FIG. 10, represented by a dashed line to distinguish the hand-drawn trajectory 820 and the hand-drawn trajectory 830).

[0166] In some embodiments, for the same input of the stylus pen, the hand-drawn trajectory drawn on the PC is different from the hand-drawn trajectory drawn on the tablet computer. For example, the line color, line thickness, line texture, and line boundary of the two hand-drawn trajectories are different. For example, as shown in FIG. 11, the tablet computer 100 locally draws the hand-drawn trajectory 901 and obtains the hand-drawn trajectory 902 from the PC 300. The line of the hand-drawn trajectory 901 locally drawn by the tablet computer 100 is thicker than the line of the hand-drawn trajectory 902 drawn by the PC 300. Thus, the hand-drawn trajectory locally drawn on the tablet computer cannot be preferably fused with the hand-drawn trajectory on the image received from the PC.

[0167] In the stylus pen input method according to an embodiment of the present invention, the PC transmits parameters (such as line color parameters, line thickness parameters, or line texture parameters) for drawing the hand-drawn trajectory to the tablet computer, and the tablet computer draws the hand-drawn trajectory based on the parameters for drawing the hand-drawn trajectory on the PC. As a result, the hand-drawn trajectory locally drawn by the tablet computer is more preferably fused with the projection image generated by the PC.

[0168] In one embodiment, the application 3 is executed on the PC. The application 3 is configured to transmit parameters for drawing the hand-drawn trajectory by the PC to the tablet computer. In some embodiments, the application 3 is the same as the application 2 executed on the tablet computer. It can be understood that the application 3 can be different from the application 2. This is not limited in the embodiments of the present invention.

[0169] An example where the application 3 running on the PC is the same as the application 2 running on the tablet computer is used. As shown in FIG. 12, the tablet computer receives the first input of the stylus pen, acquires the first hand-drawn information, and transmits the first hand-drawn information to the PC. The PC draws the first hand-drawn trajectory based on the first hand-drawn information, performs rendering and synthesis, and generates a first projection image including the first hand-drawn trajectory. The PC compresses the first projection image into a video stream and transmits the video stream to the tablet computer via wireless communication.

[0170] The application 1 further transmits to the application 2 the parameters for drawing the first hand-drawn trajectory by the application 1. In one embodiment, each time the drawing parameters are received, the application 2 transmits the drawing parameters to the tablet computer via wireless communication. In another embodiment, when the application 2 receives the drawing parameters for the first time, the application 2 transmits the drawing parameters to the tablet computer via wireless communication, and then each time the application 2 receives the drawing parameters, the application 2 compares the received drawing parameters with the previously received drawing parameters. If it is determined that any of the drawing parameters has been changed, the changed drawing parameters are transmitted to the tablet computer by wireless communication.

[0171] The tablet computer further receives the second input of the stylus pen and acquires the second hand-drawn information. The application 2 operating on the tablet computer uses the drawing parameters received from the PC to draw a second hand-drawn trajectory based on the second hand-drawn information. The tablet computer superimposes and displays the second hand-drawn trajectory on the first projection image. Since the drawing parameters used by the tablet computer to draw the second hand-drawn trajectory are the same as the drawing parameters used by the PC to draw the first hand-drawn trajectory, the second hand-drawn trajectory can be preferably fused with the first projection image including the first hand-drawn trajectory. This brings about a better visual effect.

[0172] In another embodiment, the functions of Application 3 can be implemented by Application 1, that is, Application 1 sends the parameters used by the PC to draw a freehand trajectory to the tablet computer. This is not limited in the embodiments of the present invention.

[0173] In the stylus pen input method according to the embodiments of the present invention, the electronic device uses the computing power of the processing device to generate a projection image including a freehand trajectory, locally draw the freehand trajectory, and superimpose and display the locally drawn freehand trajectory on the projection image generated by the processing device. Since the local drawing latency of the electronic device is small, the freehand trajectory drawn locally reduces the distance between the stylus pen on the screen of the electronic device and the freehand trajectory on the projection image, reduces the line drawing latency of the input of the stylus pen on the electronic device, and as a result, improves the immediacy of writing with the stylus pen.

[0174] To implement the above functions, it can be understood that the electronic device includes a hardware structure and / or a software module for executing each corresponding function. Those skilled in the art should easily recognize that the units, algorithms, and steps can be implemented by hardware or a combination of hardware and computer software in the embodiments of the present invention in combination with the examples described in the embodiments disclosed in this specification. Whether the function is executed by hardware or by hardware driven by computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the functions described for each specific application, but the implementation should not be considered to exceed the scope of the embodiments of this application.

[0175] In an embodiment of the present invention, the electronic device may be divided into functional modules based on the foregoing method embodiments. For example, each functional module may be obtained by dividing based on each corresponding function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. It should be noted that the module division in the embodiments of the present application is an example and is merely a logical function division. During actual implementation, another division method may be used.

[0176] In an embodiment, FIG. 13 is a schematic diagram of a possible structure of the electronic device in the foregoing embodiment. The electronic device 1000 includes a processing unit 1001, a storage unit 1002, a communication unit 1003, and a display unit 1004.

[0177] The processing unit 1001 is configured to control and manage the operation of the electronic device 1000. For example, the processing unit can be configured to draw a hand-drawn trajectory based on the hand-drawn information and / or can be configured to execute another processing step in the embodiment of the present invention.

[0178] The storage unit 1002 can be configured to store the program code and data of the electronic device 1000. For example, it can be configured to store the drawing parameters obtained from the processing device.

[0179] The communication unit 1003 is configured to support the electronic device 1000 to communicate with another electronic device. For example, the communication unit may be configured to send the hand-drawn information to the processing device, receive the projection image including the hand-drawn trajectory from the processing device, and / or receive the drawing parameters from the processing device.

[0180] The display unit 1004 is configured to display the interface of the electronic device 1000. For example, the display unit can be configured to display an image including a hand-drawn trajectory.

[0181] Of course, the unit modules in the electronic device 1000 include, but are not limited to, the processing unit 1001, the storage unit 1002, the communication unit 1003, and the display unit 1004. For example, the electronic device 1000 can further include a power supply unit. The power supply unit is configured to supply power to the electronic device 1000.

[0182] The processing unit 1001 may be a processor or a controller. For example, it may be a central processing device (central processing unit, CPUs), a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application-specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or another programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The storage unit 1002 may be a memory. The communication unit 1003 may be a transceiver, a transceiver circuit, etc. The display unit 1004 may be a display.

[0183] For example, the processing unit 1001 may be a processor (processor 110 shown in FIG. 5), the storage unit 1002 may be a memory (internal memory 121 shown in FIG. 5), the communication unit 1003 may be referred to as a communication interface and includes a wireless communication module (wireless communication module 160 shown in FIG. 5), and the display unit 1004 may be a display (display 150 shown in FIG. 5, and the display 150 may be a touch screen, and the display panel and the touch panel may be integrated into the touch screen). The electronic device 1000 provided in the embodiments of the present invention may be the electronic device 100 shown in FIG. 5. The processor, the memory, the communication interface, the touch screen, etc. may be connected to each other, for example, connected via a bus.

[0184] Embodiments of the present invention further provide a computer-readable storage medium. The computer-readable storage medium stores computer program code, and when the processor executes the computer program code, the electronic device executes the method in the foregoing embodiments.

[0185] Embodiments of the present invention further provide a computer program product. When the computer program product is executed on a computer, the computer is enabled to execute the method in the foregoing embodiments.

[0186] The electronic device 1000, the computer-readable storage medium, and the computer program product provided in the embodiments of the present invention are each configured to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved by the electronic device 1000, the computer-readable storage medium, and the computer program product, reference may be made to the beneficial effects in the corresponding method provided above. Details are not described again in this specification.

[0187] Based on the description of the embodiments, those skilled in the art can clearly understand that for the purpose of a suitable and concise description, the division into the foregoing functional modules is only used as an example for explanation. During actual application, functions may be assigned to different functional modules for implementation based on requirements. In other words, the internal structure of the device is divided into different functional modules to implement all or part of the above-mentioned functions.

[0188] In some embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the described device embodiments are merely examples. For example, the division into modules or units is only a logical functional division, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the disclosed coupling, direct coupling, or communication connection shown or considered may be implemented through some interfaces. The indirect coupling or communication connection between devices or units may be implemented in electrical, mechanical, or other forms.

[0189] Furthermore, the functional units in the embodiments of this application may be integrated into one processing unit, each unit may physically exist alone, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0190] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be implemented in the form of a software product. The software product is stored in a storage medium and includes some instructions for enabling a device (which may be a single-chip microcomputer, a chip, etc.) or a processor to execute all or some of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, a removable hard disk, a ROM, a magnetic disk, or an optical disk.

[0191] The above description is only a specific implementation of the present application and is not intended to limit the protection scope of the present application. Any modification or replacement within the technical scope disclosed in the present application shall be included within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A stylus pen input method, comprising: receiving, by a first electronic device, a projection interface of a first application, wherein the first application is executed on a second electronic device; receiving, by the first electronic device, an input of a user's stylus pen in the projection interface, wherein the input of the stylus pen includes a first input and a second input; displaying, by the first electronic device, the updated projection interface, wherein the updated projection interface includes a first layer and a second layer, the first layer has a first image, the first image has a first hand-drawn trajectory and is generated by the second electronic device based on the first input, and the second layer has a second hand-drawn trajectory drawn by the first electronic device based on the second input; the length of the input period of the first input is based on a first latency; the length of the input period of the second input is based on a second latency; the first latency includes the latency for the first electronic device to acquire an image from the second electronic device; the second latency includes the latency for the first electronic device to generate the second hand-drawn trajectory; A method.

2. The second layer is superimposed on the first layer; The method according to claim 1.

3. The method further comprises: acquiring, by the first electronic device, a second image, wherein the second image includes a third hand-drawn trajectory generated by the second electronic device based on the second input; displaying, by the first electronic device, the second image on the first layer; The method according to claim 1 or 2.

4. The method further comprises: covering, by the third hand-drawn trajectory on the first layer, the second hand-drawn trajectory on the second layer; The method according to claim 3.

5. stopping displaying the second hand-drawn trajectory after the second hand-drawn trajectory is displayed for a preset duration, or stopping displaying the second hand-drawn trajectory after the third hand-drawn trajectory covers the second hand-drawn trajectory; The method according to claim 3.

6. The second hand-drawn trajectory is a continuous trajectory of the first hand-drawn trajectory, or The second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory. The method according to claim 1.

7. The input period of the first input and the input period of the second input do not overlap at all or partially overlap. The method according to claim 1.

8. Predicting segments of a hand-drawn trajectory by using a stroke estimation algorithm based on the drawn hand-drawn trajectory, and combining the actually drawn hand-drawn trajectory and the predicted hand-drawn trajectory, further including the step of displaying the predicted hand-drawn trajectory in advance. The method according to claim 1.

9. The method further includes acquiring, by the first electronic device, drawing parameters transmitted by the second electronic device, drawing, by the first electronic device, the second hand-drawn trajectory by using the drawing parameters. The method according to claim 1.

10. The drawing parameters include drawing parameters used by the second electronic device to draw the first hand-drawn trajectory. The method according to claim 9.

11. The drawing parameters include at least one of a line color parameter, a line thickness parameter, and a line texture parameter. The method according to claim 10.

12. A stylus pen input method, displaying, by a first electronic device, a projection interface of a first application and receiving a first input of a stylus pen, where the first application is executed on a second electronic device, transmitting, by the first electronic device, information about the first input to the second electronic device, receiving, by the first electronic device, a first image from the second electronic device, where the first image includes a first hand-drawn trajectory drawn based on the information about the first input, displaying, by the first electronic device, the first image, receiving, by the first electronic device, a second input of the stylus pen, drawing, by the first electronic device, a second hand-drawn trajectory based on the information about the second input, displaying, by the first electronic device, the second hand-drawn trajectory, where the second hand-drawn trajectory is displayed as a continuous trajectory of the first hand-drawn trajectory. The length of the input period of the first input is based on the first latency, The length of the input period of the second input is based on the second latency, The first latency includes the latency for the first electronic device to acquire an image from the second electronic device, The second latency includes the latency for the first electronic device to generate the second hand-drawn trajectory, Method.

13. Whether the entire second hand-drawn trajectory is the continuous trajectory of the first hand-drawn trajectory, or The second hand-drawn trajectory coincides with a part of the first hand-drawn trajectory and includes a continuous part of the first hand-drawn trajectory, The method according to claim 12.

14. The first image is displayed on a first layer, The second hand-drawn trajectory is displayed on a second layer, The second layer is superimposed on the first layer. The method according to claim 12 or 13.

15. The method further includes Transmitting, by the first electronic device, the information regarding the second input to the second electronic device, and Receiving, by the first electronic device, a second image from the second electronic device, the second image including a third hand-drawn trajectory drawn based on the information regarding the second input, and Displaying, by the first electronic device, the second image, the third hand-drawn trajectory covering the second hand-drawn trajectory, The method according to claim 12.

16. After the second hand-drawn trajectory is displayed for a preset duration, stopping the display of the second hand-drawn trajectory, or Including the step of stopping the display of the second hand-drawn trajectory after the third hand-drawn trajectory covers the second hand-drawn trajectory, The method according to claim 15.

17. Predicting segments of a hand-drawn trajectory by using a stroke estimation algorithm based on the drawn hand-drawn trajectory, and combining the actually drawn hand-drawn trajectory and the predicted hand-drawn trajectory, further including the step of displaying the predicted hand-drawn trajectory in advance, The method according to claim 12.

18. The method further includes Receiving, by the first electronic device, drawing parameters of the second electronic device, The step of drawing a second freehand trajectory based on the information regarding the second input by the first electronic device includes the step of drawing a second freehand trajectory using the drawing parameters based on the information regarding the second input by the first electronic device. The method according to claim 12.

19. An electronic device, one or more processors, a display, and a memory, wherein the memory stores one or more computer programs, the one or more computer programs include instructions, when the instructions are executed by the electronic device, the electronic device is capable of executing the method according to claim 1 or 12. Electronic device.

20. A computer-readable storage medium, wherein the computer-readable storage medium stores instructions, when the instructions are executed on an electronic device, the electronic device is capable of executing the method according to claim 1 or 12. Computer-readable storage medium.

Images