A handwriting pen input method, electronic device and system

Abstract

Embodiments of the present application provide a handwriting pen input method, an electronic device and a system, and relate to the technical field of terminals. A first electronic device receives a screen projection interface of a first application from a second electronic device and displays the screen projection interface. The first electronic device receives a first input and a second input of a user using a handwriting pen on the screen projection interface; and displays an updated screen projection interface. The updated screen projection interface includes a first layer and a second layer, the first layer includes a first image containing a first writing track generated by the second electronic device according to the first input; and the second layer includes a second writing track locally drawn by the first electronic device according to the second input. Since the time delay of the first electronic device in locally drawing the second writing track is less than the time delay of obtaining the first image, the second writing track is superimposed and displayed on the first image containing the first writing track, which can reduce the distance between the first writing track and the handwriting pen, reduce the time delay of drawing a line, and improve the handwriting pen followability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a stylus input method, an electronic device and a system. BACKGROUND

[0002] A stylus pen, also known as a touch pen, is a pen-shaped tool used to input instructions to electronic devices with touch screens, such as computer screens, mobile devices, drawing boards, etc. Users can use the stylus pen to click the touch screen of the electronic device to select files, write or draw, etc., so that the user has a better interactive experience of writing or drawing on the electronic device.

[0003] Hand tracking is a key indicator that affects the user's experience of using the stylus pen. It is usually represented by the time delay of the stylus pen input to the touch screen display writing track (line drawing time delay). There are many factors that affect the line drawing time delay. How to reduce the line drawing time delay and improve the hand tracking of the stylus pen is a problem to be solved. SUMMARY

[0004] The present application provides a stylus input method, an electronic device and a system, which can reduce the line drawing time delay and improve the hand tracking of the stylus pen.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, a stylus input method is provided. The method includes: a first electronic device receiving a screen projection interface of a first application, the first application running on a second electronic device; the first electronic device receiving a stylus input of a user on the screen projection interface and displaying an updated screen projection interface; wherein the stylus input includes a first input and a second input; the updated screen projection interface includes a first layer and a second layer, the first layer includes a first image containing a first writing track generated by the second electronic device according to the first input; and the second layer includes a second writing track drawn by the first electronic device according to the second input.

[0007] In the method, the second writing track drawn locally by the first electronic device is superimposed and displayed on the screen projection image containing the first writing track generated by the second electronic device; since the time delay of locally drawing the second writing track is less than the time delay of obtaining the screen projection image containing the first writing track, the distance between the first writing track and the stylus pen can be reduced, the line drawing time delay can be reduced, and the hand tracking of the stylus pen can be improved.

[0008] According to the first aspect, the second layer is superimposed on the first layer.

[0009] According to a first aspect, or any possible implementation mode of the first aspect, the method further comprises: obtaining, by the first electronic device, a second image; the second image comprises a third writing track generated by the second electronic device according to a second input; and displaying, by the first electronic device, the second image on the first layer.

[0010] The third writing track on the first layer covers the second writing track on the second layer.

[0011] That is, the writing track drawn locally by the first electronic device is covered by the writing track drawn by the second electronic device according to the same input after a certain time length. In one implementation mode, the second writing track stops being displayed after a preset time length. In another implementation mode, the second writing track stops being displayed after the third writing track covers the second writing track.

[0012] In one implementation mode, the second writing track is a continuation of the first writing track; in another implementation mode, the second writing track coincides with part of the first writing track and contains a continuation of the first writing track.

[0013] According to the first aspect, or any possible implementation mode of the first aspect, the input time period of the first input and the input time period of the second input are completely non-overlapping or partially overlapping. In one implementation mode, the length of the input time period of the second input is determined according to a first time delay and / or a second time delay. The first time delay comprises a time delay of the first electronic device obtaining the image from the second electronic device, and the second time delay comprises a time delay of the first electronic device generating the second writing track. For example, the input time period of the second input = (the first time delay - the second time delay), i.e., the input time period of the first input and the input time period of the second input are completely non-overlapping; in this way, the second writing track is a continuation of the first writing track. For another example, the input time period of the second input = the first time delay, i.e., the input time period of the first input and the input time period of the second input are partially overlapping; in this way, the second writing track coincides with part of the first writing track and contains a continuation of the first writing track.

[0014] According to the first aspect, or any possible implementation mode of the first aspect, the method further comprises: obtaining, by the first electronic device, drawing parameters sent by the second electronic device, and drawing, by the first electronic device, the second writing track using the drawing parameters. The drawing parameters comprise at least one of a line color parameter, a line thickness parameter, and a line texture parameter. For example, the drawing parameters comprise the drawing parameters used by the second electronic device to draw the first writing track. Since the drawing parameters used by the first electronic device to draw the second writing track are the same as the drawing parameters used by the second electronic device to draw the first writing track, the second writing track and the first image comprising the first writing track can be well integrated, bringing a better visual effect.

[0015] According to the first aspect, or any possible implementation mode of the above first aspect, when the first electronic device performs local drawing, the first electronic device predicts a segment of the writing track by using a point reporting prediction algorithm based on the drawn writing track; and combines the actually drawn writing track according to the handwriting information with the predicted writing track. The predicted writing track is displayed in advance, and a visual effect of fast line drawing is formed. In this way, the time delay of the first electronic device in drawing the writing track can be reduced.

[0016] According to a second aspect, a handwriting pen input method is provided. The method includes: a first electronic device displaying a screen projection interface of a first application, and receiving a first input of a handwriting pen, wherein the first application runs on a second electronic device. The first electronic device sends information of the first input to the second electronic device, and receives a first image from the second electronic device, the first image including a first writing track drawn according to the information of the first input. The first electronic device displays the first image. The first electronic device receives a second input of the handwriting pen, draws a second writing track according to information of the second input, and displays the second writing track. The second writing track is displayed as a continuation track of the first writing track.

[0017] In the method, the first electronic device uses the computing power of the second electronic device to generate a screen projection image containing the first writing track, and locally draws the second writing track. The locally drawn writing track is superimposed and displayed on the screen projection image generated by the second electronic device. Since the first electronic device has a small local drawing time delay, the locally drawn writing track reduces the distance between the handwriting pen on the screen of the first electronic device and the writing track on the screen projection image. The line drawing time delay of the handwriting pen input on the first electronic device is reduced, and the handwriting pen followability is improved.

[0018] In the method, the first electronic device uses the computing power of the second electronic device to generate a screen projection image containing the first writing track, and locally draws the second writing track. The locally drawn writing track is superimposed and displayed on the screen projection image generated by the second electronic device. Since the first electronic device has a small local drawing time delay, the locally drawn writing track reduces the distance between the handwriting pen on the screen of the first electronic device and the writing track on the screen projection image. The line drawing time delay of the handwriting pen input on the first electronic device is reduced, and the handwriting pen followability is improved.

[0019] In an implementation mode, the first image is displayed on a first layer, and the second writing track is displayed on a second layer, and the second layer is superimposed on the first layer.

[0020] According to the second aspect, the method further includes: the first electronic device sending information of the second input to the second electronic device; the first electronic device receiving a second image from the second electronic device, the second image including a third writing track drawn according to the information of the second input; and the first electronic device displaying the second image, and the third writing track in the second image covering the second writing track.

[0021] That is, the first electronic device locally drawn writing track will be covered by the second electronic device according to the same input drawn writing track after a certain length of time. In one implementation, the second writing track stops displaying after a preset time. In another implementation, the second writing track stops displaying after the third writing track covers the second writing track.

[0022] According to the second aspect, or any one of the implementation forms of the second aspect, the length of the input time period of the second input is determined according to the first time delay and / or the second time delay. The first time delay includes a time delay of the first electronic device obtaining the image from the second electronic device, and the second time delay includes a time delay of the first electronic device generating the second writing track. For example, the input time period of the second input = (the first time delay - the second time delay), so that the second writing track is a continuation of the first writing track. For another example, the input time period of the second input = the first time delay, so that the second writing track coincides with part of the first writing track and contains a continuation of the first writing track.

[0023] According to the second aspect, or any one of the implementation forms of the second aspect, the method further includes: the first electronic device receiving drawing parameters of the second electronic device; and the first electronic device drawing the second writing track according to the information of the second input by using the drawing parameters. The drawing parameters include at least one of a line color parameter, a line thickness parameter, and a line texture parameter. Since the drawing parameters used by the first electronic device to draw the writing track two are the same as the drawing parameters used by the second electronic device to draw the writing track one, the writing track two and the first image including the writing track one can be well fused, and a better visual effect is brought.

[0024] According to the second aspect, or any one of the implementation forms of the second aspect, when the first electronic device locally draws, a point prediction algorithm is used to predict a segment of writing track through the drawn writing track; and the actually drawn writing track according to the handwriting information is combined with the predicted writing track. The predicted writing track is displayed in advance, and a visual effect of a fast line drawing speed is formed. In this way, the time delay of the first electronic device drawing the writing track can be reduced.

[0025] In a third aspect, a handwriting pen input method is provided. The method includes: a first electronic device displaying a projection interface of a first application, wherein the first application is running on a second electronic device. The first electronic device receives a first input of a handwriting pen, and sends information of the first input to the second electronic device. The second electronic device receives the information of the first input, draws a first writing track according to the information of the first input, and generates a first image containing the first writing track. The second electronic device sends the first image to the first electronic device. The first electronic device receives the first image, and displays the first image. The first electronic device receives a second input of the handwriting pen, and draws a second writing track according to information of the second input. The first electronic device displays the second writing track, and the second writing track is displayed as a continuation track of the first writing track.

[0026] In the method, the first electronic device generates a projection image containing a writing track by using the computing power of the second electronic device, and locally draws the writing track, and superimposes the locally drawn writing track on the projection image generated by the second electronic device. Since the first electronic device has a smaller local drawing delay, the locally drawn writing track reduces the distance between the handwriting pen on the screen of the first electronic device and the writing track on the projection image, and reduces the delay of the handwriting pen input on the first electronic device, and improves the handwriting pen followability.

[0027] In one implementation, the second writing track is a continuation track of the first writing track. In another implementation, the second writing track coincides with part of the first writing track and contains a continuation part of the first writing track.

[0028] According to the third aspect, the method further includes: the first electronic device sending information of the second input to the second electronic device; the second electronic device receiving the information of the second input, drawing a third writing track according to the information of the second input, and generating a second image containing the third writing track; the second electronic device sending the second image to the first electronic device; the first electronic device receiving the second image; and the first electronic device displaying the second image, and the third writing track covering the second writing track.

[0029] That is, the writing track locally drawn by the first electronic device will be covered by the writing track drawn by the second electronic device according to the same input after a certain time. In one implementation, the second writing track stops being displayed after being displayed for a preset time. In another implementation, the second writing track stops being displayed after the third writing track covers the second writing track.

[0030] According to a third aspect, or any possible implementation mode of the third aspect, the method further comprises: the second electronic device sending drawing parameters of drawing the first writing track to the first electronic device; the first electronic device receiving the drawing parameters and drawing the second writing track according to the second input information by using the drawing parameters. The drawing parameters comprise at least one of line color parameter, line thickness parameter and line texture parameter. Since the drawing parameters used by the first electronic device to draw the second writing track are the same as the drawing parameters used by the second electronic device to draw the first writing track, the second writing track can be well fused with the first image comprising the first writing track, and better visual effect can be brought.

[0031] According to a third aspect, or any possible implementation mode of the third aspect, the method further comprises: the second electronic device determining that any one of the drawing parameters changes, sending the changed drawing parameters to the first electronic device; the first electronic device drawing the second writing track according to the second input information by using the changed drawing parameters. In this implementation mode, the second electronic device sends the drawing parameters to the first electronic device when the drawing parameters change. Since the drawing parameters used by the first electronic device to draw the second writing track are the same as the drawing parameters used by the second electronic device to draw the first writing track, the second writing track can be well fused with the first image comprising the first writing track, and better visual effect can be brought.

[0032] According to a fourth aspect, a handwriting pen input method is provided, the method comprising: a first electronic device displaying a projection interface of a first application, the first application running on a second electronic device; the first electronic device receiving a first input of a handwriting pen of a user on the projection interface at t0 moment; the first electronic device displaying a first writing track corresponding to the first input on the projection interface at t1 moment; the first electronic device refreshing the projection interface at t2 moment, the refreshed projection interface comprising a second writing track corresponding to the first input, wherein t1 moment is earlier than t2 moment.

[0033] In the method, the first electronic device and the second electronic device draw the writing track according to the first input respectively. The time delay of the first electronic device in drawing the writing track locally is less than the time delay of the first electronic device in obtaining the projection image from the second electronic device, and t1 moment is earlier than t2 moment. The writing track drawn by the first electronic device locally is displayed on the screen of the first electronic device faster than the writing track drawn by the second electronic device, which can show the writing track input by the handwriting pen to the user faster, shorten the handwriting pen delay and improve the handwriting pen hand following property.

[0034] Since the first writing track and the second writing track are both drawn according to the first input information, the first writing track and the second writing track have the same line track.

[0035] The first writing track drawn locally by the first electronic device has the same line track as the second writing track drawn by the second electronic device, and the second writing track covers the first writing track; from the t2 moment, the first writing track can be stopped from being displayed. In an implementation manner, the first writing track is stopped from being displayed at the t3 moment. For example, the t3 moment is equal to the t2 moment; for another example, the t3 moment is later than the t2 moment.

[0036] According to a fourth aspect, or any possible implementation of the fourth aspect, the first writing track is drawn by the first electronic device, and the second writing track is drawn by the second electronic device.

[0037] According to the fourth aspect, or any possible implementation of the fourth aspect, the first writing track is located in a first layer, and the second writing track is located in a second layer. In an implementation manner, the first layer is superimposed on the second layer.

[0038] According to the fourth aspect, or any possible implementation of the fourth aspect, any parameter in line color, thickness, and texture of the first writing track is the same as or different from a corresponding parameter of the second writing track.

[0039] A fifth aspect provides an electronic device, including a display screen, one or more processors, a memory, and one or more computer programs; the processor is coupled with the display screen and the memory, and the one or more computer programs are stored in the memory; when the electronic device is running, the processor can execute the one or more computer programs stored in the memory, so that the electronic device executes the method in any possible implementation of any one of the above aspects.

[0040] The technical effects of the fifth aspect and any possible implementation of the fifth aspect are the same as those of the corresponding implementation of the above aspects, which will not be repeated here.

[0041] A sixth aspect provides a computer-readable storage medium. The computer-readable storage medium stores a computer program (also referred to as instructions or code), which, when executed by an electronic device, causes the electronic device to execute the method in any possible implementation of any one of the above aspects.

[0042] The technical effects of the sixth aspect and any possible implementation of the sixth aspect are the same as those of the corresponding implementation of the above aspects, which will not be repeated here.

[0043] A seventh aspect provides a computer program product. When the computer program product is running on an electronic device, the electronic device executes the method in any possible implementation of any one of the above aspects.

[0044] The technical effects of the seventh aspect and any of the implementation manners of the seventh aspect are the same as those of the corresponding implementation manners of the above aspects, which will not be repeated here.

[0045] In an eighth aspect, a circuit system is provided. The circuit system includes a processing circuit configured to perform the method of any of the implementation manners of any of the aspects above.

[0046] The technical effects of the eighth aspect and any of the implementation manners of the eighth aspect are the same as those of the corresponding implementation manners of the above aspects, which will not be repeated here.

[0047] In a ninth aspect, a chip system is provided. The chip system includes a processor and an interface circuit. The interface circuit is configured to perform a transceiving function and send an instruction to the processor. When the instruction is executed by the processor, the processor performs the method of any of the implementation manners of any of the aspects above.

[0048] The technical effects of the ninth aspect and any of the implementation manners of the ninth aspect are the same as those of the corresponding implementation manners of the above aspects, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A scene diagram to which the handwriting pen input method provided by the embodiment of the present application is applied;

[0050] Figure 2 A system architecture diagram to which the handwriting pen input method provided by the embodiment of the present application is applied;

[0051] Figure 3A A system architecture diagram to which the handwriting pen input method provided by the embodiment of the present application is applied;

[0052] Figure 3B A system architecture diagram to which the handwriting pen input method provided by the embodiment of the present application is applied;

[0053] Figure 4 A scene example diagram of the handwriting pen input method provided by the embodiment of the present application;

[0054] Figure 5 An electronic device hardware structure diagram provided by the embodiment of the present application;

[0055] Figure 6 A handwriting pen input method diagram provided by the embodiment of the present application;

[0056] Figure 7A A scene example diagram of the handwriting pen input method provided by the embodiment of the present application;

[0057] Figure 7BA scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0058] Figure 7C A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0059] Figure 8A A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0060] Figure 8B A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0061] Figure 9 A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0062] Figure 10 A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0063] Figure 11 A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0064] Figure 12 A scene instance diagram of a handwriting pen input method provided for an embodiment of the present application;

[0065] Figure 13 A structure composition diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0066] The terms used in the following embodiments are only for the purpose of describing particular embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the application, the singular forms "a," "an" and "the" are intended to include both the singular and the plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Also, it will be understood that the term "and / or," as used herein, encompasses three possibilities: one, both, or none of the items listed are present.

[0067] Reference within this specification to "one embodiment" or "an embodiment" or "embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" within this specification do not necessarily refer to the same embodiment, although it can. The terms "including," "comprising," "having" and variations thereof herein are meant to be open-ended terms that can cover the presence of substrates, integers, steps, processes, acts, elements, and / or components of the application, but do not preclude the presence or addition of one or more other substrates, integers, steps, processes, acts, elements, components, or any other suitable materials, integers, steps, processes, acts, elements, components.

[0068] Hereinafter, the terms "first", "second", etc. are used only for the purpose of description, and should not be understood to indicate or imply relative importance or to imply that a specific technical feature indicated is limited to a certain number. Thus, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0069] In the present application, the word "exemplarily" or "for example" is used to indicate an example, illustration or description. Any embodiment or design scheme described as "exemplarily" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "exemplarily" or "for example" is intended to present the relevant concept in a specific way.

[0070] The hand following property is usually represented by a stroke delay; the smaller the stroke delay, the better the hand following property; the larger the stroke delay, the worse the hand following property. Exemplarily, as shown in FIG. 1, a stroke is drawn on the screen of an electronic device using a stylus. Since there is a delay (stroke delay) between the input of the stylus on the screen and the display of the writing track, there is a certain distance between the position of the stylus on the screen and the writing track. The larger the stroke delay, the larger the distance between the position of the stylus on the screen and the writing track, and the worse the hand following property. Figure 1

[0071] ​Many factors affect responsiveness. For example, in some products, electronic devices receive input from a stylus pen on its screen, draw the writing trajectory based on the input information, synthesize the writing trajectory image, and display the image on the screen. The drawing latency, synthesis latency, and the latency of transmitting the generated writing trajectory image from the processor to the screen all affect responsiveness. As another example, in some products, after receiving input from a stylus pen on its screen, the electronic device transmits the input information via wired or wireless connection to a processing device for drawing, synthesizing an image containing the writing trajectory; the processing device sends the generated image to the electronic device; and the screen of the electronic device displays the image containing the writing trajectory. The latency of transmitting the stylus input information from the electronic device to the processing device, the latency of the processing device generating the image containing the writing trajectory, and the latency of transmitting the image containing the writing trajectory from the processing device to the electronic device all affect responsiveness.

[0072] For example, the handwriting input method provided in this application embodiment can be used for Figure 2 The system shown includes an electronic device 100 with a touchscreen, a stylus 200, and a processing device 300. An application runs on the processing device 300. The electronic device 100 and the processing device 300 are connected via wired or wireless means. The processing device 300 projects each interface of the application onto the electronic device 100 for display. Screen projection is a technology that synchronizes the interfaces of different electronic devices via wired (e.g., High Definition Multimedia Interface, HDMI) or wireless (e.g., Wi-Fi-based Miracast) transmission. The electronic device 100 displays the interface of the application, i.e., the projected interface. The electronic device 100 can also receive input from the stylus 200 on its screen (touchscreen); for example, it can receive input from the user on the projected interface. The electronic device 100 acquires handwriting information such as the input coordinates, pressure sensitivity, and tilt angle of the stylus 200 on the screen and transmits this handwriting information to the processing device 300 via wired or wireless means. The processing device 300 draws the writing trajectory on the display interface of application one based on the handwriting information, renders and composites it, and generates an image containing the writing trajectory. In one implementation, the processing device 300 compresses the generated image into a video stream and sends the video stream to the electronic device 100. The electronic device 100 decompresses the video stream, obtains the image containing the writing trajectory, and displays the image on the screen, that is, updates the projection interface.

[0073] The electronic device 100 can include a tablet computer, a handwriting board, a portable mobile device (such as a mobile phone), a handheld computer, a notebook computer, a netbook, a personal computer (PC), a smart home device (such as a smart television, a smart screen, a large screen, etc.), a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), a vehicle-mounted computer, and the like, and the embodiments of the present application do not make any limitation in this regard.

[0074] The handwriting pen 200 can include a stylus, a capacitive pen, and the like, and the embodiments of the present application do not make any limitation in this regard.

[0075] The processing device 300 can include a personal computer (PC), a portable mobile device (such as a mobile phone), a handheld computer, a notebook computer, a netbook, a tablet computer, a smart television, a smart screen, a large screen, and the like, and the embodiments of the present application do not make any limitation in this regard.

[0076] Taking a tablet computer as the electronic device 100 and a PC as the processing device 300 as an example. Figure 3A An example of a system to which the handwriting pen input method provided by the embodiments of the present application is applied is shown. The tablet computer and the PC are wirelessly connected. For example, the tablet computer and the PC are connected through wireless fidelity (Wi-Fi). The application program one is running on the PC, and the tablet computer is set as the input device of the PC; the PC also projects the interface of the application program one to the tablet computer for display. The tablet computer sends the handwriting information of the current input to the PC for processing after receiving the input of the handwriting pen of the user on the projection interface each time. The PC draws the writing track according to the handwriting information, and renders and composites the writing track with the current display interface of the application program one, to generate and display an image containing the writing track. In this way, the user can use the handwriting pen to draw a line on the tablet computer, to realize the modification of the application program one on the PC and the input of the handwriting line (such as writing, drawing, etc.) on the display interface of the application program one.

[0077] The PC also projects the image containing the writing track to the tablet computer for display. In an implementation, the PC compresses the image containing the writing track into a video stream, and transmits the video stream to the tablet computer. The tablet computer decompresses the video stream to obtain the image containing the writing track. The tablet computer displays the image containing the writing track. That is, the PC projects the image containing the writing track to the tablet computer. The image containing the writing track is referred to as a projection image. In this way, both the tablet computer and the PC display the display interface of the application program one, which includes the writing track of the handwriting pen.

[0078] In this way, on the one hand, the tablet computer does not need to install and run application one, but only displays the interface of application one; the tablet computer can use the PC's strong computing power to draw images on application one, thus improving the image generation speed. On the other hand, the PC can receive handwriting input from the tablet computer, allowing users to use the stylus to modify documents, input text in documents, or draw in drawing software. Users can use the stylus to input text and graphics by hand on devices such as PCs that do not have a touchscreen.

[0079] It should be noted that in some embodiments, the processing device 300 may not be a terminal device, but a server. For example, as Figure 3B As shown, the processing device 300 is a cloud server or a remote server. After receiving input from the stylus, the tablet computer transmits the handwriting information to the server. The server generates an image containing the writing trajectory based on the handwriting information and transmits the generated image to the tablet computer. The tablet computer acquires and displays the image containing the writing trajectory. The tablet computer utilizes the server's powerful computing capabilities to achieve high-performance stylus writing, providing users with a better writing experience.

[0080] In the above process, factors such as the delay in transmitting handwriting information from electronic device 100 to processing device 300, the delay in processing device 300 generating an image containing the writing trajectory, and the delay in transmitting the image containing the writing trajectory from processing device 300 to electronic device 100 will cause a line drawing delay; resulting in a certain distance between the stylus and the writing trajectory on the screen of electronic device 100; leading to a poor user experience with poor responsiveness.

[0081] This application provides a handwriting input method. After receiving handwriting input from a user on a projection screen, the electronic device transmits the handwriting information to a processing device for processing. The processing device generates a projection image containing a writing trajectory (first stroke) based on the handwriting information. The electronic device also performs local drawing based on the handwriting input to generate a second writing trajectory. After receiving the projection image from the processing device, the electronic device overlays the locally drawn second writing trajectory onto the projection image for display (i.e., displays the updated projection screen). Since the latency of the electronic device locally drawing the writing trajectory is less than the latency of the electronic device acquiring the projection image, overlaying the locally drawn writing trajectory with the writing trajectory drawn by the processing device reduces the distance between the handwriting pen and the writing trajectory on the electronic device screen; this reduces the line drawing latency of handwriting input on the electronic device and improves the responsiveness of the handwriting pen.

[0082] For example, such as Figure 4As shown, the time delay of the electronic device obtaining the projection image is 100 milliseconds (ms), and the time delay of the electronic device locally drawing the writing track is 20 ms. At time T, the writing track obtained by the electronic device from the projection image of the processing device is drawn according to the handwriting pen input at (T-100 ms), and the locally drawn writing track is drawn according to the handwriting pen input at (T-20 ms). The locally drawn writing track reduces the distance between the writing track drawn by the processing device and the handwriting pen.

[0083] Exemplarily, Figure 5 A structural schematic diagram of an electronic device 100 provided by an embodiment of the present application is shown.

[0084] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a display screen 150, an antenna, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, and a camera 191, etc. The sensor module 180 can include a pressure sensor, a magnetic sensor, a distance sensor, a fingerprint sensor, a touch sensor, etc.

[0085] It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In another embodiment of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0086] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0087] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0088] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0089] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0090] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to a touch sensor, a charger, a flash, a camera 191, etc. through different I2C bus interfaces, respectively. For example, the processor 110 can be coupled to a touch sensor through an I2C interface, so that the processor 110 and the touch sensor communicate through the I2C bus interface to realize the touch function of the electronic device 100.

[0091] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled with the audio module 170 through the I2S buses to enable communication between the processor 110 and the audio module 170.

[0092] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 can be coupled with the wireless communication module 160 through a PCM bus interface. Both the I2S interface and the PCM interface can be used for audio communication.

[0093] The UART interface is a universal serial bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is commonly used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with a Bluetooth module in the wireless communication module 160 through the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to implement the functionality of playing music through Bluetooth earphones.

[0094] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 150 and the camera 191. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 191 communicate through the CSI interface to implement the photographing functionality of the electronic device 100. The processor 110 and the display screen 150 communicate through the DSI interface to implement the display functionality of the electronic device 100.

[0095] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 191, the display screen 150, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0096] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0097] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0098] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 while also supplying power to the electronic device through the power management module 141.

[0099] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display 150, the camera 191, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In other embodiments, the power management module 141 can also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.

[0100] The wireless communication function of the electronic device 100 can be realized through an antenna, a wireless communication module 160, a modem processor, and a baseband processor, etc.

[0101] The antenna is used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antennas can be multiplexed as diversity antennas for a wireless local area network. In other embodiments, the antennas can be used in combination with a tuning switch.

[0102] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a microphone 170B, etc.), or displays an image or a video through the display screen 150. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as other functional modules.

[0103] The wireless communication module 160 can provide a wireless communication solution including a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like, which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via an antenna, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, perform frequency modulation, amplification, and convert the signal into an electromagnetic wave radiation via an antenna.

[0104] The electronic device 100 implements a display function through a GPU, a display screen 150, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 150 and the application processor. The GPU is configured to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0105] The display screen 150 is configured to display images, videos, and the like. The display screen 150 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 150, where N is a positive integer greater than 1.

[0106] The electronic device 100 can implement the photographing function through the ISP, the camera 191, the video codec, the GPU, the display screen 150, and the application processor.

[0107] The ISP is configured to process the data fed back by the camera 191. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 191.

[0108] The camera 191 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 191, where N is a positive integer greater than 1.

[0109] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0110] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0111] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognition applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0112] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0113] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phonebook, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0114] The electronic device 100 can realize audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0115] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functions of the audio module 170 can be disposed in the processor 110.

[0116] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or a hands-free call through the speaker 170A.

[0117] The receiver 170B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 is engaged in a call or a voice message, a user can listen to the voice through the receiver 170B by placing the receiver 170B close to the ear.

[0118] The microphone 170C, also referred to as a "microphone", "microphone", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, a user can speak into the microphone 170C by placing the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, it can also achieve noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, and realize the function of directional recording, etc.

[0119] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0120] The magnetic sensor includes a Hall sensor. The electronic device 100 can use the magnetic sensor to detect the opening and closing of the flip cover. The distance sensor is configured to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, the scene is shot, and the electronic device 100 can use the distance sensor to measure the distance to achieve fast focusing. The fingerprint sensor is configured to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application lock, fingerprint shooting, fingerprint call answering, etc.

[0121] A touch sensor, also known as a "touch panel," can be located on the display screen 150. The touch sensor and the display screen 150 together form a touchscreen, also called a "touch display." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 150. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 150.

[0122] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 150. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 150, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 may also calculate the touch position based on the detection signal from the pressure sensor. In some embodiments, the pressure sensor is used to detect the input position of a stylus, pressure sensitivity, etc.

[0123] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0124] The following describes in detail the handwriting input method provided in the embodiments of this application, taking a tablet computer as the electronic device 100 and a PC as the processing device 300 as an example.

[0125] like Figure 6 As shown, application one runs on the PC. For example, application one is a drawing application. The tablet computer is set up as the PC's input device. The PC also projects the interface of application one onto the tablet computer's display. Users can use a stylus to draw lines on the tablet computer, enabling handwriting input on the PC; for example, users can use a stylus to draw lines on the tablet computer screen (projected interface), enabling them to input drawing strokes on the drawing application's display interface.

[0126] In one implementation, the tablet's touch module receives input from a stylus pen and acquires corresponding handwriting information based on each input. This handwriting information includes any one or any combination of the following: input coordinates, pressure sensitivity, tilt angle, etc. Specifically, input at a single point (e.g., a pixel) on the tablet screen is considered one input.

[0127] As shown in Figure 7A , the user uses the stylus 200 to draw a line on the tablet 100. The input of the stylus 200 at point a on the screen of the tablet 100 is referred to as input a (first input). The handwriting information corresponding to the input a is referred to as handwriting information a (first handwriting information).

[0128] Please continue to refer to Figure 6 , the tablet sends the handwriting information a to the PC through wireless communication (such as Bluetooth). After the application program running on the PC receives the handwriting information a, the application program draws a writing track according to the handwriting information a. In an implementation, the application program draws a point according to the handwriting information a. For example, the application program determines the position of the drawn point according to the input coordinates of the handwriting information, determines the gray scale of the drawn point according to the pressure of the handwriting information, and determines the size of the drawn point according to the tilt angle of the handwriting information. The point drawn according to the handwriting information a is connected to the already drawn points on the PC as a writing track one. For example, the writing track one includes the points drawn by the PC according to the input of the stylus moving from point o to point a. The application program also renders and synthesizes according to the current display interface and the writing track one to generate a first projection image containing the writing track one. Figure 7A

[0129] In an example, during the process of drawing the writing track according to the handwriting information, a point reporting prediction algorithm is used to predict a segment of the writing track through the already drawn writing track; and the actually drawn writing track is combined with the predicted writing track. Since the predicted writing track is displayed in advance, the effect of fast line drawing speed is visually formed. In this way, the time delay of drawing the writing track on the PC, that is, the time delay of generating the first image by the PC, can be reduced. The point reporting prediction algorithm is used to predict the position of the reporting point in advance through the movement rule of the known reporting point, and a conventional point reporting prediction algorithm (such as Gaussian distribution) in the prior art can be used, which is not limited in the embodiments of the present application.

[0130] Optionally, the PC compresses the first projection image into a video stream. The PC sends the video stream to the tablet through wireless communication (such as Wi-Fi). After the tablet receives the video stream, the first projection image is obtained by decompression. The tablet displays the first projection image (the first projection image includes the writing track one).

[0131] Since the handwriting information is transmitted from the tablet to the PC, the first projection image is transmitted from the PC to the tablet, and the process of generating the first projection image by the PC takes time; and there is a time delay in obtaining the first projection image by the tablet. During the time delay, the user continues to input using the stylus on the display screen of the tablet. There is a certain distance between the writing track one displayed by the tablet and the stylus. ​

[0132] The handwriting pen input method provided by the embodiments of the present application, the touch module of the tablet computer receives the handwriting pen input, and an application program two running on the tablet computer also performs local drawing of the writing track according to the handwriting information of the input. In an implementation manner, the application program two draws a point according to the handwriting information of the input. For example, the application program two determines the position of the drawn point according to the input coordinates of the handwriting information, determines the gray scale of the drawn point according to the pressure of the handwriting information, and determines the size of the drawn point according to the inclination angle of the handwriting information. The specific method of drawing the writing track by the tablet computer according to the handwriting information can refer to the means in the conventional technology. The embodiments of the present application do not limit this.

[0133] The application program two connects the point drawn according to the input with the drawn point of the application program two as a writing track two.

[0134] In an example, in the process of drawing the writing track by the application program two according to the handwriting information, a report point prediction algorithm is used to predict a segment of the writing track through the drawn writing track; and the actually drawn writing track according to the handwriting information is combined with the predicted writing track. For example, as shown in FIG. 6, a report point prediction algorithm is used to calculate a predicted point according to the drawn point, and the predicted writing track (the predicted points are connected to form a predicted writing track) is displayed in advance, which forms a visual effect of fast line drawing. In this way, the time delay of drawing the writing track by the tablet computer can be reduced. In some examples, the application program two can also use forced single-frame drawing and synthesis and the like to reduce the time delay of drawing the writing track. Figure 7B

[0135] It can be understood that the application program two on the tablet computer only draws the writing track, and does not need to generate a complete image of the display interface of the application program one, so that the demand for the computing power of the tablet computer is low, and the electronic device can be suitable for a simple system-on-a-chip (SOC).

[0136] The tablet computer superimposes the locally drawn writing track two on the first projection image for display (that is, displays the updated projection interface). In an example, the display image (the updated projection interface) of the tablet computer includes two layers. The bottom layer is a projection layer, and the top layer is a local layer, which is suspended in the projection layer. The tablet computer takes the received first projection image as the projection layer and takes the locally drawn writing track as the local layer. The tablet computer displays the image synthesized by the projection layer and the local layer.

[0137] ​There is a first delay in the tablet computer acquiring the projected image, and a second delay in the local drawing of the writing trajectory. In one implementation, the image displayed by the tablet computer at time T is a composite image generated by the PC based on the first input at time (T - first delay) and the writing trajectory drawn by the tablet computer based on the second input at time (T - second delay).

[0138] For example, the first delay is 100ms, and the second delay is 20ms. Figure 7C As shown, the tablet receives the first input (input a) from the stylus at time (T-100ms), sends the handwriting information a corresponding to input a to the PC, and obtains the first projected image from the PC. The first projected image includes a writing trajectory, which includes points drawn by the PC based on the input during the movement of the stylus from point o to point a. Within the first delay (100ms) of the tablet obtaining the first projected image, the stylus moves from point a to point b, and time T is when the stylus moves to point b.

[0139] The tablet receives a second input (input c) from the stylus at time (T-20ms), and draws a second writing trajectory locally based on input c. The second writing trajectory includes the points drawn by the tablet during the movement of the stylus from point a to point c. It can be understood that the second writing trajectory is a continuation of the first writing trajectory.

[0140] At time T, the projection layer of the tablet's displayed image is the first projection image (including writing trajectory one), and the local layer is writing trajectory two. The locally drawn writing trajectory two reduces the distance between writing trajectory one and the stylus, reduces line drawing latency, and improves stylus responsiveness.

[0141] In one implementation, application two determines the length of the second writing trajectory based on a first delay and a second delay. For example, the tablet receives a second input from the stylus at time (T - second delay) and connects the points drawn based on the input within the time period (first delay - second delay) prior to the second input to form the second writing trajectory.

[0142] For example, such as Figure 7C As shown, the tablet receives input 'c' from the stylus at time (T-20ms). It then connects the points drawn based on the input within the 80ms preceding input (100ms-20ms) (i.e., the input between point a and point c) to form writing trajectory two. The length of writing trajectory two is the length of the trajectory between point a and point c. In this implementation, writing trajectory two is a continuation of writing trajectory one, and they do not overlap.

[0143] Understandably, in some other embodiments, the length of the second writing trajectory can also be greater than the trajectory length between point a and point c. For example, the length of the second writing trajectory can be based on the input within 150ms before input c. Figure 7C The points drawn from the input between midpoint d and point c are connected to form writing trajectory two. For example, the points drawn from the input between input o and input c are connected to form writing trajectory two. This application embodiment does not limit this. In this implementation, writing trajectory two partially overlaps with writing trajectory one, and writing trajectory two includes a continuation of writing trajectory one. It is understood that writing trajectory two, as a continuation of writing trajectory one, can also completely overlap with writing trajectory one.

[0144] It should be noted that the embodiments of this application do not limit the correspondence between the screen-projected image obtained from the PC and the locally drawn writing trajectory when the tablet computer synthesizes the display image. For example, in some other embodiments, the tablet computer obtains the screen-projected image at time T and the locally drawn writing trajectory at time (T+t), where t>0; the screen-projected image obtained at time T and the locally drawn writing trajectory obtained at time (T+t) are synthesized into a display image for display. In other embodiments, the tablet computer obtains the locally drawn writing trajectory at time T and the screen-projected image at time (T+t), where t>0; the locally drawn writing trajectory obtained at time T and the screen-projected image obtained at time (T+t) are synthesized into a display image for display.

[0145] For example, Figure 8A This illustration shows a specific example of a handwriting input method provided in an embodiment of this application. For example... Figure 8A As shown, the tablet computer 100 displays a drawing application interface; it receives input from the stylus 200 by tapping "a" on the screen and obtains handwriting information "a". The tablet computer 100 sends the handwriting information "a" to the PC 300. The PC 300 draws on the drawing application interface 710 based on the handwriting information "a", connecting the drawn point with previously drawn points to form a writing trajectory 720. The PC 300 renders and composites the drawing to generate a first projected image 730 containing the writing trajectory 720. The PC 300 compresses the first projected image 730 into a video stream and sends it to the tablet computer 100. The tablet computer 100 receives the video stream and obtains the first projected image 730.

[0146] During the process from when tablet computer 100 transmits handwriting information 'a' to PC 300 until tablet computer 100 receives the first projected image 730, the user continues to input using stylus pen 200, and tablet computer 100 continues to receive input from stylus pen 200. For example, as... Figure 8AAs shown, the stylus 200 moves from point a to point b. At a second moment, the tablet receives input from the stylus 200 at point c on the screen, obtains the corresponding handwriting information c, and draws based on the handwriting information c. The tablet 100 connects the points drawn based on the input from input a to input c to form a writing trajectory 740.

[0147] The tablet computer 100 displays an updated drawing application interface, namely, displaying image 750 (with the writing trajectory 740 superimposed on the first projected image 730). The projection layer of image 750 is the first projected image 730, and the local layer is the writing trajectory 740.

[0148] like Figure 8B As shown, the display of writing trajectory 740 (writing trajectory two) on image 750 reduces the distance between writing trajectory 720 (writing trajectory one) and stylus 200, thereby reducing the line drawing delay of tablet computer 100.

[0149] It should be noted that, Figure 8A The interface within the dashed box is not displayed separately; it is only provided as an illustrative example of an intermediate step for ease of understanding.

[0150] It should be noted that, Figure 8A PC 300 displays interface 710 and first projected image 730. It is understood that in other embodiments, PC 300 may not display interface 710 and first projected image 730, providing computing power solely to tablet PC 100.

[0151] Understandably, at time (T-20ms), the tablet receives the second input (input c) from the stylus and sends the corresponding handwriting information c to the PC. Application 1 running on the PC receives the handwriting information c, draws based on it, and renders and composites the drawing based on the application's current display interface and the drawing points, generating a second projected image containing the writing trajectory three. The PC compresses the second projected image into a video stream. The PC sends the video stream to the tablet via wireless communication (e.g., Wi-Fi). The tablet receives the video stream, decompresses it, and obtains the second projected image. Furthermore, the tablet overlays the locally drawn writing trajectory four onto the second projected image for display.

[0152] Writing trajectory three includes points drawn by the PC based on the input of the stylus as it moves from point o to point c. Writing trajectory two includes points drawn by the tablet based on the input of the stylus as it moves from point a to point c. After the tablet displays the second projected image, writing trajectory three can cover writing trajectory two.

[0153] That is, the point drawn locally by the tablet will be overwritten by the point drawn by the PC according to the same input after a certain time length. In an implementation, the writing track drawn locally by the tablet is stopped from being displayed after a preset time length.

[0154] It should be noted that the embodiments of the present application take the example of the PC and the tablet performing drawing once for each time of receiving input for introduction and description. In other embodiments, the PC and the tablet can also perform drawing once according to multiple inputs after receiving the multiple inputs. For example, drawing once according to the input of the handwriting pen received within 10 ms. The embodiments of the present application do not limit this.

[0155] For example, as shown in FIG. 1, Figure 9 At T0, the tablet 100 displays the screen projection interface of the PC, which does not include handwriting. The user uses the handwriting pen to draw a line on the tablet 100. The tablet 100 receives the input of the handwriting pen. For example, the tablet takes the input of the handwriting pen received within a first time length as one input, and sends the handwriting information corresponding to the one input to the PC for drawing of a writing track. For example, as shown in FIG. 1, Figure 9 In the example shown in FIG. 1, the input of the handwriting pen between T0 and T1b is the first input. The tablet 100 sends the handwriting information corresponding to the first input to the PC 300. The PC 300 generates a screen projection image containing writing track one according to the first input. The tablet 100 acquires the screen projection image containing writing track one.

[0156] The tablet 100 also performs local drawing according to the second input. In an example, the input time period of the second input is after the input time period of the first input, and does not overlap with the input time period of the first input completely. For example, as shown in FIG. 1, Figure 9 In the example shown in FIG. 1, the input of the handwriting pen between T1b and T2 is the second input. The tablet 100 performs local drawing according to the input of the handwriting pen between T1b and T2, and generates writing track two. Writing track two is a continued track of writing track one, and writing track two does not overlap with writing track one. In another example, the start time of the input time period of the second input is after the start time of the input time period of the first input, and the input time period of the second input partially overlaps with the input time period of the first input. For example, as shown in FIG. 1, Figure 9 In the example shown in FIG. 1, the input of the handwriting pen between T1a and T2 is the second input; wherein T1a is after T0 and before T1b. The tablet 100 performs local drawing according to the input of the handwriting pen between T1a and T2, and generates writing track two. Writing track two overlaps with part of writing track one, and writing track two contains a continued part of writing track one.

[0157] The tablet 100 displays the updated screen projection interface. The screen projection image containing the writing track one is displayed on the screen projection layer of the updated screen projection interface, and the writing track two (local layer) is superimposed and displayed on the screen projection image. The writing track two contains the continuation of the writing track one, reduces the distance between the writing track one and the stylus, and reduces the delay of the tablet 100 in drawing a line.

[0158] In an implementation manner, the tablet 100 determines the length of the input time period of the second input according to the first delay and / or the second delay. The first delay includes the delay of the tablet 100 in obtaining the screen projection image from the PC, and the second delay includes the delay of the tablet 100 in locally drawing the writing track two. In an example, the length of the input time period of the second input = (the first delay-the second delay); in this way, the writing track two drawn according to the second input is the continuation of the writing track one drawn according to the first input, and the writing track two does not coincide with the writing track one. In another example, the length of the input time period of the second input = the first delay; in this way, the writing track two drawn according to the second input coincides with part of the writing track one drawn according to the first input, and the writing track two contains the continuation of the writing track one. For example, refer to Figure 9 , the first delay = (T2-T1a), and the second delay = (T1b-T1a). In an implementation manner, the length of the input time period of the second input = (the first delay-the second delay) = ((T2-T1a)-(T1b-T1a)) = (T2-T1b); that is, the second input is the stylus input between the T1b moment and the T2 moment. In another implementation manner, the length of the input time period of the second input = the first delay = (T2-T1a); that is, the second input is the stylus input between the T1a moment and the T2 moment.

[0159] Figure 10 An example of a scenario of the stylus input method provided by the embodiments of the present application is shown.

[0160] As Figure 10As shown, the application program one runs on the PC 300. For example, the application program one is a drawing application program. The tablet 100 is set as the input device of the PC 300. The PC 300 casts the interface of the application program one to the tablet 100 for display. The tablet 100 displays the interface 810 of the drawing application program. The user uses the stylus 200 to draw a line on the interface 810. For example, at t0, the tablet 100 receives the input (first input) of the stylus 200 moving from point a to point b on the interface 810. The tablet 100 sends the information of the input of the stylus 200 moving from point a to point b to the PC 300, and performs local drawing according to the information of the input. For example, the tablet 100 locally draws a writing track 820. At t1, the tablet 100 displays the writing track 820 on the interface 810. On the other hand, the PC 300 receives the information of the input of the stylus 200 moving from point a to point b, and generates a casting image containing a writing track 830 according to the information. The PC 300 sends the casting image containing the writing track 830 to the tablet 100. The tablet 100 acquires the casting image containing the writing track 830, refreshes the casting interface at t2, and displays the casting image containing the writing track 830 on the casting layer.

[0161] It can be understood that the local layer of the refreshed casting interface includes the writing track 820, and the casting layer includes the writing track 830. The local layer is superimposed on the casting layer. The writing track 820 and the writing track 830 are both drawn according to the input of the stylus 200 moving from point a to point b. The writing track 820 and the writing track 830 have the same line track.

[0162] Since the time delay of the local drawing of the tablet 100 is less than the time delay of acquiring the casting image from the PC 300, t1 is earlier than t2. That is, the writing track drawn locally by the tablet 100 is displayed on the screen of the tablet 100 faster than the writing track drawn by the PC 300. The user can be shown the writing track input by the stylus faster, the time delay of drawing a line by the stylus is shortened, and the hand-following property of the stylus is improved.

[0163] At t2, the tablet 100 acquires the casting image containing the writing track 830 from the PC 300 and displays the casting image on the casting layer. The writing track 830 and the writing track 820 have the same line track, and the writing track 830 covers the writing track 820. Starting from t2, the writing track 820 can be stopped from being displayed. In some examples, the writing track 820 drawn locally by the tablet 100 is displayed for a period of time and then stopped from being displayed at t3. For example, t3 is equal to t2, or t3 is later than t2.

[0164] It can be understood that, in an example, between the time t1 and the time t2, the user continues to use the stylus 200 to draw a line on the tablet 100, and the stylus 200 moves from the point b to the point c. The tablet 100 performs local rendering according to the information input by the stylus from the point b to the point c, and displays the locally generated writing track (820) (the writing track 820 is represented by a solid line). Figure 10 The line segment between the point b and the point c is represented by a dashed line, so as to distinguish the writing track 820 from the writing track 830.

[0165] In some embodiments, for the same stylus input, the writing track drawn on the PC is different from the writing track drawn on the tablet. For example, the line color, the line thickness, the line texture, the line border, and the like of the two are different. For example, as shown in FIG. 8, the tablet 100 locally draws the writing track 901, and obtains the writing track 902 from the PC 300. The line of the writing track 901 drawn by the tablet 100 is thicker than the line of the writing track 902 drawn by the PC 300. In this way, the writing track locally drawn by the tablet cannot be well fused with the writing track on the image received from the PC. Figure 11

[0166] In the stylus input method provided by the embodiments of the present application, the PC sends the parameters (for example, the line color parameter, the line thickness parameter, the line texture parameter, and the like) of the writing track drawn by the PC to the tablet, and the tablet draws the writing track according to the parameters of the writing track drawn by the PC; so that the writing track locally drawn by the tablet is better fused with the projection image generated by the PC.

[0167] In an implementation manner, the application program three is run on the PC. The application program three is used to send the parameters of the writing track drawn by the PC to the tablet. In some examples, the application program three is the same as the application program two run on the tablet; it can be understood that the application program three can also be different from the application program two, and the embodiments of the present application do not limit this.

[0168] Taking the example that the application program three run on the PC is the same as the application program two run on the tablet, as shown in FIG. 9, the tablet receives the first input of the stylus, and obtains the first handwriting information; the tablet sends the first handwriting information to the PC. The PC draws the writing track one according to the first handwriting information, performs rendering and synthesis, and generates the first projection image containing the writing track one. The PC compresses the first projection image into a video stream, and sends the video stream to the tablet through wireless communication. Figure 12

[0169] ​​The application program one also sends the parameters of the drawing writing track one to the application program two. In one implementation, the application program two sends the drawing parameters to the tablet through wireless communication each time the drawing parameters are received. In another implementation, the application program two sends the drawing parameters to the tablet through wireless communication the first time the drawing parameters are received; and then compares the drawing parameters received each time with the drawing parameters received last time. If any change in the drawing parameters is determined, the changed drawing parameters are sent to the tablet through wireless communication.

[0170] The tablet also receives the second input of the stylus and obtains second handwriting information. The application program two running on the tablet draws a second writing track according to the second handwriting information and the drawing parameters received from the PC. The tablet superimposes the second writing track on the first projection image. Since the drawing parameters used by the tablet to draw the second writing track are the same as the drawing parameters used by the PC to draw the first writing track, the second writing track can be well integrated with the first projection image including the first writing track, bringing better visual effect.

[0171] In another implementation, the functions of the application program three described above can also be implemented by the application program one, i.e., the application program one sends the parameters of the writing track drawn by the PC to the tablet, which is not limited in the embodiments of the present application.

[0172] The handwriting stylus input method provided by the embodiments of the present application can generate a projection image including a writing track by using the computing power of a processing device, and draw the writing track locally, and superimpose the locally drawn writing track on the projection image generated by the processing device. Since the local drawing has small delay, the locally drawn writing track reduces the distance between the stylus on the screen of the electronic device and the writing track on the projection image, reduces the delay of the stylus input on the electronic device, and improves the hand-following property of the stylus.

[0173] It can be understood that, in order to implement the above functions, the electronic device includes hardware structures and / or software modules corresponding to the functions. Those skilled in the art can easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed in the present application, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0174] The embodiments of the present application can divide the functional modules of the electronic device according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or software functional module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, another division method can be used.

[0175] In an example, refer to Figure 13 which shows a possible structural schematic diagram of the electronic device involved in the above embodiments. The electronic device 1000 includes a processing unit 1001, a storage unit 1002, a communication unit 1003 and a display unit 1004.

[0176] The processing unit 1001 is configured to control and manage the actions of the electronic device 1000. For example, it can be used to draw a writing track according to handwriting information, and / or other processing steps in the embodiments of the present application.

[0177] The storage unit 1002 is configured to save the program code and data of the electronic device 1000. For example, it can be used to save the drawing parameters obtained from the processing device.

[0178] The communication unit 1003 is configured to support the communication between the electronic device 1000 and other electronic devices. For example, it can be used to send handwriting information to the processing device, receive a screen projection image including a writing track from the processing device, and / or receive drawing parameters from the processing device.

[0179] The display unit 1004 is configured to display the interface of the electronic device 1000. For example, it can be used to display an image including a writing track.

[0180] Of course, the unit modules in the above electronic device 1000 include but are not limited to the above processing unit 1001, storage unit 1002, communication unit 1003 and display unit 1004. For example, the electronic device 1000 can also include a power supply unit, etc. The power supply unit is configured to supply power to the electronic device 1000.

[0181] The processing unit 1001 can be a processor or a controller, for example, a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The storage unit 1002 can be a memory. The communication unit 1003 can be a transceiver, a transceiving circuit, or the like. The display unit 1004 can be a display screen.

[0182] For example, the processing unit 1001 is a processor (such as the processor 110 shown in FIG. 1), the storage unit 1002 can be a memory (such as the internal memory 121 shown in FIG. 1), the communication unit 1003 can be referred to as a communication interface, including a wireless communication module (such as the wireless communication module 160 shown in FIG. 1), and the display unit 1004 is a display screen (such as the display screen 150 shown in FIG. 1, which can be a touch screen, and the display panel and the touch panel can be integrated in the touch screen). The electronic device 1000 provided in the embodiments of the present application can be the electronic device 100 shown in FIG. 1. The above-mentioned processor, memory, communication interface, display screen, and the like can be connected together, for example, through a bus. Figure 5 Figure 5 Figure 5 Figure 5 Figure 5

[0183] The embodiments of the present application further provide a computer readable storage medium, which stores computer program codes, when the processor executes the computer program codes, the electronic device executes the method in the above-mentioned embodiments.

[0184] The embodiments of the present application further provide a computer program product, when the computer program product runs on a computer, the computer program product makes the computer execute the method in the above-mentioned embodiments.

[0185] The electronic device 1000, the computer readable storage medium, or the computer program product provided in the embodiments of the present application are all used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method provided above, which will not be described herein again.

[0186] ​​​​​Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0187] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0188] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0189] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single chip, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a magnetic disk or an optical disk, and various storage program codes.

[0190] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A handwriting pen input method, characterized by, The method comprises: A first electronic device receives a screen projection interface of a first application, the first application running on a second electronic device; The first electronic device receives a stylus input of a user on the screen projection interface, the stylus input comprising a first input and a second input; The first electronic device displays an updated screen projection interface, the updated screen projection interface comprising a first layer and a second layer, the first layer comprising a first image containing a first writing track generated by the second electronic device according to the first input, and the second layer comprising a second writing track drawn by the first electronic device according to the second input; A length of an input time period of the first input is determined according to a first time delay, and a length of an input time period of the second input is determined according to a second time delay, the first time delay comprising a time delay of the first electronic device obtaining an image from the second electronic device, and the second time delay comprising a time delay of the first electronic device generating the second writing track, the input time period of the first input partially overlapping the input time period of the second input.

2. The method of claim 1, wherein, The second layer is superimposed on the first layer.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: The first electronic device obtains a second image, the second image comprising a third writing track generated by the second electronic device according to the second input, and the first electronic device displays the second image on the first layer.

4. The method of claim 3, wherein, The method further comprises:

5. The method of claim 3, wherein, The third writing track on the first layer covers the second writing track on the second layer.

6. The method of any one of claims 1-2, 4-5, wherein, The second writing track stops being displayed after a preset time length, or the second writing track stops being displayed after the third writing track covers the second writing track.

7. The method of any one of claims 1-2, 4-5, wherein, The second writing track is a continuation of the first writing track, or the second writing track coincides with part of the first writing track and contains a continuation of the first writing track.

8. The method of any one of claims 1-2, 4-5, wherein, A point prediction algorithm is used to predict a writing track from a drawn writing track, the actual drawn writing track is combined with the predicted writing track, and the predicted writing track is displayed in advance. The method further comprises:

9. The method of claim 8, wherein, The first electronic device obtains drawing parameters sent by the second electronic device, and the first electronic device draws the second writing track using the drawing parameters.

10. The method of claim 9, wherein, The drawing parameters comprise drawing parameters used by the second electronic device to draw the first writing track.

11. A handwriting pen input method, characterized by, The drawing parameters comprise at least one of a line color parameter, a line thickness parameter, and a line texture parameter. The method comprises: A first electronic device displays a screen projection interface of a first application, receives a first input of a stylus, and the first application runs on a second electronic device; The first electronic device sends information of the first input to the second electronic device; The first electronic device receives a first image from the second electronic device, the first image comprising a first writing track drawn according to the information of the first input; The first electronic device displays the first image; The first electronic device receives a second input of a stylus; The first electronic device draws a second writing track according to the information of the second input; The first electronic device displays the second writing track, and the second writing track is displayed as a continuation track of the first writing track. A length of an input time period of the first input is determined according to a first time delay, and a length of an input time period of the second input is determined according to a second time delay. The first time delay includes a time delay of the first electronic device acquiring the image from the second electronic device, and the second time delay includes a time delay of the first electronic device generating the second writing track. The input time period of the first input partially overlaps with the input time period of the second input.

12. The method of claim 11, wherein, The second writing track is all a continuation track of the first writing track, or the second writing track coincides with a partial track of the first writing track and contains a continuation part of the first writing track.

13. The method according to claim 11 or 12, characterized in that, The first image is displayed on a first layer, and the second writing track is displayed on a second layer, and the second layer is superimposed on the first layer.

14. The method of claim 11 or 12, wherein, The method further includes: The first electronic device sends information of the second input to the second electronic device. The first electronic device receives a second image from the second electronic device, and the second image includes a third writing track drawn according to the information of the second input. The first electronic device displays the second image, and the third writing track covers the second writing track.

15. The method of claim 14, wherein, The second writing track stops being displayed after being displayed for a preset time length, or the second writing track stops being displayed after the third writing track covers the second writing track.

16. The method of any of claims 11-12, 15, wherein, A point prediction algorithm is used to predict a segment of writing track through the drawn writing track. The actually drawn writing track according to handwriting information is combined with the predicted writing track. The predicted writing track is displayed in advance.

17. The method of any one of claims 11-12, 15, wherein The method further includes: The first electronic device receives drawing parameters of the second electronic device. The first electronic device draws a second writing track according to the information of the second input includes: The first electronic device draws the second writing track according to the information of the second input using the drawing parameters.

18. An electronic device, comprising: Comprise: One or more processors; A display screen; A memory; The memory stores one or more computer programs, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 1-17.

19. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the instructions are run on an electronic device, the electronic device performs the method of any one of claims 1-17.

20. A computer program product, characterised in that, When the computer program product is run on an electronic device, the electronic device performs the method of any one of claims 1-17.

21. A chip system, characterized by The chip system includes a processor and an interface circuit, the interface circuit is used to perform a transceiving function, and sends instructions to the processor, and when the instructions are executed by the processor, the processor performs the method of any one of claims 1-17.

Images