Method for displaying input content and electronic device

Abstract

The present application relates to the technical field of terminals, and discloses a method for displaying input content and an electronic device. In response to a first operation on a text note editing interface, a handwriting area is added to the editing interface. A user can input content in the handwriting area by means of handwriting. In response to a second operation of the user, the handwriting area is deleted, and a first handwriting display area on the editing interface displays the content input by means of handwriting. In response to a third operation in the first handwriting display area, the handwriting area is displayed on the editing interface again, and the user can continue to input content in the handwriting area by means of handwriting. During this period, the text content in the editing interface is always displayed. Interface transitions are reduced, bringing smooth use experience to users. Moreover, before the editing interface is generated, handwritten input content and text content do not correspond to different layers, thereby avoiding the problem of display misalignment after form changes of a tablet computer caused by text content and handwritten input content being located on different layers.

Description

Method for displaying input content and electronic device TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of terminal, and in particular, to a method for displaying input content and an electronic device. BACKGROUND

[0002] Generally, terminal devices support note applications. Users can use note applications to record text, pictures, audio and other information.

[0003] With the increasing powerfulness of note applications, the use experience of note applications is also increasingly rich. For example, users can enter text in the text note of the note application through a keyboard, enter the content in the clipboard through copy and paste, add hyperlinks, add pictures, audio and the like, and can also write text, graphics and other content in a handwriting manner.

[0004] Inputting content in a handwriting manner is convenient and fast, and users can input text, graphics and the like at will according to their needs, which is welcomed by users. However, there are still some defects in handwriting input in the current note applications. How to optimize the handwriting interaction process of note applications and improve the use experience of users using note applications is a problem to be solved.

[0005] SUMMARY

[0006] Embodiments of the present application provide a method for displaying input content and an electronic device, which optimize the display mode of handwriting input content of note applications and improve the use experience of users inputting content in a handwriting manner.

[0007] To achieve the above object, embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a method for displaying input content is provided. The method is applied to an electronic device, such as a terminal device. In an example of the terminal device, the method comprises: displaying, by the terminal device, a first interface; the first interface comprises a first text area, and the first text area comprises first text content; in response to a first operation on a first option in the first interface, creating a first view, and displaying a second interface; the second interface comprises a first handwriting area and a second text area, the second text area comprises the first text content, and the first view corresponds to the first handwriting area; in response to a first handwriting input in the first handwriting area, displaying the first handwriting input content in the first handwriting area; in response to a second operation on the second interface (e.g., a click operation on an area in the second interface other than the first handwriting area), deleting the first view, and displaying a third interface; the third interface comprises a first handwriting display area and a third text area, the first handwriting display area comprises the first handwriting input content, and the third text area comprises the first text content; in response to a third operation on the first handwriting display area in the third interface, creating a second view, and displaying a fourth interface; the fourth interface comprises the first handwriting area and the second text area, and the second view corresponds to the first handwriting area.

[0009] In the method, in response to a first operation of a user on an editing interface of a text note, a handwriting area is added to the editing interface of the text note. The user can input content in the handwriting area by handwriting. In response to a second operation of the user, such as a click operation on an area other than the handwriting area, the handwriting area is deleted, and a first handwriting display area in the editing interface of the text note displays the content input by the user in the handwriting area by handwriting. In response to a third operation of the user on the first handwriting display area, the handwriting area is displayed on the editing interface again, and the user can continue to input content in the handwriting area by handwriting. During this period, the first text content is always displayed on the editing interface. This brings a smoother user experience, and reduces the poor experience of the user caused by the switching between the editing interface and the handwriting input interface. Moreover, the editing interface comprises a handwriting display area and a text area, the handwriting input content is displayed in the handwriting display area, and the text content is displayed in the text area. Before the editing interface is generated, the handwriting input content and the text content do not correspond to different layers. This avoids the problem of misalignment caused by the text content and the handwriting input content being located in different layers after the tablet computer changes its form.

[0010] In a possible implementation, in response to the second operation on the second interface, a second image is also created, and the second image corresponds to the first handwriting display area in the third interface. The first handwriting display area comprises the first handwriting input content. Optionally, the second image can be a picture.

[0011] That is, in the second interface, the first handwriting input content is saved in the form of a picture, and the picture also belongs to the rich text content and is located in the same layer as other text content. In this way, the display misalignment problem caused by the fact that the handwriting input content and the text content are located in different layers can be avoided.

[0012] In combination with the first aspect, in a possible implementation, the third operation includes second handwriting input, in response to the second handwriting input in the first handwriting display region of the third interface, the second handwriting input content is displayed in the first handwriting region of the fourth interface.

[0013] In this method, the user can directly start handwriting input on the handwriting picture (the first handwriting display region). In response to the handwriting input of the user on the handwriting picture, the handwriting picture is deleted, and the view (the second view) corresponding to the handwriting region is created.

[0014] In a possible implementation, the fourth interface is generated based on a preset layer (i.e., a transition layer, which is pre-drawn) and the layer in which the second view is located. Optionally, the fourth interface includes the preset layer and the layer in which the second view is located; the preset layer is located above the layer in which the second view is located. In response to the second handwriting input in the first handwriting display region of the third interface, the transition layer is set to be visible. The second handwriting input content is generated in the preset layer, and the second handwriting input content is displayed in the first handwriting region of the fourth interface.

[0015] Since the transition layer is pre-drawn and is not created after the second handwriting input is received, the second handwriting input content drawn in the transition layer can be displayed before the second view is created. Thus, before the view corresponding to the first handwriting region in the fourth interface is successfully created, the handwriting corresponding to the second handwriting input can be displayed on the screen through the transition layer. Since the transition layer is pre-drawn, the time consumed for drawing the second handwriting input content on the transition layer is short, which can effectively shorten the time interval between the handwriting input of the user and the display of the second handwriting input content on the display screen, and improve the followability of the handwriting input of the user.

[0016] After the second view is created, the second view can receive an input event corresponding to the handwriting input of the user, process the input event, and generate the second handwriting input content in the layer in which the second view is located. After the second handwriting input content is generated in the layer in which the second view is located, and no handwriting input is occurring, the transition layer is set to be invisible. The second handwriting input content can be displayed on the screen through the second view. Subsequently, the second view can process the input event generated by the handwriting input of the user in the first handwriting region, and display the handwriting input content in the first handwriting region.

[0017] With reference to the first aspect, in a possible implementation, in a case where the second interface is displayed, in response to a change in the form of the terminal device (for example, the change in the form includes rotation, folding, unfolding, or the like), the first view is deleted; after the first view is deleted, a third view is created, and a fifth interface is displayed; the fifth interface includes a second handwriting area and a fourth text area, the third view corresponds to the second handwriting area, and the second handwriting area includes the first handwriting input content, and the fourth text area includes the first text content.

[0018] In this method, when the editing interface (for example, the second interface) including the handwriting area changes in form, the view corresponding to the handwriting area before the change in the form is deleted, and the view corresponding to the handwriting area is regenerated according to the screen layout after the change in the form. The regenerated view (the third view) is laid out according to the layout rule after the change in the form.

[0019] In response to the change in the form of the terminal device, before the first view is deleted, a first image is generated according to the first view, and optionally, the first image can be a picture; in this way, the third view can be generated according to the first image and the screen display area after the change in the form of the terminal device. The first image records the display position and size of the first handwriting area in the screen.

[0020] With reference to the first aspect, in a possible implementation, in a case where the third interface is displayed, in response to a change in the form of the terminal device, a sixth interface is displayed; the sixth interface includes a second handwriting display area and a fifth text area, the second handwriting display area includes the first handwriting input content, and the fifth text area includes the first text content; during the process from the change in the form of the terminal device to the display of the sixth interface, no view is deleted or created.

[0021] In this method, the third interface includes a first handwriting display area and a third text area, after the change in the form of the terminal device, the sixth interface is generated by performing interface layout on each view in the third interface according to the screen display area, the first handwriting display area corresponds to the second handwriting display area, and the third text area corresponds to the fifth text area; no view is deleted or created.

[0022] With reference to the first aspect, in a possible implementation, the third operation includes a click operation, and the fourth interface is the same as the second interface. Optionally, the fourth interface is the same as the second interface, or the fourth interface is the second interface. The status bar does not belong to the interface; that is, if only the status bar is different between two interfaces, the two interfaces are the same.

[0023] In this method, the user can click the handwriting picture (the first handwriting display area), so that the first handwriting area is displayed in the editing interface, and handwriting input can be continued in the first handwriting area.

[0024] With reference to the first aspect, in a possible implementation, the third interface further includes a third handwriting area, the third handwriting area including third handwriting input content, and the fourth interface further includes a third handwriting display area, the third handwriting display area including the third handwriting input content. In response to a third operation on the first handwriting display area of the third interface, a third image is further created, the third image corresponding to the third handwriting display area. Optionally, the third image can be a picture.

[0025] In the method, the editing interface includes a handwriting area and a handwriting picture (handwriting display area), and in response to a third operation of the user on the handwriting picture, the handwriting area is displayed in the form of the handwriting picture.

[0026] With reference to the first aspect, in a possible implementation, the first handwriting display area and the first handwriting area are the same display area on the screen.

[0027] In the method, the user is brought the experience that the handwriting display area and the handwriting area are the same area, and the use experience of the user that the handwriting area jumps is avoided.

[0028] With reference to the first aspect, in a possible implementation, the first text content includes at least one of a character, a picture, an audio, a table, and a hyperlink. That is, the text area can include rich text content.

[0029] The second aspect provides an electronic device having a function of implementing the method of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0030] The third aspect provides an electronic device including a processor, a memory, and a display screen. The memory is configured to store computer execution instructions. When the electronic device is running, the processor executes the computer execution instructions stored in the memory, so that the electronic device executes the method of any one of the possible implementations of the first aspect.

[0031] The fourth aspect provides a computer readable storage medium, which stores instructions. When the instructions are run on a computer, the computer can execute the method of any one of the possible implementations of the first aspect.

[0032] The fifth aspect provides a computer program product including instructions, which, when run on a computer, can make the computer execute the method of any one of the possible implementations of the first aspect.

[0033] The technical effects brought by any possible implementation of the second aspect to the fifth aspect can be referred to the technical effects brought by different implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 is a schematic diagram of a user interface of a note application on a tablet computer;

[0035] Fig. 2 is a schematic diagram of a scenario example of a handwriting interaction method in a note application;

[0036] Fig. 3 is a schematic diagram of a scenario example of a handwriting interaction method in a note application;

[0037] Fig. 4A is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0038] Fig. 4B is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0039] Fig. 4C is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0040] Fig. 5A is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0041] Fig. 5B is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0042] Fig. 6 is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0043] Fig. 7A is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0044] Fig. 7B is a schematic diagram of a scenario example of a method for displaying input content provided by an embodiment of the present application;

[0045] Fig. 8 is a schematic diagram of a hardware structure of a terminal device to which the method for displaying input content provided by an embodiment of the present application is applicable;

[0046] Fig. 9 is a schematic diagram of a software architecture of a terminal device to which the method for displaying input content provided by an embodiment of the present application is applicable;

[0047] Fig. 10A is a schematic diagram of a flow of adding a handwriting area in an editing interface of a text note in the method for displaying input content provided by an embodiment of the present application;

[0048] Fig. 10B is a schematic diagram of a flow of converting a handwriting area into a handwriting picture in the method for displaying input content provided by an embodiment of the present application;

[0049] FIG. 11A is a flow diagram of a method for displaying input content according to an embodiment of the present application;

[0050] FIG. 11B is a diagram of a terminal device orientation according to an embodiment of the present application;

[0051] FIG. 12 is a flow diagram of a method for displaying input content according to an embodiment of the present application;

[0052] FIG. 13 is a diagram of a scenario for displaying input content according to an embodiment of the present application;

[0053] FIG. 14 is a flow diagram of a method for displaying input content according to an embodiment of the present application;

[0054] FIG. 15 is a diagram of a terminal device structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, objects, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, objects, and / or components thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The term "at least one of' is used to mean one or more of the listed items can be present alone, or in a combination of one or more of the listed items. The term "or" is used to refer to a nonexclusive logical or, i.e., the logical OR that allows one or the other, but not both. In addition, it is to be understood that the phraseology or terminology employed herein, and not otherwise expressly set forth in this detailed description, is intended to be interpreted in accordance with the will of the inventor(s) when reading this detailed description.

[0056] Reference within the specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified

[0057] In the present application, the word "exemplary" or "for example" is used to mean serving as an example, instance, or illustration. Any implementation described as "exemplary" or "for example" in the present application is not necessarily to be construed as preferred or advantageous over other implementations. The

[0058] The method for displaying input content provided by the embodiments of the present application can be applied to an electronic device supporting handwriting input. For example, the electronic device can be a terminal device. The terminal device can include a mobile phone, a tablet computer, a notebook computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a smart home device (such as a smart television, a smart screen, a large screen, a smart speaker, a smart air conditioner, etc.), a personal digital assistant (PDA), a wearable device (such as a smart watch, a smart bracelet, etc.), a vehicle-mounted device, a virtual reality device, and the like, and the embodiments of the present application do not make any limitation thereto.

[0059] In an example, a terminal device is installed with a note application. For example, taking the terminal device as a tablet computer, FIG. 1 shows a user interface schematic diagram of the note application on the tablet computer. It can be understood that on different terminal devices, the note application can have different names, for example, a notebook, a notepad, and the like.

[0060] Referring to FIG. 1, the tablet 100 displays a user interface 101 of a note application, the user interface 101 including an "add" button 102. In response to a click operation of the "add" button 102 by a user, a "text note" option 1021, a "handwriting note" option 1022, etc. are displayed on the user interface 101. The user can click the "text note" option 1021 to create a text note, and the tablet 100 displays an editing interface of the text note; the user can input text, symbols, pictures, audio, hyperlinks, etc. on the editing interface of the text note, and the user can also input content (e.g. characters, graphics, etc.) by handwriting on the editing interface of the text note. The user can click the "handwriting note" option 1022 to create a handwriting note; the user can input characters, graphics, etc. by handwriting on a user interface (canvas) of the handwriting note.

[0061] For example, as shown in FIG. 1, in response to a click operation of the "text note" option 1021, the tablet 100 displays an editing interface 103 of a text note, and optionally, the user can create a text note by clicking the "text note" option 1021; the editing interface 103 includes a text area, and the user can input rich text content (e.g. characters, symbols, pictures, hyperlinks, audio, etc.) in the text area. In one implementation, the editing interface 103 includes a virtual keyboard 1031, and the user can input rich text content (e.g. characters, numbers, symbols, etc.) in the text area of the editing interface 103 by clicking virtual keys in the virtual keyboard 1031; in another implementation, the terminal device includes an input device (e.g. a keyboard), and the user can input rich text content (e.g. characters, numbers, symbols, etc.) in the text area of the editing interface 103 by using the keyboard.

[0062] In one implementation, the editing interface 103 includes a handwriting option 1032, and after the user clicks the handwriting option 1032, the user can input by handwriting in a handwriting area of the editing interface 103; after the tablet 100 receives handwriting input of the user in the handwriting area, the handwriting area displays handwriting traces (characters, numbers, symbols, graphics, etc.) of the user; the handwriting traces input by the user are kept in the handwriting area and do not disappear when the handwriting traces are drawn.

[0063] For example, the editing interface 103 further includes a "close" button 1033, in response to a click operation on the "close" button 1033, the virtual keyboard 1031 is no longer displayed on the editing interface 103. For example, the editing interface 103 further includes a text tool 1034, through which the user can modify the font of the text in the text area. For example, the editing interface 103 further includes an "add" button 1035, in response to a click operation of the user on the "add" button 1035, "gallery", "table", "hyperlink" and the like options are displayed on the editing interface 103, the user can add a picture to the text area of the editing interface 103 by clicking the "gallery" option; the user can add a table to the text area of the editing interface 103 by clicking the "table" option; the user can add a hyperlink to the text area of the editing interface 103 by clicking the "hyperlink" option, and the like. For example, the editing interface 103 further includes a "recording" button 1036, through which the user can add a recording file to the text area of the editing interface 103.

[0064] The handwriting input in the text note is convenient and fast, and there are various implementation manners in the specific implementation.

[0065] In an implementation manner, referring to (a) of FIG. 2, the editing interface of the text note is generated based on at least two layers, wherein layer A includes rich text content such as text, picture, table, hyperlink and the like, and layer B includes handwriting input content.

[0066] For example, referring to (b) of FIG. 2, the tablet computer 100 displays the editing interface 201 of the text note, the text area of the editing interface 201 includes text content such as text and picture, and further includes a mark figure of handwriting input, for example, the black shaded block in (b) of FIG. 2.

[0067] In this implementation, before the editing interface of the text note is generated based on the at least two layers, the content of the handwriting input and the text content are located in different layers (exemplarily, one layer can be one View or Layer), and the handwriting input content can be superimposed on the text content. However, after the terminal device changes the form, for example, the tablet computer rotates, for example, the folding screen mobile phone folds or unfolds, the text content and the handwriting input content can be misaligned. Exemplarily, as shown in (b) of FIG. 2, the mark figure is located above the text "note content", "list", and "style"; after the tablet computer rotates, as shown in (c) of FIG. 2, the mark figure is misaligned with the text "note content", "list", and "style". This is because the text content and the handwriting input content are located in different layers, and the two layers are independent, and the tablet computer independently layouts each layer. In different layout modes, the text content and the handwriting input content can be misaligned.

[0068] In another implementation, the tablet computer displays the editing interface of the text note, and the user can input rich text content such as text and audio to the editing interface. When the user needs to use handwriting input, the tablet computer displays the handwriting input interface of the text note, the handwriting input interface does not include the rich text content, and the user can input handwriting input content to the handwriting input interface in a handwriting manner; after the user stops handwriting input, the tablet computer no longer displays the handwriting input interface, and displays the updated editing interface, and the updated editing interface includes a picture generated according to the handwriting input content.

[0069] Exemplarily, referring to FIG. 3, the tablet computer 100 displays the editing interface 301 of the text note, and the editing interface 301 includes a picture 3021 and a text area 3022. The content of the picture 3021 is generated by handwriting track, and the text area 3022 includes text content. In one example, in response to a click operation of the user on the picture 3021, the tablet computer 100 displays a handwriting input interface 303, and the handwriting input interface 303 includes a handwriting area 3031, and the user can perform handwriting input in the handwriting area 3031 of the handwriting input interface 303. As shown in FIG. 3, the area of the handwriting area 3031 is large, and almost occupies the entire display area of the handwriting input interface 303, and the handwriting input interface 303 does not include a text area.

[0070] After the user ends the handwriting input, for example, receiving a click operation of the user on the "save" button 3032 in the handwriting input interface 303, the tablet 100 displays an updated editing interface 301, which includes a picture 3023 generated according to the content of the handwriting input of the user in the handwriting area 3031. It should be noted that the dashed boxes in the editing interface 301 and the handwriting input interface 303 shown in FIG. 3 are only used to indicate the range of the area, and in actual applications, the dashed boxes will not be displayed.

[0071] In this embodiment, the handwriting input content (handwriting track) is displayed in the editing interface in the form of a picture, and before the editing interface is generated, the handwriting input content and the text content are located in the same layer, thereby avoiding the problem of misalignment of the display of the text content and the handwriting input content after the tablet is rotated. However, the handwriting input needs to be operated in the handwriting input interface, and the switching between the editing interface and the handwriting input interface occurs, which makes the user experience poor.

[0072] The embodiment of the present application provides a method for displaying input content, which does not need handwriting input in the handwriting input interface, but can perform handwriting operation in the handwriting area of the editing interface of the text note, and after the handwriting input is ended, the handwriting input content in the handwriting area is saved and displayed in the updated editing interface in the form of a picture. Since the handwriting input content is displayed in the form of a picture, the handwriting input content and the text content are located in the same layer, thereby avoiding the problem of misalignment of the display of the text content and the handwriting input content after the tablet is changed in form. In addition, the switching between the editing interface and the handwriting input interface is reduced, thereby improving the user experience.

[0073] In one example, the tablet displays an editing interface one of the text note, the editing interface one includes a text area one, and the user can input rich text content such as text, symbol, picture, table, hyperlink, audio, etc. in the text area one. In response to the user's operation of adding a handwriting area (for example, the operation of clicking the handwriting button), the tablet displays an editing interface two of the text note, the editing interface two includes a text area two and a handwriting area; the user can input rich text content such as text, symbol, picture, table, hyperlink, audio, etc. in the text area two, and the user can write handwriting input content into the handwriting area by handwriting. The text area two corresponds to the text area one, the rich text content displayed in the text area two is the same as the rich text content displayed in the text area one, the starting positions of the text area two and the text area one can be different, and the sizes of the areas of the text area two and the text area one displayed on the screen can also be different. Optionally, the initial size (for example, width and height) of the handwriting area in the editing interface two is a preset value, the size of the handwriting area can be dynamically adjusted according to the handwriting input content; the size of the handwriting area can also be changed in response to the user's operation of adjusting the boundary of the handwriting area. In one example, the horizontal direction is the X axis, and the vertical direction is the Y axis, the width of the handwriting area represents the length of the handwriting area in the X axis direction, and the height of the handwriting area represents the length of the handwriting area in the Y axis direction.

[0074] For example, referring to (a) of FIG. 4A, the tablet 100 displays an editing interface 401 of a text note, and the editing interface 401 includes a title bar, an editing area, a toolbar, etc.; the title bar includes the title of the text note, the editing time, the classification option, etc.; in the editing interface 401, the entire editing area is a text area, for example, a text area 11, the user can input rich text content such as text, symbol, picture, table, hyperlink, audio, etc. in the text area 11, for example, the text area 11 includes text a; the bottom toolbar includes multiple controls, for example, a handwriting option 4021, a text tool 4022, an "add" button 4023, a "recording" button 4024, etc. It should be noted that the dashed box in the editing interface 401 shown in (a) of FIG. 4A is only used to indicate the range of the area, and is not displayed in actual application. Optionally, the editing interface 401 can be the editing interface 103 shown in FIG. 1, at this time, in response to the user's clicking operation on the "fold" button 1033, the virtual keyboard 1031 is not displayed. The handwriting option 4021 can be the handwriting option 1032 in FIG. 1, the text tool 4022 can be the text tool 1034 in FIG. 1, the "add" button 4023 can be the "add" button 1035 in FIG. 1, and the "recording" button 4024 can be the "recording" button 1036 in FIG. 1.

[0075] In one example, as shown in (b) of FIG. 4A, in response to the user's click operation on the handwriting option 4021, the tablet 100 displays an editing interface 403 of the text note, which includes a title bar, an editing area, a bottom toolbar, etc.; wherein the title bar in the editing interface 403 is the same as the title bar in the editing interface 401; the editing area includes a handwriting area 12 and a text area 11'; the user can write handwriting input content into the handwriting area 12 in a handwriting manner, and the user can input text, symbols, pictures, tables, hyperlinks, audio, etc. rich text content into the text area 11'; the bottom toolbar includes a handwriting tool button, and the user can click the handwriting tool button to select the line thickness, pen shape, color, etc. used for handwriting input in the handwriting area 12. In one implementation, after the user clicks the "add" button 4023, the tablet 100 adds the View of the handwriting area 12 in the View corresponding to the editing area, so as to realize adding the handwriting area in the editing area of the editing interface; accordingly, the display area of the text area in the editing area becomes smaller; for example, the display area of the text area 11' in (b) of FIG. 4A is smaller than the display area of the text area 11 in (a) of FIG. 4A; the rich text content included in the text area 11' is the same as the rich text content included in the text area 11. In one implementation, the View corresponding to the text area 11' is the View of the text area 11. Optionally, in the editing interface, the handwriting tool button of the bottom toolbar appears at the same time as the handwriting area; that is, when the editing area of the editing interface includes the handwriting area, the bottom toolbar includes the handwriting tool button.

[0076] It should be noted that the dashed box in the editing interface 403 shown in (b) of FIG. 4A is only used to indicate the range of the area, and is not displayed in actual application.

[0077] For example, as shown in (c) of FIG. 4A, the user can input text, numbers, graphics, etc. in the handwriting area 12 in a handwriting manner.

[0078] After the user ends the handwriting input in the handwriting area 12, the user can perform a save operation; for example, the save operation includes clicking an area other than the handwriting area in the editing interface, etc. For example, as shown in (c) of FIG. 4A, the user clicks the text area 11'.

[0079] In one implementation, as shown in FIG. 4B, in response to a user's save operation (e.g., a click operation within the text area 11') on the handwriting area 12, the tablet 100 displays an editing interface 404 of the text note, the editing area of the editing interface 404 including the picture 13 and the text area 11". The picture 13 is generated according to the handwriting input content (handwriting track) in the handwriting area 12, and in one example, the picture 13 has the same size as the handwriting area 12 and is displayed at the same position on the screen as the handwriting area 12; the text area 11" has the same display area and display position as the text area 11'. The elements in the picture 13 are generated according to the handwriting input content (handwriting track) in the handwriting area 12, and the text area 11" includes the text a.

[0080] In another implementation, as shown in FIG. 4C, in response to a user's save operation (e.g., a click operation within the text area 11') on the handwriting area 12, the tablet 100 displays an editing interface 404 of the text note, the editing area of the editing interface 404 including the picture 13 and the text area 11". The picture 13 has the same starting display position (top-left corner position of the picture) as the handwriting area 12, and the display area of the picture 13 is different from that of the handwriting area 12; for example, the height of the picture 13 is different from that of the handwriting area 12. For example, the handwriting input content in the handwriting area 12 is less, and the height of the handwriting track in the handwriting area is less than the height of the handwriting area, and the height of the picture 13 is determined according to the height of the area covered by the handwriting input content in the handwriting area. The elements in the picture 13 are generated according to the handwriting input content in the handwriting area 12. Correspondingly, the height of the text area 11" is different from that of the text area 11', and the text area 11" includes the text a.

[0081] Referring to FIGS. 4B and 4C, the editing interface 404 does not include a handwriting area, and correspondingly, does not include a handwriting tool button. For example, as shown in FIGS. 4B and 4C, the editing interface 404 includes a handwriting option 4051, a text tool 4052, an "add" button 4053, a "record" button 4054, and the like. The handwriting option 4051 can be the handwriting option 4021 in (a) of FIG. 4A, the text tool 4052 can be the text tool 4022 in (a) of FIG. 4A, the "add" button 4053 can be the "add" button 4023 in (a) of FIG. 4A, and the "record" button 4054 can be the "record" button 4024 in (a) of FIG. 4A.

[0082] For convenience of description, the picture generated according to the handwriting input content (handwriting track) in the handwriting area is referred to as a handwriting picture in the embodiments of the present application. For example, the picture 13 is a handwriting picture. The display area occupied by the handwriting picture is referred to as a handwriting display area in the embodiments of the present application.

[0083] Optionally, in an implementation, in response to the user's save operation on the handwriting region 12, the tablet 100 saves the information of the handwriting track in the handwriting region 12, for example, the information of the handwriting track in the handwriting region 12 is saved in a binary file; the tablet 100 further determines the display position and size of the picture 13 according to the display position and size of the handwriting region 12; and the tablet 100 further generates the elements in the picture 13 according to the handwriting track in the handwriting region 12.

[0084] It can be understood that the dashed box in the editing interface 404 shown in FIG. 4B and FIG. 4C is only used to indicate the range of the picture 13; in actual application, the dashed box can not be displayed.

[0085] In this example, during the user's handwriting input, the editing interface of the text note is always displayed, and the rich text content of the text region is always displayed on the editing interface, thus providing a smooth user experience. Moreover, the handwriting input content can be saved as elements of the picture 13; the picture 13 and the text a are located in the same layer, and will not be misaligned due to the rotation of the tablet 100.

[0086] In some embodiments, the user can perform an editing operation (for example, click the handwriting picture) on the handwriting picture to continue handwriting input in the handwriting region. In an example, the editing operation includes a click operation on the handwriting picture. For example, referring to FIG. 5A, the tablet 100 displays the editing interface 404 including the picture 13, and in response to the user's click operation on the picture 13, the tablet 100 displays the editing interface 403; the picture 13 in the editing interface 404 corresponds to the handwriting region 12 in the editing interface 403. Optionally, in an implementation, after receiving the user's click operation on the picture 13, the tablet 100 determines the position and size of the corresponding handwriting region (the handwriting region 12) according to the display position and size of the picture 13, and further generates the handwriting track of the corresponding handwriting region (the handwriting region 12) according to the information of the handwriting track of the handwriting region saved when the picture 13 is generated. For example, the handwriting tracks “1”, “2”, “3”, and “4” are displayed in the handwriting region 12. The user can also continue handwriting input in the handwriting region 12. For example, in response to the user's handwriting input in the handwriting region 12, the tablet 100 displays the updated editing interface 403, and in the updated editing interface 403, the handwriting track “5” is displayed in the handwriting region 12.

[0087] In this way, the user experience is that the position of the handwriting region does not change, and the text content of the text region is always displayed on the editing interface, thus providing a more convenient and smooth user experience.

[0088] In some embodiments, the user does not need to perform an editing operation on the handwritten picture, but can directly start handwriting input on the handwritten picture. The tablet receives the handwriting input of the user on the handwritten picture, deletes the handwritten picture, generates a handwriting area corresponding to the handwritten picture, and displays the handwriting input content in the handwriting area. For example, referring to FIG. 5B, the tablet 100 displays the editing interface 404 including the picture 13; in response to the handwriting input of the user on the picture 13, the tablet 100 displays the editing interface 403 including the handwriting area 12.

[0089] The picture 13 corresponds to the handwriting area 12. In an example, the display position and size of the handwriting area 12 are consistent with the display position and size of the picture 13. The handwriting input content (handwriting track) displayed in the handwriting area 12 includes two parts; one part of the handwriting track is the same as the element in the picture 13, for example, the handwriting tracks “1”, “2”, “3”, and “4”, and in an implementation, the part of the handwriting track is generated according to the information of the handwriting track saved when the picture 13 is generated; the other part of the handwriting track is generated according to the handwriting input of the user on the picture 13.

[0090] In this way, the user does not need to trigger the generation of the handwriting area through an editing operation (for example, a click operation on the handwritten picture), but can directly start handwriting input on the picture, which is convenient for the user to write.

[0091] In some embodiments, the tablet displays an editing interface one of the text note, the editing interface one including a handwritten picture one and a handwriting area two; optionally, the editing interface one further includes a text area. In response to the handwriting input of the user on the handwritten picture one, the tablet displays an editing interface two, the editing interface two including a handwriting area one and a picture two; optionally, the editing interface two further includes a text area. The handwriting area one corresponds to the picture one, and the handwriting area two corresponds to the picture two; the text content in the text area on the editing interface two is the same as the text content in the text area on the editing interface one.

[0092] In an example, the position and size of the handwriting area one are determined according to the position and size of the picture one, and the handwriting input content in the handwriting area one is generated according to the handwriting input of the user on the handwritten picture one and the information of the handwriting track of the handwriting area one saved by the tablet; the display position and size of the picture two are determined according to the position and size of the handwriting area two, and the elements of the picture two are generated according to the handwriting track in the handwriting area two.

[0093] As shown in FIG. 6(a), the tablet 100 displays an editing interface 405 of the text note, the editing interface 405 including the picture 14 and a handwriting area 15; wherein the picture 14 is a handwriting picture, and the elements of the picture 14 are generated according to the handwriting input content. For example, the picture 14 is the picture 13 in FIG. 4B; that is, the elements in the picture 14 are generated according to the handwriting input content (handwriting strokes “1”, “2”, “3”, “4”) in the handwriting area 12, and the elements of the picture 14 are the same as the handwriting strokes in the handwriting area 12. The handwriting area 15 includes the handwriting stroke b. Optionally, the editing interface 405 further includes a text area la.

[0094] In an example, the user performs handwriting input on the picture 14 (or clicks the picture 14), and the tablet 100 receives the handwriting input of the user on the picture 14 (or the operation of clicking the picture 14). As shown in FIG. 6(b), in response to the handwriting input of the user on the picture 14 (or the operation of clicking the picture 14), the tablet 100 displays an editing interface 406, the editing interface 406 including a handwriting area 16 and a picture 17.

[0095] The picture 17 is generated according to the handwriting area 15. In an example, the display position and size of the picture 17 are determined according to the display position and size of the handwriting area 15; for example, the display position and size of the picture 17 are consistent with the display position and size of the handwriting area 15. The elements of the picture 17 are the same as the handwriting strokes in the handwriting area 15. For example, the elements in the picture 17 are the handwriting stroke b.

[0096] The handwriting area 16 is generated according to the picture 14. In an example, the size (width and height) of the handwriting area 16 is the same as the size (width and height) of the picture 14, and the display position of the handwriting area 16 is the same as the display position of the picture 14. Optionally, the handwriting area 16 is generated in response to the handwriting input of the user on the picture 14, and the handwriting strokes displayed in the handwriting area 16 are generated based on two parts of handwriting strokes, one part of the handwriting strokes is generated according to the handwriting input of the user on the picture 14, and the other part of the handwriting strokes is the handwriting strokes that are the same as the elements in the picture 14. Optionally, the handwriting area 16 is generated in response to the clicking operation of the user on the picture 14, and the handwriting strokes displayed in the handwriting area 16 are the handwriting strokes that are the same as the elements in the picture 14.

[0097] In some embodiments, when the user uses the terminal device, the terminal device can change its form, for example, the tablet is rotated, for example, the folding screen phone is folded or unfolded.

[0098] In one example, when the tablet displays the editing interface of the text note, the tablet rotates. It can be understood that after the tablet rotates, the editing interface of the text note is re-laid out according to the change of the screen display area.

[0099] If the editing interface includes the handwriting area, the display position and size of the handwriting area change with the change of the screen display area.

[0100] For example, referring to FIG. 7A, the tablet 100 displays the editing interface 403 in portrait mode, and the editing area of the editing interface 403 includes the handwriting area 12 and the text area 11'; for example, the text area 11' includes the text a; optionally, the title bar of the editing interface 403 includes the "title" control, the "unclassified" control, etc.; and the bottom toolbar of the editing interface 403 includes a plurality of handwriting tool buttons, etc. The tablet rotates from portrait mode to landscape mode, and the tablet 100 displays the editing interface 407 in landscape mode. In one implementation, the editing interface 407 is obtained by re-laying out the editing interface 403 according to the screen display area in landscape mode. For example, as shown in FIG. 7A, the text area 11' of the editing interface 403 corresponds to the text area 18 of the editing interface 407, and the handwriting area 12 of the editing interface 403 corresponds to the handwriting area 19 of the editing interface 407.

[0101] Optionally, when the editing interface is re-laid out in response to the rotation of the tablet, the text a, the "title" control, the "unclassified" control, the handwriting toolbar, etc. can be laid out according to the rules of interface layout in conventional techniques.

[0102] For the handwriting area 12, in one implementation, when it is detected that the tablet 100 starts to rotate, picture one is determined according to the position and size of the handwriting area 12; optionally, the position of picture one in the editing interface 403 is the same as the position of the handwriting area 12 in the editing interface 403, and the size of picture one is the same as the size of the handwriting area 12; thus, the position and size of the handwriting area 12 in the editing interface 403 can be saved through picture one. For example, the top-left corner position of picture one is determined according to the top-left corner position of the handwriting area 12, and the size (e.g., width and height) of picture one is determined according to the size (e.g., width and height) of the handwriting area 12.

[0103] After detecting that the rotation of the tablet 100 is completed, the position and size of the handwriting area included in the editing interface of the text note in the horizontal screen are determined according to the picture one. In an implementation, the top-left corner position of the handwriting area (the handwriting area 19 in FIG. 7A) included in the editing interface in the horizontal screen is determined according to the top-left corner position of the picture one. It can be understood that the layout of the editing interface is different between the vertical screen and the horizontal screen of the tablet 100, and the position of the handwriting area 19 in the editing interface in the horizontal screen can be different from the position of the picture one in the editing interface in the vertical screen. In an implementation, the size of the handwriting area 19 is determined according to the size of the picture one. In an example, the height of the handwriting area 19 is the same as the height of the picture one, and the width of the handwriting area 19 is the width of the screen display area of the tablet 100 in the horizontal screen. In this way, the position and size of the handwriting area are saved through the picture one, and the position and size of the handwriting area after the rotation of the tablet are determined by combining the position and size of the handwriting area before the rotation of the tablet and the change of the screen display area of the tablet, thereby avoiding the problem of misalignment of handwriting input content after the rotation of the tablet.

[0104] In an example, the picture one is used to determine the position and size of the handwriting area 19, and the picture one is not displayed on the screen of the tablet 100; for example, a rotation animation is displayed on the screen during the rotation of the tablet 100.

[0105] In the method for displaying input content provided by the embodiments of the present application, a handwriting area can be added in the editing interface of the text note, and the handwriting area can be used to receive handwriting input content. After the user finishes handwriting input, the editing interface is updated, and the updated editing interface no longer includes the handwriting area, but displays a handwriting picture generated according to the handwriting input content of the handwriting area. In this way, if the terminal device changes in form such as rotation, folding, unfolding, etc., the handwriting picture participates in the re-layout of the user interface together with the rich text content (such as text, audio, hyperlinks, etc.), thereby avoiding the problem of misalignment of handwriting input content and rich text content. When the user performs handwriting input on the handwriting picture again, the editing interface is updated again, and the editing interface updated again includes a handwriting area generated according to the handwriting picture. The handwriting picture and the handwriting area are switched at the same position, and the rich text content is always displayed in the editing interface, thereby providing a smooth writing experience for the user.

[0106] If the editing interface includes the handwriting picture, the display position and size of the handwriting picture change with the change of the screen display area.

[0107] For example, as shown in FIG. 7B, the editing interface 404 of the text note displayed by the tablet 100 in the vertical screen includes the picture 13 and a text area 11”, and the text area 11” includes the text a.

[0108] The tablet 100 rotates from a portrait mode to a landscape mode, and the tablet 100 displays the editing interface 408 in the landscape mode. In an implementation, the editing interface 408 is obtained by re-arranging the editing interface 404 according to a screen display area in the landscape mode. For example, as shown in FIG. 7B, the text area 11' in the editing interface 404 corresponds to the text area 20 in the editing interface 408, and the picture 13 in the editing interface 404 corresponds to the picture 21 in the editing interface 408.

[0109] The editing interface 408 is generated by re-arranging the views (for example, the text area 11', the picture 13, and the like) in the editing interface 404 according to a screen display area in the landscape mode of the tablet 100. During the rotation of the tablet 100 to display the editing interface 408, no view is deleted or created.

[0110] The following describes a specific implementation of the method for displaying input content provided in the embodiments of the present application in detail with reference to the accompanying drawings.

[0111] The method for displaying input content provided in the embodiments of the present application can be applied to a terminal device supporting handwriting input. FIG. 8 shows a structural schematic diagram of a terminal device. The terminal device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a key 190, a camera 191, a display screen 192, and the like. The sensor module 180 can include a pressure sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an acceleration sensor, a gyroscope sensor, and the like.

[0112] It can be understood that the structure shown in the embodiments is not a specific limitation on the terminal device 100. In other embodiments, the terminal device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0113] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0114] The controller can be the nerve center and command center of the terminal device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0115] The memory can also be provided in the processor 110, for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or are recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0116] In some embodiments, the processor 110 can include one or more interfaces. The interface can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0117] It can be understood that the interface connection relationship between the modules in the embodiment is only illustrative and does not constitute a structural limitation on the terminal device. In other embodiments, the terminal device can also use different interface connection modes or combinations of multiple interface connection modes.

[0118] The charging management module 140 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the terminal device. The charging management module 140 charges the battery 142 while also providing power to the terminal device through the power management module 141.

[0119] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display screen 192, the camera 191, and the wireless communication module 160.

[0120] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0121] The terminal device 100 implements the display function through the GPU, the display screen 192, and the application processor. The GPU is a microprocessor for image processing, connected to the display screen 192 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0122] The display screen 192 is configured to display images, videos, and the like. The display screen 192 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Mini-LED, a Micro-OLED, a quantum dot light emitting diode (QLED), or the like.

[0123] In some embodiments, the display screen 192 can be configured to display a user interface, such as a desktop interface, a note application interface, and the like.

[0124] The terminal device 100 can implement the photographing function through the ISP, the camera 191, the video codec, the GPU, the display screen 192, and the application processor, and the like.

[0125] The camera 191 is configured to capture still images or videos. In some embodiments, the terminal device can include one or N cameras 191, where N is a positive integer greater than 1.

[0126] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0127] The external memory interface 120 can be configured to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, audio, video, and the like files can be saved in the external memory card.

[0128] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 performs various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 121. For example, in the embodiments of the present application, the processor 110 can execute the instructions stored in the internal memory 121, and the internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application program required by a function. The data storage area can store data created during the use of the terminal device, etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0129] The pressure sensor is used to sense a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be disposed on the display screen 192. There are many types of pressure sensors, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates with conductive material. When a force acts on the pressure sensor, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure according to the change of the capacitance. When a touch operation acts on the display screen 192, the terminal device 100 detects the intensity of the touch operation according to the pressure sensor. The terminal device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor.

[0130] The touch sensor, also known as a "touch panel". The touch sensor can be disposed on the display screen 192, and the touch sensor and the display screen 192 form a touch screen, also known as a "touch screen". The touch sensor is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the touch event type. The visual output related to the touch operation can be provided through the display screen 192.

[0131] In some embodiments, the touch operation of the user acting on the screen can be detected by the pressure sensor or the touch sensor, and text, graphics, etc. can be generated according to the touch operation.

[0132] The acceleration sensor can detect the acceleration of the terminal device 100 in each direction (generally three axes). When the terminal device 100 is stationary, the size and direction of gravity can be detected. It can also be used to identify the posture of the terminal device, and applied to landscape / portrait screen switching, pedometer, etc.

[0133] The gyroscope sensor can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor.

[0134] The keys 190 include a power key, a volume key, and the like. The keys 190 can be mechanical keys. They can also be touch keys.

[0135] In the embodiments of the present application, the terminal device described above is a terminal device that can run an operating system and install application programs. Optionally, the operating system running on the terminal device can be Android, iOS, Windows, and the like.

[0136] The software system of the terminal device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take a layered architecture system as an example to exemplarily illustrate the software structure of the terminal device 100.

[0137] FIG. 9 is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0138] The layered architecture divides software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Android runtime and system library, and a kernel layer.

[0139] The application layer can include a series of application packages. For example, camera, gallery, calendar, call, map, navigation, and the like. Exemplarily, the application layer can include a note application.

[0140] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0141] As shown in FIG. 9, the application framework layer can include a window manager service (WMS), an input management service, a content provider service, a view system, a resource management service, and the like.

[0142] The window manager service is used to manage window programs. For example, adding a window, deleting a window, modifying a window, and the like. The window manager service can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and take a screenshot, and the like.

[0143] The input management service is used to manage input events, such as distributing input events.

[0144] The content provider service is used to store and retrieve data, and make the data accessible to applications. The data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0145] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a note application includes a text note interface, which can include a view for displaying text and a view for displaying images, etc.

[0146] The resource management service provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc.

[0147] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the system.

[0148] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0149] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, security and exception management, and garbage collection.

[0150] The system library can include multiple functional modules. For example: surface management module (surface manager), synthesis rendering (surface flinger, SF) module, input synthesis (input flinger) module, media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL) etc.

[0151] The surface management module is used to manage the display subsystem, and provides fusion of 2D and 3D layers for multiple applications.

[0152] The compositing and rendering module is configured to synthesize and render layers according to interface display data (e.g., the number of layers included in the interface, display elements of each layer). The compositing and rendering module determines a compositing instruction (e.g., an OPENGL instruction or an HWComposer instruction) corresponding to each display data according to the display data, and calls the compositing instruction to generate each layer of the application interface and send the display.

[0153] The input compositing module is configured to generate an input event corresponding to a signal generated by an input device, and perform event compositing.

[0154] The sensor module is configured to generate sensor data corresponding to a signal collected by a sensor, and process the sensor data.

[0155] The media library supports playback and recording of a plurality of commonly used audio and video formats, and static image files. The media library can support a plurality of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0156] The three-dimensional graphics processing library is configured to implement three-dimensional graphics drawing, image rendering, compositing, and layer processing.

[0157] The 2D graphics engine is a drawing engine for 2D drawing.

[0158] The kernel layer is a layer between hardware and software. The kernel layer can include display drivers, sensor drivers, and keyboard drivers.

[0159] In an example, after receiving a user operation on the screen of the terminal device 100, the sensor driver sends an operation event corresponding to the user operation to the input compositing module, the input compositing module sends the operation event to the input management service, and the input management service sends the operation event to the WMS. The WMS determines an application corresponding to the operation event according to the window display positions of the applications, for example, the user operation acts on a user interface (an editing interface) of a note application, and the WMS sends the operation event to the note application. The note application performs corresponding processing according to the operation event, for example, updates interface data of the editing interface. The note application notifies the WMS that the interface data of the editing interface has been updated. The compositing and rendering module performs compositing and rendering according to the window management rule of the WMS, according to the interface data of the updated editing interface, generates a layer of the updated editing interface, and calls a compositing instruction to send the layer of the updated editing interface to the display.

[0160] The method of displaying input content provided in the embodiments of the present application will be described in detail below in combination with the function modules shown in FIG. 9.

[0161] In some embodiments, a handwriting area can be added in the editing interface of the text note.

[0162] For example, referring to the scenario shown in FIG. 4A, FIG. 10A shows a flowchart of adding a handwriting area in the editing interface. As shown in FIG. 10A, the method comprises the following steps.

[0163] S1001, after the note application is started, the tablet computer displays an editing interface one of a text note; the note application creates an Activity 1 when it is started.

[0164] In an example, the note application creates an Activity 1 when it is started. After the note application is started, the tablet computer displays a user interface of the note application, for example, an editing interface one of a text note. For example, the editing interface one is the editing interface 401 shown in (a) of FIG. 4A.

[0165] It can be understood that the elements in the user interface of the note application are all created based on View and ViewGroup. View and ViewGroup are combined through a tree structure to form the user interface of the note application (for example, the editing interface of a text note).

[0166] For example, referring to the editing interface 401 in (a) of FIG. 4A. The root node of the View tree structure corresponding to the Activity 1 is decorView, the child nodes of the decorView include a ViewContainer 1, the child nodes of the ViewContainer 1 include a WebView 1, and the WebView 1 is the View corresponding to the text area 11. Further, child nodes can be added to the WebView 1, for example, image files, audio files, etc.; in this way, it is possible to add picture, audio, and other rich text content in the text area 11.

[0167] Optionally, the child nodes of the ViewContainer 1 further include a ViewGroup 1 corresponding to the title bar, a ViewGroup 2 corresponding to the bottom toolbar, etc. For example, referring to (a) of FIG. 4A, the bottom toolbar includes a handwriting option 4021, a text tool 4022, an “add” button 4023, a “recording” button 4024, etc. Correspondingly, the child nodes of the ViewGroup 2 include a View corresponding to the handwriting option 4021, a View corresponding to the text tool 4022, a View corresponding to the “add” button 4023, a View corresponding to the “recording” button 4024, etc.

[0168] S1002, in response to the clicking operation of the user on the handwriting button, the sensor driver sends a clicking event to the input synthesis module.

[0169] The bottom toolbar of the editing interface 401 includes a handwriting option 4021. A user can click the handwriting option 4021 to create a handwriting area on the editing interface 401.

[0170] In an example, a sensor (e.g. a touch sensor, or a touch panel) detects a click operation of a user on a screen, and a sensor driver generates a click event according to the click operation and reports the click event to an input synthesis module. Optionally, the click event includes an action position of the click operation, an action intensity of the click operation, etc. It can be understood that the click operation includes a down operation and an up operation. In an implementation, the sensor driver generates a down event according to the down operation and generates an up event according to the up operation; and the sensor driver sends the click event to the input synthesis module includes sending the down event and the up event.

[0171] S1003. The input synthesis module sends the click event to an input management service, and the input management service sends the click event to a WMS.

[0172] S1004. The WMS sends the click event to a note application.

[0173] In an implementation, the WMS determines that the foreground application is the note application according to a window currently acquiring a focus, and sends the click event to the note application for processing.

[0174] S1005. The note application generates a canvas view (HandView) corresponding to the handwriting area, and adds the canvas view (HandView) to an activity 1 (Activity 1).

[0175] In an implementation, the note application receives the click event, and obtains the action position of the click operation, the action intensity of the click operation, etc. according to the click event. In an example, the click event is distributed to a View corresponding to the handwriting option 4021 along a View tree. The View corresponding to the handwriting option 4021 completes consumption of the click event.

[0176] In one example, the consumption of the click event includes generating a HandView and adding the HandView to the ViewContainer 1. Thus, the child nodes of the ViewContainer 1 include the WebView 1, the HandView, etc. The WebView 1 corresponds to the text area 11, and the HandView corresponds to the handwriting area 12. Optionally, the display position of the HandView is determined according to the position of the cursor on the display screen, and the size of the HandView is a preset value. It can be understood that the View includes the HandView, the WebView, etc., that is, the HandView is a kind of View.

[0177] Optionally, in one implementation, corresponding to the generation of the HandView, an image file 1 is also generated, which includes the information of the picture 1, such as the size of the picture 1, the elements of the picture 1, etc. For example, the size of the picture 1 is the same as the display size of the HandView, and the elements of the picture 1 are empty (i.e., the picture 1 is a blank image). The image file 1 is added to the WebView 1, so as to add a blank picture to the text area, which has the same size as the display size of the HandView. The display position of the blank picture is also the same as the display position of the HandView. In this way, it can be achieved that the display position of the text in the text area is changed as the handwriting area is added to the editing area. For example, as shown in (b) of FIG. 4A, after the handwriting area 12 is added to the editing area, the display position of the text a is moved downward.

[0178] S1006, the note application stores the data of the updated Activity 1 in the Graphic Buffer of the system library, and notifies the WMS that the Activity 1 has been updated.

[0179] In one implementation, the note application transmits the data of the updated Activity 1 to the Graphic Buffer of the system library through the ViewRootlmpl.java, that is, stores the data of the updated Activity 1 in the Graphic Buffer of the system library.

[0180] The note application also notifies the WMS that the Activity 1 has been updated.

[0181] S1007, when the next synchronization signal (Vsync) arrives, the synthesis rendering module acquires configuration information of the current image frame from the WMS, and performs synthesis and rendering based on the configuration information and the updated data of the Activity 1 saved in the Graphic Buffer to generate a layer of the updated editing interface.

[0182] S1008, the synthesis rendering module sends the layer of the updated editing interface to the display driver, and the display driver displays the updated editing interface (editing interface two) on the display screen according to the layer of the updated editing interface.

[0183] For example, the editing interface two is the editing interface 403 shown in (b) of FIG. 4A.

[0184] In some embodiments, after the user ends the handwriting input in the handwriting area, the handwriting area is saved in the editing interface in the form of a picture.

[0185] For example, as shown in FIG. 10B, the method further includes:

[0186] S1009, in response to the click operation of the user outside the handwriting area of the editing interface two, the sensor driver reports a second click event to the input synthesis module.

[0187] For example, referring to FIG. 4B, the tablet computer 100 displays the editing interface 403, and the editing interface 403 includes the handwriting area 12. The touch sensor detects the click operation of the user on the display screen, and the click operation is performed on an area outside the handwriting area 12 in the editing interface 403.

[0188] The sensor driver generates a second click event according to the click operation, and reports the second click event to the input synthesis module. Optionally, the second click event includes the action position of the click operation, the action intensity of the click operation, etc. It can be understood that the click operation includes a down operation and an up operation. In one implementation, the sensor driver generates a down event according to the down operation, and generates an up event according to the up operation; and the sensor driver sending the second click event to the input synthesis module includes sending the down event and the up event.

[0189] S1010, the input synthesis module sends the second click event to the input management service, and the input management service sends the second click event to the WMS.

[0190] S1011, the WMS sends the second click event to the note application.

[0191] S1012, the note application generates an image file one according to the HandView in the Activity 1, adds the image file one to the Activity 1, and deletes the HandView from the Activity 1.

[0192] The note application generates an image file one according to the HandView in the Activity 1, and the image file one includes information of the picture one, such as the size of the picture one, elements of the picture one, etc. Exemplarily, the image file is in JPG format.

[0193] In an example, the size of the picture one is the same as the display size of the HandView, and the elements of the picture one are generated according to the handwriting track on the HandView. Exemplarily, referring to FIG. 4B, the HandView is the View corresponding to the handwriting area 12, the picture one is the picture 13, the size of the picture 13 is the same as that of the handwriting area 12, and the elements of the picture 13 are the handwriting track in the handwriting area 12.

[0194] The image file one is added to the WebView 1, so as to add the picture one in the text area, and the size of the picture one is the same as the display size of the HandView, and the display position of the picture one is also the same as the display position of the HandView. Optionally, the blank picture in the WebView 1 is deleted. Further, the HandView is deleted from the ViewContainer 1. In this way, the handwriting area in the editing interface is replaced by the picture one.

[0195] In an implementation, before the HandView is deleted, a binary file is generated according to the handwriting track on the HandView, and the binary file saves the information of the handwriting track on the HandView. In this way, when the handwriting area is restored in the editing interface later, the handwriting track in the handwriting area can be restored according to the binary file.

[0196] S1013, the note application stores the data of the updated Activity 1 in the Graphic Buffer of the system library, and notifies the WMS that the Activity 1 has been updated.

[0197] S1014, when the next Vsync signal arrives, the compositing and rendering module obtains the configuration information of the current image frame from the WMS, and performs compositing and rendering based on the configuration information and according to the data of the updated Activity 1 saved in the Graphic Buffer, to generate the layer of the updated editing interface.

[0198] S1015, the synthesis rendering module sends the layer of the updated editing interface to the display driver, and the display driver displays the updated editing interface (editing interface three) on the display screen according to the layer of the updated editing interface.

[0199] For example, the editing interface three is the editing interface 404 shown in FIG. 4B.

[0200] In some embodiments, after the terminal device changes the form such as rotation, folding, unfolding, etc., the user interface is re-laid out, and each control in the editing interface, such as text, picture, handwriting area, etc., changes the display position according to the screen display area after the form change.

[0201] For example, FIG. 11A shows a flowchart of a method for displaying input content provided by an embodiment of the present application, in which the terminal device rotates. As shown in FIG. 11A, the method includes:

[0202] S1101, starting a note application.

[0203] In an example, the note application is started in response to a click operation of a user on an application icon.

[0204] S1102, the note application notifies the WMS to start Activity 1.

[0205] In an implementation, the note application calls the function startActivity to notify the WMS to start Activity 1.

[0206] S1103, the note application notifies the WMS to create Activity 1, and in the function onCreate 1, the note application registers an orientation listener to the WMS.

[0207] In an example, the note application calls the function onCreate to notify the WMS to create Activity 1.

[0208] In an implementation, in onCreate, the note application registers an orientation listener (onOrientationChangeListener) to the WMS. After the orientation listener is registered, the WMS will send notification information to the note application after the orientation of the terminal device changes.

[0209] Optionally, in other embodiments, the note application registers a posture listener to the WMS; the WMS will send notification information to the note application after the posture of the terminal device changes.

[0210] S1104, the sensor driver periodically reports the posture or orientation information of the terminal device to the WMS.

[0211] In an implementation manner, the orientation of the terminal device includes up, down, left and right. The orientation of the terminal device is taken as the orientation of the head of the terminal device (the part where the camera is arranged). For example, referring to FIG. 11B, the X axis, the Y axis and the Z axis of the terrestrial coordinate system are taken as references, where the positive direction of the X axis points to the east, the positive direction of the Y axis points to the north, and the positive direction of the Z axis, the positive direction of the X axis and the positive direction of the Y axis satisfy the right-hand rule. When the included angle between the orientation of the terminal device and the positive direction of the Z axis is less than a preset threshold, the orientation of the terminal device is upward; when the included angle between the orientation of the terminal device and the negative direction of the Z axis is less than the preset threshold, the orientation of the terminal device is downward; when the included angle between the orientation of the terminal device and the positive direction of the X axis is less than the preset threshold, the orientation of the terminal device is rightward; and when the included angle between the orientation of the terminal device and the negative direction of the X axis is less than the preset threshold, the orientation of the terminal device is leftward.

[0212] For example, the orientation of the terminal device can be represented by an orientation flag, where the orientation flag is 0, indicating upward, the orientation flag is 1, indicating rightward, the orientation flag is 2, indicating downward, and the orientation flag is 3, indicating leftward.

[0213] Optionally, the orientation of the terminal device can be determined according to the posture of the terminal device.

[0214] In an implementation manner, the sensor driver periodically reports the posture information of the terminal device to the window management service (WMS).

[0215] In an implementation manner, the sensor driver determines the orientation of the terminal device according to the posture of the terminal device, and periodically reports the orientation (Orientation) information of the terminal device to the window management service (WMS).

[0216] S1105, the WMS determines that the orientation of the terminal device changes, i.e., determines that the terminal device rotates, according to the posture or orientation information of the terminal device.

[0217] In an implementation manner, the WMS receives the posture information of the terminal device, and determines the orientation of the terminal device according to the posture information.

[0218] In an implementation manner, if it is determined that the current received orientation information is different from the last received orientation information, the WMS determines that the terminal device rotates.

[0219] S1106, after determining that the terminal device rotates, the WMS sends configuration change information to the note application.

[0220] Since the note application has registered the orientation towards the listener with the window management service, after determining that the terminal device rotates, the window management service sends the configuration change information to the note application. In an example, the WMS calls the function onConfigureChanged(), and sends the configuration change information to the note application, which carries the current orientation information of the terminal device.

[0221] At this time, the terminal device displays the editing interface two of the text note. In an example, referring to FIG. 7A, the editing interface two is the editing interface 403, and the editing interface 403 includes the handwriting area 12. At this time, the Activity 1 includes the canvas view (HandView).

[0222] S1107, the note application generates an image file one according to the canvas view (HandView) in the Activity 1, adds the image file one to the Activity 1, and deletes the canvas view (HandView) from the Activity 1.

[0223] The specific implementation of the note application can refer to the related description in S1012, which will not be described here.

[0224] S1108, the note application stores the data of the updated Activity 1 in the graphic buffer of the system library, and notifies the WMS that the Activity 1 has been updated.

[0225] S1109, when the next synchronization signal (Vsync) arrives, the synthesis rendering module obtains the configuration information of the current image frame from the WMS, and performs synthesis and rendering based on the configuration information and the data of the updated Activity 1 stored in the graphic buffer, to generate the layer of the updated editing interface.

[0226] Optionally, during the rotation of the terminal device, the layer of the editing interface is not displayed on the screen of the terminal device, and a rotation animation is displayed. For example, the rotation animation can be an animation generated according to a screenshot of the screen display interface when the terminal device starts to rotate. In an example, the layer of the rotation animation is displayed above the layer of the editing interface.

[0227] S1110, the note application generates a canvas view 2 (HandView 2) according to the image file one in the Activity 1, adds the canvas view 2 to the Activity 1, and deletes the image file one from the Activity 1.

[0228] It can be understood that after the terminal device rotates, the terminal device orientation and the screen display area change accordingly. The note application re-arranges Activity 1 according to the interface layout rule corresponding to the current orientation of the terminal device. For example, the editing interface 403 includes a text area 11, a handwriting area 12, a title bar, a bottom toolbar, etc.; correspondingly, Activity 1 includes a WebView 1, an image file 1, a ViewGroup 1 corresponding to the title bar, a ViewGroup 2 corresponding to the bottom toolbar, etc. Among them, the WebView 1, the ViewGroup 1 corresponding to the title bar, and the ViewGroup 2 corresponding to the bottom toolbar can be re-arranged according to the interface layout rule after the terminal device rotates, using a solution in a conventional technology, which will not be described in detail in the embodiment of the present application.

[0229] For the image file 1, the image file 1 includes information of the picture 1, such as the display position and size of the picture 1, etc.; the display position and size of the picture 1 are determined according to the display position and size of the handwriting area 12. In an implementation mode, the note application generates a canvas view 2 (HandView 2), which corresponds to the handwriting area after the terminal device rotates. In an example, the display position and size of the HandView 2 are determined according to the display position and size of the picture 1. For example, the position of the top-left vertex of the picture 1 is determined as the display position of the top-left vertex of the HandView 2, and the height of the picture 1 is determined as the height of the HandView 2. In an example, the handwriting track on the HandView 2 is generated according to the saved information of the handwriting track on the HandView. In this way, the display position, size and display content (handwriting track) of the HandView 2 are determined.

[0230] Further, the HandView 2 is added to the ViewContainer 1 of Activity 1. In this way, the child nodes of the ViewContainer 1 include the WebView 1, the canvas view 2 (HandView 2), etc.

[0231] Optionally, the image file 1 is deleted, and the saved information of the handwriting track on the HandView is deleted.

[0232] S1111, the note application stores the data of the updated Activity 1 (Activity 1) into the graphic buffer (Graphic Buffer) of the system library, and notifies the WMS that Activity 1 (Activity 1) has been updated.

[0233] S1112, when the next synchronization signal (Vsync) arrives, the synthesis rendering module acquires the configuration information of the current image frame from the WMS, and performs synthesis and rendering based on the configuration information and the updated data of Activity 1 saved in the graphic buffer, to generate the layer of the updated editing interface.

[0234] S1113, the synthesis rendering module sends the layer of the updated editing interface to the display driver, and the display driver displays the updated editing interface (editing interface three) on the display screen according to the layer of the updated editing interface.

[0235] For example, referring to FIG. 7A, after the tablet 100 is rotated, the editing interface 407 (editing interface three) is displayed, and the editing interface 407 includes the handwriting area 19 (HandView 2). The handwriting area 19 is a handwriting area corresponding to the handwriting area 12.

[0236] In some embodiments, when the handwriting area is displayed in the form of a picture on the editing interface, the editing interface is updated in response to the handwriting input of the user on the handwriting area (picture form), and the handwriting picture is replaced by an editable handwriting area. For example, referring to FIG. 5B, the tablet 100 displays the editing interface 404, and the editing interface 404 includes the picture 13; in response to the handwriting input of the user on the picture 13, the tablet 100 displays the editing interface 403, and the editing interface 403 includes the handwriting area 12. For example, referring to FIG. 6, the tablet 100 displays the editing interface 405, and the editing interface 405 includes the picture 14 and the handwriting area 15; in response to the handwriting input of the user on the picture 14, the tablet 100 displays the editing interface 406, and the editing interface 406 includes the handwriting area 16 and the picture 17; wherein the handwriting area 16 corresponds to the picture 14, and the picture 17 corresponds to the handwriting area 15.

[0237] When the handwriting area is displayed in the form of a picture on the editing interface, the user can directly start handwriting input on the handwriting area in the form of a picture (handwriting picture). In response to the handwriting input of the user on the handwriting area in the form of a picture, the terminal device updates the editing interface, and the updated editing interface includes a handwriting area corresponding to the handwriting picture.

[0238] In one example, before the editing interface is updated, the text area of the editing interface includes picture one (e.g., picture 13), and picture one is a handwritten picture. Correspondingly, ViewContainer 1 in Activity 1 includes WebView 1, and WebView 1 includes image file one, which includes the information of picture one. In response to the user's handwritten input on picture one, the editing interface is updated, and after the editing interface is updated, picture one is deleted, and a handwritten area is added. Correspondingly, image file one in WebView 1 of Activity 1 is deleted, and HandView is added in ViewContainer 1. HandView is the view corresponding to the handwritten area.

[0239] Before the editing interface is completely updated, the handwritten area has not been generated, that is, HandView has not been added in ViewContainer 1, and the canvas view (HandView) corresponding to the handwritten area will not receive the input event corresponding to the handwritten input. This will cause the first stroke of the handwritten input to be lost, and the first stroke of the handwritten input cannot form a handwritten track in the handwritten area.

[0240] The method for displaying input content provided by the embodiments of the present application, with reference to FIG. 12, before the editing interface is completely updated, that is, before the HandView corresponding to the handwritten area is generated, the input event on the handwritten picture is received by the web view (WebView), and the web view stores the received input event in the blocking queue, that is, the web view is the event production sub-thread of the blocking queue. After the HandView corresponding to the handwritten area is generated, the canvas view (HandView) can receive and process the input event. The canvas view (HandView) takes out the input event in the blocking queue as the event consumption sub-thread of the blocking queue. The canvas view processes according to the time sequence of the input event, and draws the handwritten track according to the input event, so that the loss of the handwritten input (e.g., the first stroke of the handwritten input) of the user on the handwritten picture can be avoided.

[0241] In the scenario that the user directly performs handwriting input on the handwriting picture, the canvas view of the handwriting region can process the input event corresponding to the received handwriting input only after the canvas view of the handwriting region is drawn, and generate handwriting input content according to the input event. For example, referring to FIG. 12, the webpage view receives the input event corresponding to the handwriting input on the handwriting picture, and saves the input event corresponding to the handwriting input on the handwriting picture to the blocking queue. After the canvas view of the handwriting region is drawn, the canvas view as an event consumption sub-thread takes out the input event in the blocking queue, and draws handwriting according to the input event. Since it takes a certain time to draw the canvas view of the handwriting region, the handwriting of the first stroke of the handwriting input is generated after a certain time interval from the handwriting input (for example, the first stroke of the handwriting input) of the user on the handwriting picture, which causes poor hand-following of the first stroke of the handwriting input of the user, and the user experience is that the handwriting of the first stroke is displayed after a period of time of writing the first stroke.

[0242] The method for displaying input content provided by the embodiment of the present application, the note application pre-draw a transition layer, which is located at the top layer of the editing interface of the note application. Before the handwriting region is added to the editing interface (the HandView corresponding to the handwriting region is successfully created), the transition layer is visible. The handwriting on the handwriting picture is drawn on the transition layer according to the handwriting input, so that the handwriting corresponding to the handwriting input on the handwriting picture can be displayed on the screen through the transition layer before the handwriting region is successfully created. After the handwriting region is successfully created, the canvas view of the handwriting region can process the input event corresponding to the handwriting input, draw handwriting on the canvas view according to the input event of the handwriting input, and set the transition layer to be invisible, so that the handwriting drawn on the canvas view is displayed on the screen.

[0243] For example, referring to FIG. 13, the tablet computer 100 displays the editing interface 405, which includes the picture 14, which is a handwriting picture; and the handwriting region 15. In response to receiving the handwriting input of the user on the picture 14, the transition layer is set to be visible. The handwriting on the handwriting picture is drawn on the transition layer according to the handwriting input of the picture 14, and the handwriting is displayed on the screen through the transition layer. In response to receiving the handwriting input of the user on the picture 14, the handwriting region 16 is added to the updated editing interface (the editing interface 406). After the canvas view corresponding to the handwriting region 16 is created, the note application draws handwriting on the canvas view of the handwriting region according to the handwriting input. The transition layer is set to be invisible, and the handwriting can be displayed on the screen through the canvas view of the handwriting region.

[0244] In this way, before the handwriting area is successfully created, the handwriting is drawn on the transition layer; after the handwriting area is successfully created, the transition layer is invisible, and the handwriting is drawn on the canvas view. Since the transition layer is pre-drawn, the handwriting is drawn on the transition layer in a short time, which can effectively shorten the time interval between the user handwriting input and the display of the handwriting on the display screen, and improve the hand following property of the user handwriting input.

[0245] For example, in the scenario shown in FIG. 6, FIG. 14 shows a flowchart of a method for displaying input content provided by an embodiment of the present application when the user starts handwriting input on the handwriting picture. As shown in FIG. 14, the method comprises the following steps:

[0246] S1401, in response to the handwriting input of the user on the handwriting picture, the input management service sends an input event to the window management service.

[0247] When the user performs handwriting input on the handwriting picture, the terminal device displays the editing interface four, and the editing interface four comprises the text area one and the handwriting picture. For example, in the scenario shown in FIG. 6, the tablet computer 100 displays the editing interface 405, and the editing interface 405 comprises the picture 14, which is the handwriting picture. The editing interface 405 further comprises the handwriting area 15.

[0248] For example, the input event can comprise a down event, an up event, a move event, etc. When the sensor detects a down operation, the sensor generates a down event, sends the down event to the input management service, and the input management service sends the down event to the window management service; when the sensor detects an up operation, the sensor generates an up event, sends the up event to the input management service, and the input management service sends the up event to the window management service; when the sensor detects a move operation, the sensor generates a move event, sends the move event to the input management service, and the input management service sends the move event to the window management service.

[0249] S1402, the window management service sends the input event to the note application.

[0250] In an implementation mode, the window management service determines that the foreground application is the note application according to the window currently acquiring the focus, and sends the click event to the note application for processing. In an example, the window management service calls the function onTouchEvent, and sends the input event to the note application through ViewRootlmpl.java in the function onTouchEvent.

[0251] S1403, the note application saves the received input event.

[0252] The note application receives an input event each time, and calls the function obtainevent to save the input event to the blocking queue.

[0253] In an implementation, the root node of the View tree structure corresponding to the Activity corresponding to the editing interface of the note application receives an input event, and distributes the input event to the next level in the View tree structure. In an example, the input event is sent to the web view (WebView) corresponding to the text area, and the web view (WebView) stores the input event in the blocking queue as an event generation sub-thread. In this way, the input event corresponding to the handwriting input on the handwriting picture is stored in the blocking queue before the handwriting area is successfully created; after the handwriting area is successfully created, the canvas view (HandView) corresponding to the handwriting area can obtain the input event from the blocking queue and draw the handwriting according to the input event; and the handwriting input (such as the first stroke of the handwriting input) on the handwriting picture is avoided from being lost.

[0254] On the other hand, one input action of a user on the display screen can include multiple input events. For example, the process of a user inputting a stroke on the display screen by handwriting can include a down event, multiple move events, and an up event; wherein the starting point of inputting a stroke corresponds to the down event, and the ending point of inputting a stroke corresponds to the up event. For example, the double-finger pressing action of a user on the display screen corresponds to two consecutive down events. For example, the double-finger dragging action of a user on the display screen corresponds to a down event, a move event, a down event, a move event, and so on. The note application saves the input event each time it is received, so that the input action of the user can be determined according to multiple input events.

[0255] S1404, if a down event is received, the note application determines the action position of the down event. If the action position of the down event is on the handwriting picture one, S1405 is executed; if the action position of the down event is not on the handwriting picture one, the web view processes the down event.

[0256] In an example, the terminal device displays the editing interface four of the note application, the editing interface four includes the text area one, and the text area one is implemented by the web view (WebView) 1. The editing interface four also includes the handwriting picture one, and the handwriting picture one is implemented by the image file one; optionally, the image file one is a child node of the WebView 1. The editing interface four also includes the handwriting area two, and the handwriting area two is implemented by the canvas view (HandView) 3. In an example, the HandView 3 and the WebView 1 are child nodes of the ViewContainer 1.

[0257] After receiving the press event, the note application calls the function getCurrentClickedImage to traverse the handwritten images in the text area to determine on which handwritten image the action position of the press event is located.

[0258] If it is determined that the action position of the press event is located on a handwritten image, the note application continues to wait for a move event and executes S1405; if it is determined that the action position of the press event is not located on a handwritten image, the note application processes the press event by the web view.

[0259] S1405, if a move event is received, the note application determines the action position of the move event. If the action position of the move event is located on the first handwritten image and the number of move events on the first handwritten image is greater than a preset number, the note application executes S1406-S1407, S1411-S1412 and S1421.

[0260] The note application determines the action position of the move event, for example, the note application calls the function getCurrentClickedImage to traverse the handwritten images in the text area to determine on which handwritten image the action position of the move event is located.

[0261] If it is determined that the action position of the move event is located on the first handwritten image, the note application determines the number of move events continuously received on the first handwritten image. If it is determined that the number of move events continuously received on the first handwritten image is greater than a preset number (for example, the preset number is 2), the note application executes S1406-S1407, S1411-S1412 and S1421. For example, the note application calls the function onClickImageAtGraffitiMode to execute S1406-S1407, S1411-S1412 and S1421.

[0262] If it is determined that the action position of the move event is not located on the first handwritten image, the note application processes the move event by the web view.

[0263] S1406, the note application sets the transition layer to be visible.

[0264] The note application pre-draw the transition layer. In one implementation, the size of the transition layer is the size of the editing area in the editing interface. The transition layer is located above all the layers in the editing interface.

[0265] The transition layer is set to be invisible by default. After receiving one press event and a preset number of move events on the first handwritten image, it indicates that the user performs handwriting input on the first handwritten image, and the note application sets the transition layer to be visible (setSwitchSmoothHandWritingViewVisible), so that the handwriting strokes drawn on the transition layer can be displayed on the screen.

[0266] S1407, the note application creates the handwriting region one according to the handwriting picture one.

[0267] In an example, the note application calls the function switchHandWritingView to create the handwriting region one. Optionally, the note application also generates the handwriting picture two according to the handwriting region two. In an example, the handwriting region 16 is created according to the handwriting picture 14 in the scenario shown in FIG. 6. Correspondingly, the handwriting picture 17 is generated according to the handwriting region 15.

[0268] In an implementation, the note application calls the function saveAndExitAsync to close the canvas view corresponding to the handwriting region two (the handwriting region 15), and calls the function addHandWritingLayout to create the canvas view corresponding to the handwriting region one (the handwriting region 16) according to the handwriting picture one (the picture 14). In an example, the note application can execute the operation of closing the canvas view corresponding to the handwriting region two and the operation of creating the canvas view corresponding to the handwriting region one asynchronously, i.e., the operation of creating the canvas view corresponding to the handwriting region one can be executed without waiting for the operation of closing the canvas view corresponding to the handwriting region two to be completed. Optionally, the note application also deletes the image file corresponding to the picture one (the picture 14), and generates the image file corresponding to the picture two (the picture 17) according to the handwriting region two (the handwriting region 15).

[0269] After the handwriting region one is created successfully, the canvas view corresponding to the handwriting region one can receive the input event corresponding to the handwriting input, and draw the handwriting track (the handwriting input content) according to the handwriting input (the input event) of the user.

[0270] S1411, after the handwriting region one is created successfully, the canvas view corresponding to the handwriting region one reads the input event in the blocking queue as the consumer sub-thread of the blocking queue, the note application draws the handwriting track on the canvas view according to the input event, and then deletes the processed input event in the blocking queue.

[0271] In one example, after the first handwriting area is successfully created, i.e. the HandView corresponding to the first handwriting area is generated, the note application calls the function transferTouchEventsToHandWritingView, in which the HandView corresponding to the first handwriting area reads the input events in the blocking queue (mTouchEventQueue) as a consumer thread of the blocking queue. The HandView calls the function drawTransferredEvents to consume the input events. Illustratively, the HandView draws the handwriting traces according to the input events. Then, the note application deletes the consumed input events in the blocking queue.

[0272] S1412, if the last input event in the blocking queue is the up event and all the input events in the blocking queue are processed, the note application sets the transition layer to be invisible.

[0273] The up event is received, indicating that the first pen drawing is completed. If all the input events in the blocking queue are processed, i.e. the mTouchEventQueue is empty, it indicates that the handwriting input on the handwriting picture is completed, i.e. there is no handwriting input currently. After the last input event in the blocking queue is the up event and all the input events in the blocking queue are processed, the first handwriting area is successfully created, and the canvas view corresponding to the first handwriting area can process the input events to draw the handwriting traces according to the input events, without losing the handwriting input of the user. The note application sets the transition layer to be invisible (setSwitchSmoothHandWritingViewInVisible), and the handwriting traces drawn on the canvas view corresponding to the first handwriting area can be displayed on the screen.

[0274] S1421, the transition layer reads the input events in the blocking queue and draws the handwriting traces according to the input events.

[0275] In one example, the note application calls the function onTouchEvent, in which the transition layer calls the function drawOneTransferredEvents to consume the input events. Illustratively, the transition layer reads the input events in the blocking queue (at this time, the input events are not deleted from the blocking queue by the canvas view) and draws the handwriting traces according to the input events.

[0276] It can be understood that, because it takes a long time to create a corresponding canvas view for a handwriting region, for the same input event in the blocking queue, the canvas view consumes the input event later than the transition layer. Therefore, the transition layer consumes the input event earlier than the canvas view, and when the transition layer reads the input event in the blocking queue, the input event has not been consumed by the canvas view from the blocking queue.

[0277] In addition, before the handwriting region is created, the handwriting strokes generated by the handwriting input are displayed on the screen through the transition layer. The transition layer is pre-drawn, and compared with drawing the handwriting track on the canvas view, drawing the handwriting strokes on the transition layer causes less delay, and provides a better follow-up experience for the user.

[0278] It can be understood that, in order to implement the above functions, the terminal device provided by the embodiments of the present application includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed in the present application, the embodiments of the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0279] The embodiments of the present application can divide the functions of the terminal device according to the above method examples, for example, each function module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be realized in the form of hardware or software function module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. When actually implemented, another division method can be used.

[0280] In an example, please refer to FIG. 15, which shows a possible structural schematic diagram of the terminal device involved in the above embodiments. The terminal device 2000 includes a processing unit 2010, a storage unit 2020, a display unit 2030 and a sensor unit 2040.

[0281] The processing unit 2010 is configured to control and manage the actions of the terminal device 2000. The storage unit 2020 is configured to store program codes and data of the terminal device 2000. The processing unit 2010 invokes the program codes stored in the storage unit 2020 to perform the steps in the above method embodiments. The display unit 2030 is configured to display the user interface of the terminal device 2000, such as the application interface of the note application, and the like. The sensor unit 2040 is configured to detect the user input, such as the user's handwriting input, and is also configured to detect the morphological changes of the terminal device, such as the rotation, folding, unfolding, and the like of the terminal device.

[0282] Of course, the unit modules in the terminal device 2000 described above include but are not limited to the processing unit 2010, the storage unit 2020, the display unit 2030, and the sensor unit 2040. For example, the terminal device 2000 can further include a communication unit, a power supply unit, an audio unit, and the like. The communication unit is configured to enable the terminal device 2000 to communicate with other electronic devices. The power supply unit is configured to supply power to the terminal device 2000. The audio unit is configured to play audio.

[0283] The processing unit 2010 can be a processor or a controller, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. The storage unit 2020 can be a memory or the like. The display unit 2030 can be a display screen or the like. The sensor unit 2040 can include a touch sensor, a pressure sensor, an acceleration sensor, a gyroscope sensor, and the like.

[0284] For example, the processing unit 2010 is a processor (such as the processor 110 shown in FIG. 8), the storage unit 2020 can be a memory (such as the internal memory 121 shown in FIG. 8), the display unit 2030 can be a display screen (such as the display screen 192 shown in FIG. 8), and the sensor unit 2040 can be a sensor module (such as the sensor module 180 shown in FIG. 8). The terminal device 2000 provided by the embodiments of the present application can be the terminal device 100 shown in FIG. 8. The above processor, memory, display screen, sensor, and the like can be connected together, such as through a bus. The processor invokes the program codes stored in the memory to perform the steps in the above method embodiments.

[0285] The embodiments of the present application further provide a chip system, which comprises at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (for example, a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (for example, a processor). Illustratively, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the terminal device can perform various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not specifically limit this.

[0286] The embodiments of the present application further provide a computer readable storage medium, which comprises computer instructions. When the computer instructions are run on the above electronic device, the electronic device performs various functions or steps performed by the terminal device in the above method embodiments.

[0287] The embodiments of the present application further provide a computer program product, which makes the computer perform various functions or steps performed by the terminal device in the above method embodiments when the computer program product is run on the computer.

[0288] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of the functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions.

[0289] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0290] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0291] In addition, each of the functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist alone physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0292] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a number of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and various other media that can store program codes.

[0293] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for displaying input content, applied to an electronic device, characterized in that, The method comprises: displaying a first interface; the first interface comprises a first text area, and the first text area comprises first text content; in response to a first operation on a first option in the first interface, creating a first view, and displaying a second interface; the second interface comprises a first handwriting area and a second text area, the second text area comprises the first text content, and the first view corresponds to the first handwriting area; in response to a first handwriting input in the first handwriting area, first handwriting input content is displayed in the first handwriting area; in response to a second operation on the second interface, deleting the first view, and displaying a third interface; the third interface comprises a first handwriting display area and a third text area, the first handwriting display area comprises the first handwriting input content, and the third text area comprises the first text content; in response to a third operation on the first handwriting display area of the third interface, creating a second view, and displaying a fourth interface; the fourth interface comprises the first handwriting area and the second text area, and the second view corresponds to the first handwriting area.

2. The method of claim 1, wherein, The third operation comprises a second handwriting input, and the method further comprises: in response to a second handwriting input on the first handwriting display area of the third interface, second handwriting input content is displayed in the first handwriting area of the fourth interface.

3. The method of claim 2, wherein, The method further comprises: in response to a second handwriting input on the first handwriting display area of the third interface, setting a preset layer as visible; after creating the second view, setting the preset layer as invisible; when the preset layer is visible, the fourth interface is generated based on the preset layer.

4. The method of claim 3, wherein a preset layer, and the method further comprises: in response to a second handwriting input on the first handwriting display area of the third interface, generating the second handwriting input content on the preset layer, and the preset layer is located above a layer where the second view is located.

5. The method of claim 4, wherein, The setting of the preset layer as invisible comprises: after generating the second handwriting input content on the layer where the second view is located, and without handwriting input being in progress, setting the preset layer as invisible.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: in a case where the second interface is displayed, in response to a form change of the electronic device, deleting the first view; after deleting the first view, creating a third view, and displaying a fifth interface; the fifth interface comprises a second handwriting area and a fourth text area, the third view corresponds to the second handwriting area, the second handwriting area comprises the first handwriting input content, and the fourth text area comprises the first text content.

7. The method of claim 6, wherein: before the deleting of the first view in response to the form change of the electronic device, the method further comprises: generating a first image according to the first view; the creating of the third view comprises: generating the third view according to the first image and a screen display area after the form change of the electronic device.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: In a case where the third interface is displayed, in response to the electronic device changing in form, a sixth interface is displayed; the sixth interface includes a second handwritten display area and a fifth text area, the second handwritten display area includes the first handwritten input content, and the fifth text area includes the first text content; from the electronic device changing in form to the sixth interface being displayed, there is no view being deleted or created.

9. The method of claim 1, wherein, The third operation includes a click operation, and the fourth interface is the same as the second interface.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: In response to a second operation on the second interface, a second image is created; the second image corresponds to the first handwritten display area in the third interface.

11. The method according to any one of claims 1 to 10, characterized in that, The third interface further includes a third handwritten area, the third handwritten area includes third handwritten input content, the fourth interface further includes a third handwritten display area, and the third handwritten display area includes the third handwritten input content; the method further includes: In response to a third operation on the first handwritten display area in the third interface, a third image is created, and the third image corresponds to the third handwritten display area.

12. The method according to any one of claims 1 to 11, characterized in that, The first handwritten display area and the first handwritten area are the same display area on the screen.

13. The method according to any one of claims 1 to 12, characterized in that, The first text content includes at least one of a character, a picture, an audio, a table, and a hyperlink.

14. The method according to any one of claims 1 to 13, characterized in that, The second operation on the second interface includes a click operation on an area in the second interface other than the first handwritten area.

15. The method of claim 7, wherein, The form change includes rotation, folding, or unfolding.

16. An electronic device, comprising: One or more processors; A display screen; A memory; The memory has one or more computer programs stored therein, and the one or more computer programs include instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-15. The computer program / instructions, when executed by the processor, implement the method of any one of claims 1-15.

17. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the method of any one of claims 1-15.

18. A computer program product comprising computer programs / instructions, characterized in that, ​

Images