Display method, electronic device and storage medium

Abstract

The present application provides a display method, an electronic device, and a storage medium. In the method, the display interface of the target application includes a canvas and a canvas boundary line, and the display interface is displayed based on the canvas view and the canvas boundary line view, and the canvas view is transparent. In response to the user's finger sliding from a first position to a second position when dragging the canvas boundary line, the canvas boundary line view is updated. After the canvas boundary line view is updated, the canvas boundary line is displayed at the second position of the application interface. After the canvas boundary line view is updated, the canvas view is updated according to the update of the canvas boundary line view. In this way, the update of the canvas boundary line view no longer depends on the update of the canvas view, which improves the hand tracking when the position of the canvas boundary line changes.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent terminals, and in particular to a display method, an electronic device and a storage medium. Background Art

[0002] As note-taking apps become more powerful, the user experience of note-taking apps is also becoming richer. For example, users can type text in the text notes of note-taking apps, paste content copied to the clipboard, add hyperlinks, add pictures, add audio, etc., and can also input text, graphics, and other content by handwriting. Summary of the invention

[0003] The present application provides a display method, an electronic device and a storage medium. In this method, when a user drags a canvas boundary line in a note application interface to adjust the canvas height, the update of the canvas boundary line does not depend on the update of the canvas height, thereby improving the hand tracking when the position of the canvas boundary line changes.

[0004] In a first aspect, an embodiment of the present application provides a display method. The method includes:

[0005] The electronic device displays a first interface of a target application, the first interface is displayed based on a first view and a second view, the first view corresponds to a target boundary line of a handwriting canvas, the second view corresponds to the canvas, and the second view is transparent; the target boundary line is a top boundary line or a bottom boundary line; the electronic device updates the first view in response to a finger position sliding from a first position to a second position when dragging the target boundary line; after updating the first view, the electronic device displays a second interface of the target application; in the second interface, the target boundary line is displayed at a second position; after updating the first view, the electronic device updates the height of the second view according to the update result of the first view.

[0006] For example, the target application may be a note application, and the first interface may be an editing interface of the note application, wherein the editing interface includes a handwriting area (also called a canvas). The first interface may include a first view and a second view, wherein the first view is used to display a canvas boundary line, and the second view is used to display the canvas.

[0007] Here, updating the first view may be understood as updating at least one of the size, position, etc. of the first view.

[0008] In this way, since the canvas is transparent and invisible in the display interface, what the user can directly see in the display interface is the canvas boundary line. After decoupling the two, the update of the canvas boundary line is no longer dependent on the update of the canvas height, so that when the user quickly drags the canvas boundary line, the canvas boundary line can also smoothly follow the finger sliding, and the canvas boundary line can slide and stay at the boundary position of the handwriting.

[0009] According to a first aspect, the electronic device updates the first view, including: the electronic device updates the first view through a sub-thread of a target application.

[0010] Exemplarily, the child thread may be a thread of the target application JavaScript end. In this way, the update of the canvas boundary line is no longer processed by the target application main thread with heavy tasks, thereby improving the response speed to the user dragging the boundary line.

[0011] According to the first aspect, or any implementation of the first aspect above, the electronic device updates the height of the second view according to the update result of the first view, including: updating the height of the second view according to the update result of the first view through the main thread of the target application.

[0012] Among them, the main thread can be the main thread of the target application Android terminal.

[0013] In this way, the canvas boundary line and the canvas are decoupled, and the update of the canvas boundary line is controlled and processed by the JS (JavaScript) end thread of the note application, and the update of the canvas is controlled and processed by the Android (Android) end thread of the note application.

[0014] According to the first aspect, or any implementation of the first aspect above, the method also includes: in the process of dragging the target boundary line, generating multiple canvas update tasks in sequence through the sub-thread of the target application, and adding the multiple canvas update tasks to the task queue in sequence; obtaining the canvas update task in the task queue through the main thread of the target application, and updating the height of the second view according to the canvas update task.

[0015] While the user is dragging the canvas boundary, the sub-thread of the note application continuously generates canvas height update tasks. If each canvas height update task is directly passed to the main thread for processing, the main thread task will be busy. Moreover, the production speed of the canvas height update task is greater than the consumption speed, which may cause task blocking and increase the probability of task loss. For this reason, a new task queue is added to the note application. After the sub-thread of the note application generates the canvas height update task, it puts the canvas height update task into the task queue. As a consumer of the task queue, the main thread reads the canvas height update task in the task queue and executes the canvas height update task.

[0016] According to the first aspect, or any implementation of the first aspect above, the main thread of the target application obtains a canvas update task in a task queue, and updates the height of the second view according to the canvas update task, including:

[0017] The main thread of the target application obtains multiple canvas update tasks from the task queue each time, and updates the height of the second view according to the last canvas update task among the multiple canvas update tasks.

[0018] In this way, when canvas update tasks need to be executed frequently and continuously, and the canvas update tasks take a long time, executing these canvas update tasks in sequence will affect the update operations of other UIs. The main thread can determine whether the old canvas update tasks are waiting to be executed. If so, it can remove them and give priority to the new canvas update tasks, thereby improving the utilization efficiency of the main thread and enhancing the system's response speed to user operations.

[0019] According to the first aspect, or any implementation of the first aspect above, the method further includes:

[0020] In response to the operation of stopping dragging the target boundary line and raising the hand, the electronic device updates the first view according to the hand-raising position through the sub-thread of the target application, and generates a target canvas update task; the electronic device passes the target canvas update task to the main thread of the target application through the sub-thread of the target application; the electronic device executes the target canvas update task through the main thread of the target application.

[0021] In this way, when the user raises his hand, the child thread updates the canvas boundary view according to the position of the finger raised, and the main thread can also update the canvas view height according to the latest generated canvas height update task, so that the position change of the canvas boundary line and the canvas height are more in line with user expectations.

[0022] According to the first aspect, or any implementation method of the first aspect above, the electronic device updates the first view according to the hand-raising position through the sub-thread of the target application, including: the sub-thread of the target application calculates the offset of the first view according to the current position of the finger; the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view; the sub-thread of the target application updates the first view according to the calibrated offset of the first view.

[0023] This embodiment calibrates the canvas boundary line offset calculated based on the finger sampling position to avoid the problem of the canvas boundary line offset exceeding the limit, and also avoids the problem that the user cannot accurately drag the canvas boundary line to the lowest or highest point of the handwriting.

[0024] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the bottom boundary line; the second position is higher than the first position; correspondingly, the subthread of the target application calibrates the offset of the first view according to the offset limit of the first view, including:

[0025] The sub-thread of the target application calculates the cumulative sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the cumulative sum is less than the minimum height value of the second view; the minimum height value is the difference between the current height of the second view and the first height value, and the first height value is the difference between the position of the target boundary line and the third position; the third position is the lower limit position to which the bottom boundary line of the canvas is allowed to offset; the sub-thread of the target application recalculates the offset of the first view based on the first position as the offset of the first view after calibration.

[0026] In this way, when the bottom border line of the canvas is dragged and slid, it is possible to avoid the situation where the bottom border line of the canvas stays at a position exceeding the lowest point of the handwritten trace.

[0027] According to the first aspect, or any implementation of the first aspect above, the third position is the lowest point position of the handwriting trajectory on the canvas; or, the third position is the fourth position, the fourth position is below the lowest point position, and the difference between the fourth position and the lowest point position is less than a preset threshold.

[0028] This can avoid the problem of the lowest point of the handwriting being covered due to the handwriting being too thick.

[0029] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the bottom boundary line; the second position is lower than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including: the sub-thread of the target application calculates the cumulative sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the cumulative sum is less than the maximum height value of the second view; the sub-thread of the target application recalculates the offset of the first view according to the maximum height value of the second view as the calibrated offset of the first view.

[0030] In this way, in the scene where the bottom boundary line of the canvas is dragged downward, the final position of the bottom boundary line of the canvas is restricted by the maximum height value of the canvas.

[0031] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the top boundary line; the second position is lower than the first position; correspondingly, the subthread of the target application calibrates the offset of the first view according to the offset limit of the first view, including:

[0032] The sub-thread of the target application calculates the cumulative sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the cumulative sum is less than the minimum height value of the second view; the minimum height value is the difference between the current height of the second view and the second height value, and the second height value is the difference between the fifth position and the position of the target boundary line; the fifth position is the upper limit position to which the top boundary line of the canvas is allowed to offset; the sub-thread of the target application recalculates the offset of the first view based on the fifth position as the offset of the first view after calibration.

[0033] In this way, in the scenario where the top border line of the canvas is dragged downward, it is possible to avoid the situation where the bottom border line of the canvas stays beyond the high and low points of the handwriting.

[0034] According to the first aspect, or any implementation of the first aspect above, the fifth position is the highest point position of the handwriting trajectory on the canvas; or, the fifth position is the sixth position, the sixth position is above the highest point position, and the difference between the sixth position and the highest point position is less than a preset threshold.

[0035] This can avoid the problem of the lowest point of the handwriting being covered due to the handwriting being too thick.

[0036] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the top boundary line; the second position is higher than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including: the sub-thread of the target application calculates the cumulative sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the cumulative sum is less than the maximum height value of the second view; the sub-thread of the target application recalculates the offset of the first view according to the maximum height value of the second view as the offset of the calibrated first view.

[0037] In this way, in the scene where the top border line of the canvas is dragged and slid, the final position of the top border line of the canvas is restricted by the maximum height value of the canvas.

[0038] In a second aspect, an embodiment of the present application provides an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the display method of the first aspect and any one of the first aspects.

[0039] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0040] In a third aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program, and when the computer program is executed on an electronic device, the electronic device executes the display method of the first aspect and any one of the first aspects.

[0041] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0042] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed, enables a computer to execute a display method as described in the first aspect or any one of the items in the first aspect.

[0043] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0044] In a fifth aspect, the present application provides a chip, the chip comprising a processing circuit and a transceiver pin, wherein the transceiver pin and the processing circuit communicate with each other through an internal connection path, and the processing circuit executes the display method of the first aspect or any one of the first aspects to control the receiving pin to receive a signal, and to control the sending pin to send a signal.

[0045] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0047] Figure 2 A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0048] Figure 3a A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0049] Figure 3bA schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0050] Figure 3c A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0051] Figure 4a A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0052] Figure 4b A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0053] Figure 5a A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0054] Figure 5b A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0055] Figure 5c A schematic diagram of a usage scenario of a note-taking application is shown as an example;

[0056] Figure 6a The following is a flowchart of a problem scenario of a note-taking application shown as an example;

[0057] Figure 6b A schematic diagram of the layout of a canvas view and a canvas boundary line view is shown as an example;

[0058] Figure 7a is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0059] Figure 7b A schematic diagram of the software structure of an electronic device is shown as an example;

[0060] Figure 8a is a schematic diagram of module interaction shown as an example;

[0061] Figure 8b is a schematic diagram of module interaction shown as an example;

[0062] Fig. 9 is a schematic diagram of module interaction shown as an example;

[0063] Fig.10 The figure is a schematic diagram of the dragging limit of the canvas boundary line shown as an example;

[0064] Fig.11 The flowchart of calculating and calibrating the offset of the canvas boundary line on the JS side is shown as an example;

[0065] Fig.12a A schematic diagram of updating a canvas boundary line view is shown as an example;

[0066] Figure 12b A schematic diagram of updating a canvas boundary line view is shown as an example;

[0067] Fig.13 The figure is a schematic diagram of an exemplary usage scenario of a note-taking application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0070] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0071] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0072] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0073] The display method provided in the embodiment of the present application can be applied to electronic devices that support handwriting input. For example, the electronic device can be a terminal device. The above-mentioned terminal devices can include mobile phones, tablet computers, laptops, personal computers (personal computers, PCs), ultra-mobile personal computers (ultra-mobile personal computers, UMPCs), handheld computers, netbooks, smart home devices (such as smart TVs, smart screens, large screens, smart speakers, smart air conditioners, etc.), personal digital assistants (personal digital assistants, PDAs), wearable devices (such as smart watches, smart bracelets, etc.), vehicle-mounted devices, virtual reality devices, etc., and the embodiments of the present application do not impose any restrictions on this.

[0074] In one example, a note taking application is installed on the terminal device. For example, taking the terminal device as a tablet computer, Figure 1 A user interface diagram of a note application on a tablet computer is shown. It is understandable that on different terminal devices, the note application may have different names, such as notebook, notepad, etc.

[0075] Reference Figure 1 In (1), the tablet computer displays a first user interface 101 of a note-taking application, and the first user interface 101 includes a first "add" control 1012. In response to a user clicking the first "add" control 1012, the tablet computer displays a second user interface 102 of the note-taking application, such as Figure 1 As shown in (2). Among them, the "text note" option 1021, the "handwritten note" option 1022, etc. are displayed on the second user interface 102. The user can click the "text note" option 1021 to create a new text note. The user can enter text, symbols, pictures, audio, hyperlinks, etc. on the text note editing interface. The user can also enter content (such as text, graphics, etc.) by handwriting on the text note editing interface. The user can also click the "handwritten note" option 1022 to create a handwritten note. The user can enter text, graphics, etc. by handwriting on the handwritten note user interface (canvas).

[0076] For example, refer to Figure 1In (2), in response to a click operation on the "text note" option 1021, the tablet computer displays the first editing interface 103 of the text note. Exemplarily, the first editing interface 103 includes a text area, and the user can enter text, symbols, pictures, hyperlinks, audio and other rich text content in the text area. In one implementation, the first editing interface 103 includes a virtual keyboard 1031, and the user can enter text, numbers, symbols and other rich text content into the text area of ​​the first editing interface 103 by clicking virtual keys in the virtual keyboard 1031; in another implementation, the terminal device includes an input device, such as a keyboard, and the user can enter text, numbers, symbols and other rich text content into the text area of ​​the first editing interface 103 through the keyboard.

[0077] In one implementation, the first editing interface 103 includes a first handwriting option 1032. After the user clicks the first handwriting option 1032, the user can make handwriting input in the handwriting area of ​​the first editing interface 103; after the tablet computer receives the user's handwriting input in the handwriting area, the handwriting area will display the user's handwriting trajectory (text, numbers, symbols, graphics, etc.); the handwriting trajectory generated by the user's handwriting input will be retained in the handwriting area and will not disappear when the handwriting trajectory is drawn.

[0078] Exemplarily, in some implementations, the first editing interface 103 may further include other controls, for example, the first editing interface 103 may further include a "fold" control 1033, and in response to a click operation on the "fold" control 1033, the virtual keyboard 1031 is no longer displayed on the editing interface. For example, the first editing interface 103 may further include a text tool 1034, and the user may modify the font of the text in the text area by clicking the text tool 1034. For example, the first editing interface 103 may further include a second "add" control 1035, and in response to a click operation on the second "add" control 1035 by the user, options such as "gallery", "table", and "hyperlink" may be displayed on the editing interface, and the user may add a picture to the text area of ​​the editing interface by clicking the "gallery" option; the user may add a table to the text area of ​​the editing interface by clicking the "table" option; the user may add a hyperlink to the text area of ​​the editing interface by clicking the "hyperlink" option, etc. For example, the editing interface may further include a "recording" control 1036, and the user may add a recording file to the text area of ​​the editing interface by clicking the "recording" control 1036.

[0079] In one implementation, the tablet computer displays an editing interface for text notes, and the user can input rich text content such as text and audio into the editing interface. When the user needs to use handwriting input, the tablet computer displays a handwriting input interface for text notes, which does not include rich text content, and the user can input handwriting input content into the handwriting input interface by handwriting. After the user stops handwriting input, the tablet computer no longer displays the handwriting input interface, and displays an updated editing interface, which includes a picture generated according to the handwriting input content.

[0080] For example, refer to Figure 2 , the tablet computer displays a second editing interface 201 of a text note, and the second editing interface 201 includes a first picture 2011 and a first text area 2012. The content of the first picture 2011 is generated by a handwriting trajectory, and the first text area 2012 includes text content. In one example, in response to a user clicking on the first picture 2011, the tablet computer displays a handwriting input interface 202, and the handwriting input interface 202 includes a first handwriting area 2021. The user can perform handwriting input in the first handwriting area 2021 of the handwriting input interface 202. Figure 2 As shown in (2), the first handwriting area 2021 is relatively large, almost occupying the entire display area of ​​the handwriting input interface 202, and the handwriting input interface 202 does not include a text area.

[0081] After the user finishes handwriting input, for example, the tablet computer displays the third editing interface 203 after receiving the user's click operation on the "Save" control 2022 in the handwriting input interface 202. The third editing interface 203 includes a second picture 2023, which is generated based on the handwriting input content of the user in the first handwriting area 2021. It should be noted that Figure 2 The dashed boxes in the second editing interface 201 and the handwriting input interface 202 are only used to indicate the area range. In actual application, the dashed boxes will not be displayed. Some dashed boxes in the following figures are also only used to indicate the area range. In actual application, the dashed boxes will not be displayed, which will not be repeated.

[0082] In this embodiment, when the user selects the handwriting input function, the display interface of the tablet computer jumps from the second editing interface 201 of the text note to the handwriting input interface 202, so that the user can complete the handwriting input. As mentioned above, the handwriting input interface 202 only includes the content of the user's handwriting input, and does not include the text content included in the first text area 2012. In other words, when the user performs the handwriting input operation in the handwriting input interface 202, the existing content in the text note (i.e., the text content included in the first text area 2012) cannot be directly viewed. If the user needs to see the existing content in the text note, it is necessary to click the "Save" control 2022 so that the tablet computer displays the third editing interface 203. In this case, if the user still needs to perform handwriting input based on the original handwriting input content, the second picture 2023 can be clicked to make the tablet computer display the handwriting input interface (the interface includes the handwriting input content corresponding to the second picture 2023), and then the user can adjust the handwriting input content. In this way, the user's operation is relatively cumbersome, especially when the user needs to adjust his handwriting input content multiple times according to the existing content in the text note, resulting in a poor user experience.

[0083] In order to solve the above problem, the embodiment of the present application provides a method for displaying input content, which does not require the user to perform handwriting input in the handwriting input interface. The user can perform handwriting input operations in the handwriting area of ​​the text note editing interface. After the user completes the handwriting input operation, the handwriting input content in the handwriting area is saved and displayed in the updated editing interface in the form of a picture.

[0084] In one example, the tablet computer displays a fourth editing interface of a text note, the fourth editing interface includes a second text area, and the user can enter rich text content such as text, symbols, pictures, tables, hyperlinks, audio, etc. in the second text area. In response to the user's operation of adding a handwriting area (for example, the operation of clicking a handwriting control), the tablet computer displays a fifth editing interface of a text note, the fifth editing interface includes a third text area and a handwriting area; the user can enter rich text content such as text, symbols, pictures, tables, hyperlinks, audio, etc. in the third text area, and the user can write handwriting input content into the handwriting area by handwriting. Among them, the third text area corresponds to the second text area, the rich text content displayed in the third text area is the same as that displayed in the second text area, the starting position of the third text area and the second text area can be different, and the area size displayed on the screen of the third text area and the second text area can also be different. Optionally, the initial size (such as width and height) of the handwriting area in the fifth editing interface is a preset value, and the size of the handwriting area can be dynamically adjusted according to the handwriting input content; the size of the handwriting area can also be changed in response to the user's operation of adjusting the boundary of the handwriting area. In one example, with the horizontal direction as the X-axis and the direction perpendicular to the ground as the Y-axis, the width of the handwriting area represents the length of the handwriting area in the X-axis direction, and the height of the handwriting area represents the length of the handwriting area in the Y-axis direction.

[0085] For example, refer to Figure 3a In (1), the tablet computer displays a fourth editing interface 301 of a text note, and the fourth editing interface 301 includes a first title bar 3011, a first editing area 3012, a first toolbar 3013, etc.; wherein the first title bar 3011 includes a title of the text note, an editing time, a classification option, etc.; in the fourth editing interface 301, the entire first editing area is a text area, such as a second text area 31, and a user can enter text, symbols, pictures, tables, hyperlinks, audio, and other rich text content in the second text area 31. Exemplarily, the second text area 31 includes text a; the first toolbar 3013 includes multiple controls, for example, a second handwriting option 30131, a text tool, an "add" control, a "recording" control, etc.

[0086] In one example, if Figure 3aAs shown in (2), in response to the user clicking the second handwriting option 30131, the tablet computer displays the fifth editing interface 302 of the text note, and the fifth editing interface 302 includes a second title bar 3021, a second editing area 3022, a bottom toolbar 3023, etc. Among them, the second title bar 3021 can be the same as the first title bar 3011; the second editing area 3022 includes a second handwriting area 32 and a third text area 31'; the user can write handwriting input content to the second handwriting area 32 by handwriting, and the user can enter rich text content such as text, symbols, pictures, tables, hyperlinks, audio, etc. in the third text area 31'; the bottom toolbar 3023 includes a handwriting tool control, and the user can click the handwriting tool control to select the line thickness, pen shape, color, etc. used for handwriting input in the second handwriting area 32. In one implementation, after the user clicks the second handwriting option 30131, the tablet computer adds a View of the second handwriting area 32 in the View corresponding to the editing area, thereby adding a handwriting area to the editing area of ​​the editing interface; accordingly, the display area of ​​the text area in the editing area becomes smaller. For example, Figure 3a The display area of ​​the third text area 31' in (2) is smaller than Figure 3a The display area of ​​the second text area 31 in (1); the rich text content included in the third text area 31' is the same as the rich text content included in the second text area 31. In one implementation, the View corresponding to the third text area 31' is the View of the second text area 31. Optionally, in the editing interface, the handwriting tool control of the bottom toolbar appears at the same time as the handwriting area; that is, when the editing area of ​​the editing interface includes the handwriting area, the bottom toolbar includes the handwriting tool control.

[0087] For example, refer to Figure 3a In the sixth editing interface 303 shown in (3), the user can input text, numbers, graphics, etc. by handwriting in the second handwriting area 32. After the user finishes the handwriting input in the second handwriting area 32, the user can perform a save operation. For example, the save operation can be an operation of clicking an area outside the handwriting area in the editing interface, such as the save operation can be an operation of the user clicking the third text area 31'.

[0088] In one implementation, Figure 3bAs shown, in response to the user's save operation on the second handwriting area 32 (for example, a click operation in the third text area 31'), the tablet computer displays the seventh editing interface 304 of the text note, and the editing area of ​​the seventh editing interface 304 includes the third picture 33 and the fourth text area 31''. The third picture 33 is generated based on the second handwriting area 32. In one example, the third picture 33 is the same size as the second handwriting area 32, and the positions where they are displayed on the screen are also the same; the display area and display position of the fourth text area 31'' can be the same as those of the third text area 31'. The elements in the third picture 33 are generated based on the handwriting input content (handwriting trajectory) in the second handwriting area 32, and the fourth text area 31'' includes text a.

[0089] In another implementation, Figure 3c As shown, in response to the user's save operation on the second handwriting area 32 (for example, a click operation in the third text area 31'), the tablet computer displays the eighth editing interface 304' of the text note, and the editing area of ​​the eighth editing interface 304' includes the fourth picture 33' and the fifth text area 31'''. The fourth picture 33' has the same starting display position (upper left vertex position of the picture) as the second handwriting area 32, and the display area of ​​the fourth picture 33' is different from that of the second handwriting area 32. For example, the height of the fourth picture 33' is different from that of the second handwriting area 32. Exemplarily, there is less handwriting input content in the second handwriting area 32, and the height covered by the handwriting track in the handwriting area is less than the height of the handwriting area. The height of the fourth picture 33' is determined according to the height of the area covered by the handwriting input content in the handwriting area. The elements in the fourth picture 33' are generated according to the handwriting input content in the second handwriting area 32. Accordingly, the height of the fifth text area 31''' is different from that of the third text area 31'', and the fifth text area 31''' includes the text a.

[0090] Reference Figure 3b and Figure 3c The seventh editing interface 304 and the eighth editing interface 304' do not include a handwriting area, and accordingly, do not include a handwriting tool control. Figure 3b As shown, the seventh editing interface 304 includes handwriting options, text tools, an “add” control, a “recording” control, etc. The eighth editing interface 304 ′ is similar and will not be described in detail.

[0091] For the convenience of description, in the embodiment of the present application, the picture generated according to the handwriting input content (handwriting trajectory) of the handwriting area is called a handwriting picture. For example, the third picture 33 and the fourth picture 33' are handwriting pictures. In the embodiment of the present application, the display area occupied by the handwriting picture is called a handwriting display area.

[0092] Optionally, in one implementation, in response to the user's save operation on the second handwriting area 32, the tablet computer saves information of the handwriting trajectory within the second handwriting area 32, for example, the information of the handwriting trajectory within the second handwriting area 32 is saved in a binary file; the tablet computer also determines the display position and size of the third picture 33 (or the fourth picture 33') according to the display position and size of the second handwriting area 32; the tablet computer also generates elements in the third picture 33 (or the fourth picture 33') according to the handwriting trajectory within the second handwriting area 32.

[0093] It can be understood that the boundary line frame of the third picture 33 (or the fourth picture 33 ′) in the figure is only used to indicate the range of the picture, and in actual application, the boundary line frame may not be displayed.

[0094] In this example, when the user is inputting handwriting, the editing interface of the text note is always displayed, and the rich text content of the text area is always displayed on the editing interface, so that the user can always view the original content in the text area when inputting handwriting, bringing a smooth user experience.

[0095] After the handwriting area is saved, that is, after the handwriting image corresponding to the handwriting area is generated, the user can also adjust the first input content. For example, the user can continue to perform the handwriting input operation based on the existing handwriting input content, or delete, modify, etc. the existing handwriting input.

[0096] In some embodiments, the user performs an editing operation on the handwritten image, so that the tablet computer displays a handwriting area corresponding to the handwritten image and displays the original handwriting input content in the handwriting area. Exemplarily, the editing operation may be a click operation on the handwritten image.

[0097] For example, refer to Figure 4a In (1), the tablet computer displays the seventh editing interface 304 of the text note, and the seventh editing interface 304 includes the third handwritten image 33. In response to the user's click operation on the third handwritten image 33, the tablet computer displays the ninth editing interface 305 of the text note, such as Figure 4aAs shown in (2). The third picture 33 in the seventh editing interface 304 corresponds to the second handwriting area 32 in the ninth editing interface 305. Optionally, in one implementation, after receiving a click operation on the third picture 33 by the user, the tablet computer determines the position and size of the corresponding handwriting area (that is, the second handwriting area 32) according to the display position and size of the third picture 33, and also generates a handwriting trajectory of the corresponding handwriting area (that is, the second handwriting area 32) according to the information of the handwriting trajectory of the handwriting area saved when the third picture 33 is generated. Exemplarily, the handwriting trajectories "1", "2", "3", and "4" are displayed in the second handwriting area 32. The user can also continue to input handwriting in the second handwriting area 32. Exemplarily, in response to the user's handwriting input in the second handwriting area 32, the tablet computer displays the updated tenth editing interface 306, which can be referred to Figure 4a In the updated tenth editing interface 306 , the handwriting track “5” is displayed in the second handwriting area 32 .

[0098] In some embodiments, the user does not need to perform an editing operation on the handwritten image, but can directly start handwriting input on the handwritten image. The tablet computer receives the user's handwriting input on the handwritten image, deletes the handwritten image, generates a handwriting area corresponding to the handwritten image, and displays the original handwriting input content in the handwriting area. In this way, the user does not need to trigger the generation of the handwriting area through an editing operation (such as a click operation on the handwritten image) first, but can directly start handwriting input on the image, which is convenient for the user to write.

[0099] For example, refer to Figure 4b In (1), the tablet computer displays the seventh editing interface 304 of the text note, and the seventh editing interface 304 includes the third picture 33. In response to the user's handwriting input on the third picture 33, the tablet computer displays the eleventh editing interface 307 of the text note, and the eleventh editing interface 307 includes the third handwriting area 34. Figure 4b (2). The third picture 33 corresponds to the third handwriting area 34. In one example, the display position and size of the third handwriting area 34 may be consistent with the display position and size of the third picture 33. The handwriting input content (handwriting trajectory) displayed in the third handwriting area 34 includes two parts; one part of the handwriting trajectory is the same as the elements in the third picture 33, for example, the handwriting trajectory "1", "2", "3", "4", which is generated based on the information of the handwriting trajectory saved when the third picture 33 is generated; the other part of the handwriting trajectory is generated based on the handwriting input of the user on the third picture 33.

[0100] For the convenience of description, in the embodiment of the present application, the handwriting area will be referred to as a canvas or a handwriting canvas. In the embodiment of the present application, the user can adjust the height of the canvas by custom dragging (or dragging) the canvas boundary line to meet the user's use requirements for the canvas. It can be understood that the border line of the canvas may include a top border line of the canvas and a bottom border line of the canvas. The user can adjust the height of the canvas by dragging the top border line of the canvas and / or the bottom border line of the canvas, that is, adjust the size of the canvas.

[0101] The following explanation is given using an example where a user drags the bottom border line of the canvas.

[0102] Reference Figure 5a In (1), the tablet computer displays the twelfth editing interface 401 of the note application, and the twelfth editing interface 401 includes a canvas 41, a bottom border line 4101 of the canvas, and a top border line 4102 of the canvas. It can be understood that the area between the bottom border line 4101 of the canvas and the top border line 4102 of the canvas is the canvas 41 that supports user handwriting input. In the twelfth editing interface 401, the states of the bottom border line 4101 of the canvas and the top border line 4102 of the canvas are both in the initial state. The user can adjust the height of the canvas 41, that is, adjust the size of the canvas 41, by long pressing and dragging the bottom border line 4101 of the canvas and / or the top border line 4102 of the canvas.

[0103] For example, Figure 5a As shown in (1), the user long presses the bottom border line 4101 of the canvas, and specifically long presses the hot zone of the bottom border line 4101 of the canvas. In response to the user operation, the tablet computer displays the thirteenth editing interface 402 of the note application, such as Figure 5a In the thirteenth editing interface 402, the state of the bottom border line 4101 of the canvas is the adjustment state. Exemplarily, when the state of the bottom border line 4101 of the canvas is switched from the initial state to the adjustment state, the line of the bottom border line 4101 of the canvas becomes thicker and / or the color becomes darker, etc., so as to prompt the user that the state of the bottom border line of the canvas has changed.

[0104] Continue to refer to Figure 5a In (2) and (3), the user slowly drags the bottom border line 4101 of the canvas downward. In response to the user operation, the bottom border line 4101 of the canvas moves downward, and the height of the canvas 41 is adjusted according to the position of the bottom border line 4101 of the canvas. It can be understood that as the bottom border line 4101 of the canvas moves downward, the height of the canvas 41 increases. The user can release the canvas after dragging the bottom border line 4101 to the desired position. For example, refer to Figure 5aIn (3), the tablet computer displays the fourteenth editing interface 403 of the note application, and the user has dragged the bottom boundary line 4101 of the canvas to the desired position. In response to the user's hand-off operation, the tablet computer displays the fifteenth editing interface 404 of the note application, which can be referred to as Figure 5a In the fifteenth editing interface 404, the state of the bottom border line 4101 of the canvas is restored to the initial state. At this point, the user has completed the size adjustment of the canvas.

[0105] In this way, when the user slowly drags the canvas boundary line, the display position of the canvas boundary line in the editing interface is updated accordingly (that is, the canvas boundary line slides in the editing interface), and the height of the canvas will also change accordingly. Figure 5a In the scenario shown in which the user drags the bottom border line of the canvas, the sliding of the canvas border line and the finger sliding are basically consistent, the hand tracking is good, and the user can drag the bottom border line of the canvas to the bottom border position of the handwriting (that is, the position corresponding to the lowest point of the handwriting track). However, in actual application, if the user drags the canvas border line with his finger too fast or too much, the canvas border line may be stuck in sliding, or the canvas border line may not stop accurately at the position required by the user (for example, the bottom border line of the canvas cannot be dragged to the bottom border position of the handwriting).

[0106] Figure 5b The following example shows the problem scenario of the canvas border line sliding update being stuck. Figure 5b As shown in (1), the tablet displays the sixteenth editing interface 405 of the note application, and the user uses his finger to drag the bottom border line 4101 of the canvas downward. When the user slides his finger to position 1 of the editing interface, the bottom border line 4101 of the canvas is also displayed at position 2 of the editing interface. The difference between position 1 and position 2 is greater than or equal to a preset threshold, and the difference between the two can be perceived by the user or even clearly perceived. For example, Figure 5b As shown in (2), the tablet displays the seventeenth editing interface 406 of the note application, and the user uses his finger to greatly drag the bottom boundary line 4101 of the canvas upward. When the user slides his finger to position 3 of the editing interface, the bottom boundary line 4101 of the canvas is also displayed at position 4 of the editing interface. Among them, the difference between position 3 and position 4 is greater than or equal to the preset threshold, and the difference between the two can be perceived by the user or even obviously perceived.

[0107] The same is true for the scenario where the user significantly drags the top border line of the canvas, which will not be described in detail.

[0108] In this way, when the user drags the canvas boundary line to adjust the canvas height, the sliding update of the canvas boundary line is prone to lag, resulting in poor tracking of the sliding of the canvas boundary line, which brings a poor user experience.

[0109] Figure 5c The example shows a problem scenario where the canvas boundary line cannot be accurately slid to the target position. Figure 5c As shown in (1), the tablet displays the eighteenth editing interface 407 of the note application, and the user drags the bottom border line 4101 of the canvas upwards with his finger. As the user slides his finger upwards, the bottom border line 4101 of the canvas slides upwards, and the height of the canvas 41 decreases accordingly. Figure 5c The nineteenth editing interface 408 is shown in (2). As mentioned above, when the user drags the bottom boundary line 4101 of the canvas, the sliding update of the canvas boundary line is stuck. Continuing to refer to the twentieth editing interface 409 of the note application shown in (3) in 5c, the user's finger quickly slides to position 5 and leaves. Among them, position 5 can be the lower boundary position of the handwriting on the canvas, or it can be a position above the lower boundary position of the handwriting on the canvas. In response to the user operation, the bottom boundary line 4101 of the canvas slides with the finger to position 6 and stops. The bottom boundary line 4101 of the canvas no longer continues to slide upward, and the state of the bottom boundary line 4101 of the canvas is restored to the initial state. You can refer to Figure 5c The twenty-first editing interface 410 shown in (4) is as follows. Figure 5c As shown in (3), position 6 is located below position 5, and the bottom boundary line of the canvas does not slide accurately to the user's ideal position.

[0110] The same is true for the scenario where the user quickly drags the top border line of the canvas, which will not be described in detail.

[0111] In this way, when the user quickly drags the bottom border line of the canvas upwards, and releases the finger after sliding to the bottom border position of the handwriting, the bottom border line of the canvas cannot accurately stop at the bottom border position of the handwriting, but will follow the finger sliding and stop at an intermediate position (such as position 6), and cannot continue to slide upward. Similarly, when the user quickly drags the top border line of the canvas downwards, and releases the finger after sliding to the top border position of the handwriting (that is, the position corresponding to the highest point of the handwriting track), the top border line of the canvas cannot accurately stop at the top border position of the handwriting, but will follow the finger sliding and stop at an intermediate position, and cannot continue to slide downward.

[0112] In the design of the note application, the control of the canvas and canvas boundaries is completely the responsibility of the main thread of the note application. This means that the main thread not only takes on the task of updating the user interface (UI) of the entire application, but also needs to handle some logical tasks, which also rely on the execution of the main thread. However, when the user slides the canvas boundary line sharply or quickly, the main thread needs to perform an abnormally large number of tasks, resulting in a longer time to execute canvas updates and canvas boundary line updates, which causes the problem of stuck canvas boundary line sliding updates.

[0113] Reference Figure 6a In (1), the editing interface of the note application (or the view in the editing interface) includes a canvas view (View) and a canvas boundary line view (View). It can also be understood that the editing interface is displayed based on the canvas View and the canvas boundary line View. Among them, the canvas View corresponds to the canvas (i.e., the handwriting area) in the editing interface, and the canvas boundary line View corresponds to the canvas boundary line (top boundary line and / or bottom boundary line) displayed in the editing interface. In the embodiment of the present application, the canvas View is transparent, that is, the canvas in the editing interface is transparent.

[0114] In one implementation, referring to Figure 6b In (1), the canvas boundary line View may include the canvas top boundary line View and the canvas bottom boundary line View. The canvas top boundary line View and the canvas top boundary line View are located in the same layer (Layer), and the canvas View is also located in this layer. Among them, the canvas top boundary line View corresponds to the canvas top boundary line displayed in the editing interface, and the canvas bottom boundary line View corresponds to the canvas bottom boundary line displayed in the editing interface. The widths of the canvas top boundary line View and the canvas bottom boundary line View may be the same. The widths of the canvas top boundary line View and the canvas bottom boundary line View may be the same as or different from the width of the canvas View, and this embodiment does not limit this.

[0115] In one implementation, referring to Figure 6b In (2), the size of the canvas boundary line View is larger than the canvas View, and the layer where the canvas boundary line View is located (such as Layer 2) is below the layer where the canvas View is located (such as Layer 1). Among them, the top boundary line of the canvas and the bottom boundary line of the canvas displayed in the editing interface correspond to one canvas boundary line View. Continue to refer to Figure 6b In (2), the height of the canvas View is H1, the height of the canvas boundary line View is H2, and H2 is greater than H1. The width of the canvas boundary line View may be the same as or different from the width of the canvas View, which is not limited in this embodiment.

[0116] Reference Figure 6a The process shown in (2) and Figure 5bIn the scenario shown, for example, in response to the user dragging the bottom border line of the canvas from position x1 to position x2, the note application adjusts the size of the canvas View according to position x2, and adjusts the position of the corresponding canvas border line View according to the offset of the adjusted canvas View. When the user drags the bottom border line of the canvas significantly, for example, the user drags the bottom border line of the canvas from position x2 to position x3, from position x3 to position x4, etc., the main thread needs to perform an abnormally large number of tasks, resulting in a longer time to execute the canvas View update, which in turn results in a longer time to update the corresponding canvas border line View according to the update result of the canvas View, thereby causing the problem of stuck sliding updates of the bottom border line of the canvas.

[0117] Reference Figure 6a The process shown in (3) and Figure 5c The scenario shown is an example of an operation in which the user quickly drags the bottom border line of the canvas to position xn (such as the bottom border position of the handwriting or the position above the bottom border position of the handwriting) and then releases the hand. Since the user drags too fast, the finger sliding data sampling may be incomplete, causing the canvas View update and the canvas boundary line View update to stop prematurely. In other words, the note application cannot update the canvas View according to the position xn, and then cannot update the corresponding canvas boundary line View based on the update result, so that the user perceives that the bottom border line of the canvas cannot be accurately dragged to the bottom border position of the handwriting.

[0118] In order to solve the above problems, an embodiment of the present application provides a display method. In this method, the canvas boundary line and the canvas are decoupled, and the update of the canvas boundary line is controlled and processed by the JS (JavaScript) end thread of the note application, and the update of the canvas is controlled and processed by the Android (Android) end thread of the note application. Since the canvas is transparent and invisible in the display interface, the user can directly see the canvas boundary line in the display interface. After decoupling the two, the update of the canvas boundary line is no longer processed by the heavy-duty Android end thread, so that when the user quickly drags the canvas boundary line, the canvas boundary line can also smoothly follow the finger slide, and the canvas boundary line can slide and stay at the boundary position of the handwriting.

[0119] It should be pointed out that the Android thread of the note application can be understood as the main thread of the note application, which is responsible for the user interface update task and some logic tasks of the note application. The JS end of the note application can be understood as a sub-thread of the main thread of the note application, which is implemented based on JavaScript programming.

[0120] It should be pointed out that the canvas boundary line is decoupled from the canvas, which can also be understood as the update of the canvas boundary line View is decoupled from the update of the canvas View. The update of the canvas boundary line View is implemented on the JS side of the note application, and the update of the canvas View is implemented on the Android side of the note application. The update of the canvas boundary line View does not need to wait for the canvas View to be updated before execution.

[0121] Figure 7a Schematic diagram of the structure of an electronic device is shown. For example, the electronic device may be the tablet computer mentioned above. It should be understood that Figure 7a The electronic device shown is only one example of an electronic device, and the electronic device may have more or fewer components than shown in the figures, may combine two or more components, or may have a different configuration of components. Figure 7a The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0122] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0123] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0124] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0125] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0126] In some embodiments, the processor 110 may include one or more interfaces. It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0127] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0128] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0129] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied in electronic devices. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc., applied in electronic devices.

[0130] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0131] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0132] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0133] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0134] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0135] The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., which are not listed here one by one and the present application does not limit this.

[0136] The pressure sensor is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor can be set on the display screen 194. When a force acts on the pressure sensor, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device detects the intensity of the touch operation based on the pressure sensor. The electronic device can also calculate the touch position based on the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0137] The touch sensor is also called a "touch panel". The touch sensor can be arranged on the display screen 194, and the touch sensor and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor can also be arranged on the surface of the electronic device, which is different from the position of the display screen 194.

[0138] In the embodiment of the present application, the display screen 194 can receive handwriting input and display handwriting input, etc. The touch sensor can be used to receive click operations and collect the trajectory of handwriting input, etc. The pressure sensor can detect the pressure acting on the display screen 194, so as to determine whether it is handwriting input by using the size of the pressure, for example, when the pressure is large, it is handwriting input. Exemplarily, assuming that the pressure detected is a first pressure value, it is considered to be wiping the screen, and the pressure detected is a second pressure value, it is considered to be handwriting input, wherein the first pressure value is less than the second pressure value.

[0139] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device.

[0140] Figure 7b It is a software structure block diagram of the electronic device according to an embodiment of the present application.

[0141] The layered architecture of electronic devices divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android Runtime) and the system library, and the kernel layer.

[0142] The application layer can include a series of application packages.

[0143] like Figure 7b As shown, the application package may include applications such as notes, camera, gallery, etc. The application package may also include applications such as calls, calendars, maps, navigation, music, videos, short messages, WLAN (wireless local area networks), Bluetooth, etc.

[0144] In this embodiment, the note application includes an Android side and a JS side. The Android side may include a web view (Web View), a web fragment editor (Edit Web Fragment), and a handwriting view manager (Hand Write ViewManager). In the note application, the JS side thread is used to continuously generate canvas height update tasks during the user's finger sliding process, and put the generated canvas height update tasks into the task queue. The Android side thread (such as the main thread) acts as a consumer of the task queue, obtains the canvas height update task from the task queue and executes it. The JS side thread is used to pass the latest canvas height update task directly to the Android side thread when the user's finger is lifted, and the Android side thread executes the latest canvas height update task to update the canvas View height.

[0145] On the Android side of the note-taking app, it can include Web View, Edit Web Fragment, and Hand WriteView Manager.

[0146] Web View is a user interface element used to display web content in an application. It has a built-in browser engine that can load and display web pages, web applications, or other online content. Users can browse the web directly within the application without leaving the application. Web View is mainly used to display web content. It supports functions such as loading, rendering, and interacting with web pages, and can achieve a complete interactive experience with web content. In this embodiment, Web View can be used to display note content, especially when the note content contains complex layouts, pictures, or requires online content support, Web View can achieve complex interface effects.

[0147] Edit Web Fragment is used to implement editing functions in Web View, especially editing handwritten notes. Edit Web Fragment encapsulates the components that manage Web View. It provides specific interfaces or methods and also contains some logic for processing handwriting input, such as recognizing handwritten text and converting it to text format.

[0148] Hand Write View Manager is a component responsible for managing handwriting input. Hand Write ViewManager can be responsible for handling all operations related to handwriting, such as recognition, saving, editing, etc. Hand Write ViewManager can interact with Edit Web Fragment and other related components. When the user performs handwriting input, Hand Write View Manager captures these inputs and converts these handwriting inputs into editable text format through recognition algorithms or other services. Then, it can pass these texts to Edit Web Fragment for display or further editing. In addition, Hand Write View Manager may also provide some other functions, such as eraser, undo / redo, etc.

[0149] On the Android side of the note-taking application, you can also include ihand write. ihand write is an interface that defines the methods and properties related to handwriting input processing. The Hand Write View Manager and other components that need to use the handwriting function will interact with the specific handwriting processing implementation through the ihand write interface. This means that if a class implements the ihand write interface, it can be used as a handwriting processing component.

[0150] In general, Web View is responsible for displaying the note content, Edit Web Fragment provides editing functions, HandWrite View Manager manages handwriting input and processing, and the ihand write interface defines the methods and properties related to handwriting processing. These components and interfaces work together to implement functions such as creating, editing, and displaying handwritten notes.

[0151] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0152] like Figure 7b As shown, the application framework layer may include a window manager service, an input manager service, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0153] Among them, the input manager service is responsible for managing input devices, such as touch screens, keyboards, mice, etc. It enables input events (such as touch events, key events, etc.) to be correctly received and processed by various applications in the system, and is also responsible for handling configurations and policies related to input devices.

[0154] The window manager service is responsible for managing all windows and views in the system. It enables each application's window to be displayed correctly on the screen and handles various window-related events, such as window creation, destruction, movement, and resizing. The window manager service also works closely with other system services (such as the rendering engine) to enable the content on the screen to be updated and displayed efficiently and smoothly.

[0155] The window manager is used to manage window programs. The content provider is used to store and retrieve data and make the data accessible to applications. The view system includes visual controls, such as controls for displaying text and controls for displaying pictures. The view system can be used to build applications. The display interface can be composed of one or more views. For example, the display interface including the SMS notification icon can include a view for displaying text and a view for displaying pictures.

[0156] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0157] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0158] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0159] System libraries can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), input distributor (input flinger), graphics synthesizer (surface flinger), open graphics library for embedded systems (OpenGL for embedded systems, OpenGL).

[0160] Among them, input flinger is responsible for passing input events to the corresponding window or application. When inputmanager service determines that the input event should be passed to an application or window, it transmits the event to inputflinger. Input flinger will pass the input event to the correct window or application based on the current window layout and focus information. When there are multiple handwriting input events (handwriting input for short), input flinger can synthesize the events and pass them to the corresponding window or application.

[0161] In handwritten notes, the input flinger ensures that the stylus input events distributed by the input manager service can be accurately delivered to the handwritten note application. In this way, the note application can process and display the user's writing actions accordingly.

[0162] The surface flinger is responsible for compositing the contents of multiple applications or windows into a final image and displaying it on the screen. The surface flinger receives drawing requests from various applications or windows and, based on the layout, size, position and other information of the windows, combines the contents into a final image. It then displays the image on the screen.

[0163] In handwritten notes, when the user writes, the handwritten note application will draw its content (including handwriting, background, etc.) onto a surface and transmit it to the surface flinger for synthesis. The surface flinger will synthesize the content of the handwritten note application with the content of other applications or windows into a final image and display it on the screen, so that users can see the content of their handwritten notes.

[0164] Among them, OpenGL is an efficient graphics rendering library that supports complex graphics, image rendering, and various special effects and animations.

[0165] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, audio driver, Wi-Fi driver, sensor driver, etc. Among them, the hardware includes at least processor, display, Wi-Fi module, sensor, etc.

[0166] Understandably, Figure 7b The layers in the software structure shown and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, or combine certain components, or split certain components, or arrange components differently, which is not limited in the present application.

[0167] It is understandable that, in order to implement the display method in the embodiment of the present application, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0168] In the embodiment of the present application, the canvas boundary line View and the canvas View are decoupled, and the update of the canvas boundary line View is controlled and processed by the JS-side thread of the note application, and the update of the canvas View is controlled and processed by the Android-side main thread of the note application. Among them, the update of the canvas View by the Android-side main thread is based on the update result (or update situation) of the canvas boundary line View by the JS-side thread. Therefore, when the JS-side thread updates the canvas boundary line View, it generates a canvas height update task according to its update situation, and transmits the canvas height update task to the Android-side main thread, and executes the canvas height update task through the Android-side main thread.

[0169] In an embodiment of the present application, the parameter of the canvas height update task may be the offset of the canvas boundary line View dragged by the user, that is, the offset of the canvas boundary line View in the height direction. The Android-side main thread may adjust the height of the canvas View according to the offset of the canvas boundary line View dragged by the user. Taking the user dragging the bottom boundary line of the canvas as an example, illustratively, if the user drags the bottom boundary line of the canvas downward, and the offset of the canvas boundary line View is a positive value, the Android-side main thread adjusts the height of the canvas View according to the offset of the canvas boundary line View, that is, increases the offset value based on the original height of the canvas View.

[0170] It is understandable that while the user is dragging the canvas boundary line, the JS-side thread of the note application continuously generates canvas height update tasks. If each canvas height update task is directly passed to the Android-side main thread for processing, the Android-side main thread task will be busy. Moreover, the production speed of the canvas height update task is greater than the consumption speed, which may cause task blocking and increase the probability of task loss. To this end, an embodiment of the present application adds a task queue to the Android side of the note application. Optionally, the task queue is a first-in-first-out queue. After the JS-side thread of the note application generates the canvas height update task, it puts the canvas height update task into the task queue. As a consumer of the task queue, the Android-side main thread reads the canvas height update task in the task queue and executes the canvas height update task.

[0171] In this way, the update of the canvas boundary line View no longer depends on the update of the canvas View, but is directly updated according to the sampling results of the finger sliding. In other words, in the note application editing interface, the position change of the canvas boundary line no longer needs to wait for the canvas height to be adjusted, thereby optimizing the tracking of the position change of the canvas boundary line and improving the user experience.

[0172] In one embodiment, referring to Figure 8a , when the user drags the canvas boundary line, the JS thread of the note application continuously updates the canvas boundary line View according to the sampling results of the finger drag, and then continuously generates canvas height update tasks according to the position change of the canvas boundary line View, and writes the generated canvas height update tasks into the task queue in sequence. As a consumer of the task queue, the Android main thread of the note application sequentially takes out the canvas height update tasks from the task queue and executes the canvas height update tasks. In this way, the canvas height can be updated in the order of tasks.

[0173] When the user finishes dragging the canvas boundary and releases the finger (or lifts the finger), the JS thread of the note application updates the canvas boundary View according to the position of the finger lifted, and then generates a canvas height update task_New according to the position change of the canvas boundary View, and writes the canvas height update task_New into the task queue. When the user releases the finger, the Android main thread of the note application reads the latest canvas height update task_New in the task queue and executes the canvas height update task_New.

[0174] In another embodiment, referring to Figure 8bWhen the user drags the canvas boundary line, the JS thread of the note application continuously updates the canvas boundary line View according to the sampling results of the finger dragging, and then continuously generates canvas height update tasks according to the position change of the canvas boundary line View, and writes the generated canvas height update tasks into the task queue in sequence. As a consumer of the task queue, the Android main thread of the note application sequentially takes out the canvas height update tasks from the task queue and executes the canvas height update tasks.

[0175] Different from Figure 8a In the manner shown, when the user finishes dragging the canvas boundary line and releases the hand (or lifts the hand), the JS thread of the note application updates the canvas boundary line View according to the position of the finger lifted, and then generates a canvas height update task_New according to the position change of the canvas boundary line View, and directly submits the canvas height update task_New to the Android main thread, so that the Android main thread directly executes the canvas height update task_New. It can be understood that when the JS thread directly submits the canvas height update task_New to the Android main thread, there may still be canvas height update tasks cached in the task queue, but the Android main thread no longer reads the canvas height update task from the task queue.

[0176] In this way, when the user raises his hand, the JS thread updates the canvas boundary line View according to the position of the finger raised, and the Android main thread can also update the canvas View height according to the latest generated canvas height update task, so that the position change of the canvas boundary line and the canvas height are more in line with user expectations.

[0177] Optionally, when the user presses the border of the canvas, the Android main thread creates a new task queue to cache the canvas height update task generated by the JS thread. Optionally, when the JS thread directly submits the canvas height update task_New to the Android main thread, the task queue can be destroyed because the Android main thread no longer needs to read tasks from the task queue. Optionally, when the user lets go, the Android main thread of the note application reads the latest canvas height update task_New in the task queue and destroys the task queue.

[0178] When the user drags the canvas boundary, especially when the finger slides sharply or quickly, the speed at which the JS thread generates canvas height update tasks is higher than the speed at which the Android main thread consumes canvas height update tasks. In this process, in order to increase the consumption speed of canvas height update tasks, the Android main thread can obtain tasks from the task queue at a certain interval for consumption based on the current accumulated number of tasks in the task queue.

[0179] Among them, the Android main thread obtains tasks from the task queue at a certain interval for consumption. It can be understood that the Android main thread reads n (n is an integer) tasks in the task queue in order each time, but only executes the last task of these n tasks, and the first n-1 tasks are not processed (such as discarded). Among them, the value of n can be dynamically set by the Android main thread, and the value of n is related to the current accumulated number of tasks in the task queue. When the current accumulated number of tasks in the task queue is larger, the Android main thread can set n to a larger value. Taking n as 3 as an example, the Android main thread reads 3 tasks in the task queue in order each time, discards the first two tasks, and only executes the third task.

[0180] In this way, when canvas update tasks need to be executed frequently and continuously, and the canvas update tasks take a long time, executing these canvas update tasks in sequence will affect the update operations of other UIs. The Android main thread determines whether the old canvas update tasks are waiting to be executed. If so, it can remove them and give priority to the new canvas update tasks, thereby improving the utilization efficiency of the main thread and enhancing the system's response speed to user operations.

[0181] It should be pointed out that the main thread of the Android end obtains tasks from the task queue for consumption at a certain interval, which can be applied to Figure 8a or Figure 8b The embodiment shown is not described in detail here.

[0182] The following uses the example of a user dragging the bottom boundary line of the canvas to explain the method provided in the embodiment of the present application.

[0183] Fig. 9 The schematic diagram of module interaction is shown as an example. Fig. 9 As shown, the process of the display method provided in the embodiment of the present application may include the following steps:

[0184] S501, in response to a user operation, the web page view displays a canvas, and a top boundary line and a bottom boundary line of the canvas.

[0185] For example, the user operation may be an operation of creating a new canvas. Figure 3a The scene shown.

[0186] For example, the user operation may be clicking on a handwritten image. Figure 4a The scene shown.

[0187] For example, the user operation may be an operation of handwriting input on a handwriting image. Figure 4b The scene shown.

[0188] In this embodiment, the canvas is a handwriting area on the boundary interface of the note application, and the canvas is transparent. From the user's perspective, the top boundary line and the bottom boundary line of the canvas are used to define the area range of the canvas.

[0189] S502: In response to the user completing the handwriting input operation, the web page view generates an instruction to calculate a canvas boundary line offset limit value, and sends the instruction to the web page fragment editor.

[0190] The operation of the user completing the writing input can be understood as the operation of the finger or the stylus leaving the canvas when the user uses the finger or the stylus to perform handwriting input.

[0191] The canvas border line offset limit values ​​may include: the upper limit position (topline_upper limit) to which the top border line of the canvas is allowed to offset, the lower limit position (such as bottom line_lowerlimit) to which the bottom border line of the canvas is allowed to offset, and the maximum height (Height_max) of the canvas. Among them, top line_upper limit and Height_max can limit the range of the allowed top border line offset of the canvas, and bottom line_lower limit and Height_max can limit the range of the allowed bottom border line offset of the canvas.

[0192] It can be understood that when the top boundary line of the canvas is offset upward, it means that the height of the canvas is increased; when the top boundary line of the canvas is offset downward, it means that the height of the canvas is reduced; when the bottom boundary line of the canvas is offset downward, it means that the height of the canvas is increased; when the bottom boundary line of the canvas is offset upward, it means that the height of the canvas is reduced.

[0193] It should be noted that in the embodiment of the present application, for example, when the border line of the canvas is shifted upward, the position coordinate decreases, and when the border line of the canvas is shifted downward, the position coordinate increases. It can be understood that the opposite setting is also similar, and will not be repeated here.

[0194] The maximum canvas height Height_max may be preset, and this embodiment does not limit the maximum canvas height. It is understandable that when the top border line of the canvas is dragged upward to increase the canvas height to Height_max, the top border line of the canvas is offset upward to the lower limit position (top line_lowerlimit) to which the top border line of the canvas is allowed to be offset; when the bottom border line of the canvas is dragged downward to increase the canvas height to Height_max, the bottom border line of the canvas is offset downward to the upper limit position (bottom line_upper limit) to which the bottom border line of the canvas is allowed to be offset.

[0195] In this embodiment, the upper limit position to which the top border line of the canvas is allowed to deviate is determined according to the position corresponding to the highest point of the handwriting track in the canvas. Fig.10 In (1), when the user drags the top border line of the canvas downwards so that the top border line of the canvas reaches or exceeds the position corresponding to the highest point of the handwriting track, the top border line of the canvas is offset to the top line_upperlimit position. Similarly, the lower limit position to which the bottom border line of the canvas is allowed to be offset is determined based on the position corresponding to the lowest point of the handwriting track in the canvas. Fig.10 In (1), when the user drags the bottom border line of the canvas upward so that the bottom border line of the canvas reaches or exceeds the position corresponding to the lowest point of the handwriting track, the bottom border line of the canvas is offset to the bottom line_lower limit position.

[0196] Considering that the pen types selected by users for handwriting input are different, such as ballpoint pens, marker pens, fluorescent pens, etc., the thickness of the handwriting track displayed on the canvas is also different. In order to avoid the problem of blocking the highest point or the lowest point of the handwriting due to dragging the canvas boundary line, this embodiment corrects the positions corresponding to the highest point and the lowest point of the handwriting track. Fig.10 The position ' corresponding to the highest point of the handwriting trajectory and the position ' corresponding to the lowest point of the handwriting trajectory shown in (2). Among them, the position ' corresponding to the highest point of the handwriting trajectory is located above the position corresponding to the highest point of the handwriting trajectory, and the interval between the two can be less than the preset value, for example, the interval between the two is a first preset number (such as 8) pixels; the position ' corresponding to the lowest point of the handwriting trajectory is located below the position corresponding to the lowest point of the handwriting trajectory, and the interval between the two can be a second preset number (such as 8) pixels. The values ​​of the first preset number and the second preset number can be equal or different, and this embodiment does not limit this. Therefore, referring to Fig.10 In (2), when the user drags the top border line of the canvas downward so that the top border line of the canvas reaches or exceeds the position corresponding to the highest point of the handwriting track, the top border line of the canvas is offset to the top line_upperlimit position. Fig.10 In (2), when the user drags the bottom border line of the canvas upward so that the bottom border line of the canvas reaches or exceeds the position corresponding to the lowest point of the handwriting track, the bottom border line of the canvas is offset to the bottom line _lowerlimit position.

[0197] S503: The handwriting view manager obtains a canvas boundary line offset limit value, and returns the canvas boundary line offset limit value to the web page fragment editor.

[0198] The handwriting view manager uses the position corresponding to the highest point of the handwriting track in the canvas or the position 'L_top_1 as the top line_upper limit, and uses the position corresponding to the lowest point of the handwriting track in the canvas or the position 'L_bottom_1 as the bottom line _lower limit.

[0199] Exemplarily, the handwriting view manager may calculate the top line_upper limit, the bottom line _lower limit, and the maximum canvas height Height_max through the getHandWriteMaxHeight function.

[0200] S504, the web page fragment editor sends the current height of the canvas View and the canvas boundary line offset limit value to the JS end thread.

[0201] Exemplarily, the web page fragment editor can pass the current height of the canvas View and the canvas boundary line offset limit value to the JS side thread by calling restrictHandWriteRange on the JS side.

[0202] S505, the JS end thread saves the current height of the canvas View and the canvas boundary line offset limit value.

[0203] In this way, when the user starts handwriting input or drawing, the current height of the canvas View and the canvas boundary line offset limit value saved by the JS side will be updated. Among them, after the user completes the handwriting input, if the highest point position and / or the lowest point position of the handwriting changes, the top line_upper limit and bottom line _lower limit will also change, and the JS side thread needs to save the latest canvas boundary line offset limit value.

[0204] In this embodiment, when the user pulls up the handwriting input or draws, the JS side will generate a placeholder image according to the current height of the canvas View. A placeholder image generally refers to a temporary image or placeholder used to replace the actual content. These images usually have simple geometric shapes or fuzzy patterns, and are used to fill blank areas where the content has not yet been determined, so that the layout and typesetting can be previewed. In this embodiment, the placeholder image can be used to replace the position or element whose content cannot be determined temporarily, and specifically can be used to replace the handwriting area or handwritten picture. For example, when the user leaves a blank area in a handwritten note, but has not yet determined the handwritten picture to be filled, a placeholder image can be used to represent this position, which helps the user maintain the integrity and logic of the interface layout when handwriting input.

[0205] S506: In response to the user pressing the bottom border line of the canvas, the JS-side thread determines that the dragged object is the bottom border line of the canvas, and passes the dragged object to the webpage fragment editor.

[0206] The drag events for the canvas border line can be divided into down events, move events and up events. Among them, down events, move events and up events are usually used to describe the process of pressing, moving and releasing an object on an interface element. The down event can refer to the event acting on the bottom border line of the canvas, the move event can refer to the moving event that adjusts the position of the bottom border line of the canvas, and the up event can be the event that completes the adjustment of the position. The move event can also be understood as a moving event that adjusts the height of the canvas, and the up event can be the event that completes the adjustment of the height of the canvas.

[0207] For example, when the user presses (or long presses) the bottom border line of the canvas, the JS thread listens to the down event for the bottom border line of the canvas, and the handleBottomLineDown method of the JS is triggered, and it is determined that the dragged object is the bottom border line of the canvas. Then, the JS thread calls the startDragging method of the Android side to pass the information that the dragged object is the bottom border line of the canvas to the webpage fragment editor.

[0208] S507: The web page fragment editor generates an instruction for calculating a limit value of the bottom boundary line offset of the canvas.

[0209] The canvas bottom boundary line offset limit value may include a lower limit position to which the canvas bottom boundary line is allowed to offset, bottom line _lower limit, and a maximum canvas height Height_max.

[0210] S508: The handwriting view manager calculates a canvas bottom boundary line offset limit value, and returns the canvas bottom boundary line offset limit value to the web page fragment editor.

[0211] The calculation method of bottom line _lower limit can be found in the previous article and will not be repeated here.

[0212] S509: The web page fragment editor sends the canvas bottom boundary line offset limit value to the JS end thread.

[0213] Optionally, the maximum canvas height Height_max can be pre-stored on the JS side, and the handwriting view manager only needs to calculate the lower limit position bottom line _lower limit to which the bottom boundary line of the canvas is allowed to offset, and return the bottom line _lower limit to the web page fragment editor, so that the bottom line _lower limit can be passed to the JS side thread through the web page fragment editor.

[0214] Exemplarily, the web page fragment editor can pass the current height of the canvas View and the canvas boundary line offset limit value to the JS side thread by calling restrictHandWriteRange on the JS side.

[0215] Optionally, after the web page fragment editor passes the current height of the canvas View and the canvas boundary line offset limit value to the JS side thread by calling restrictHandWriteRange on the JS side, the state of the target flag is set to state 1, indicating that the current height of the canvas View and the canvas boundary line offset limit value have been passed to the JS side.

[0216] S510, the JS end thread saves the canvas bottom boundary line offset limit value.

[0217] S511, in response to the user dragging the bottom border line of the canvas, the JS end thread calculates and calibrates the offset of the bottom border line of the canvas according to the offset limit value of the bottom border line of the canvas and each sampled finger position.

[0218] For example, when the user drags the bottom border line of the canvas, the JS thread listens to the move event for the bottom border line of the canvas. The handleBottomLineMove method of the JS is triggered, and then the JS thread continuously generates canvas height update tasks based on the offset of the bottom border line of the canvas, and can pass the canvas height update tasks to the Android side through the updateHandWriteImageMarginBottom method to add these canvas height update tasks to the task queue of the Android side.

[0219] When the user drags the bottom border of the canvas, the position of the user's finger sliding is sampled. For each finger position obtained by sampling, the difference between the finger position and the current position of the bottom border of the canvas is calculated as the offset of the bottom border of the canvas this time.

[0220] In this embodiment, for example, when the offset is a positive value, it indicates that the border line is offset downward, and when the offset is a negative value, it indicates that the border line is offset upward. It can be understood that when the offset value of the top border line of the canvas is negative, it indicates that the height of the canvas is increased; when the offset value of the top border line of the canvas is positive, it indicates that the height of the canvas is decreased; when the offset value of the bottom border line of the canvas is positive, it indicates that the height of the canvas is increased; and when the offset value of the bottom border line of the canvas is negative, it indicates that the height of the canvas is decreased.

[0221] This embodiment also performs a calibration process on the offset of the bottom border line of the canvas calculated based on the finger sampling position to avoid the problem of the bottom border line of the canvas offset exceeding the limit, and also to avoid the problem that the user cannot accurately drag the canvas border line to the lowest or highest point of the handwriting.

[0222] Reference Fig.11 The process of calculating and calibrating the offset of the bottom border line of the canvas in response to the user dragging the bottom border line of the canvas by the JS thread may include:

[0223] S5111, the JS thread obtains the saved current height of the canvas View, the lower limit position to which the bottom boundary line of the canvas is allowed to offset, and the maximum height of the canvas.

[0224] S5112, the JS thread calculates the currently allowed minimum height of the canvas according to the current height of the canvas View and the lower limit position to which the bottom boundary line of the canvas is allowed to offset.

[0225] Among them, the currently allowed minimum canvas height = canvas View current height - (canvas bottom boundary line position - the lower limit position to which the canvas bottom boundary line is allowed to offset).

[0226] S5113, the JS thread obtains the current finger position and calculates the offset of the canvas boundary line View according to the current finger position.

[0227] The current finger position is the finger position currently sampled.

[0228] In this process, the offset of the canvas boundary line View refers to the offset corresponding to the canvas boundary line View used to offset the bottom boundary line of the canvas in the height direction. Among them, the offset of the canvas boundary line View = the current finger position - the current position of the bottom boundary line of the canvas.

[0229] S5114, the JS-side thread determines whether the bottom boundary line of the canvas is currently being dragged upward or downward. If the bottom boundary line of the canvas is currently being dragged upward, S5115 is executed; if the bottom boundary line of the canvas is currently being dragged downward, S5117 is executed.

[0230] When the offset of the canvas boundary line View is a positive value, it indicates that the bottom boundary line of the canvas is currently being dragged downward to increase the height of the canvas; when the offset of the canvas boundary line View is a negative value, it indicates that the bottom boundary line of the canvas is currently being dragged upward to reduce the height of the canvas.

[0231] S5115, the JS thread determines whether the height of the canvas after the offset does not reach the minimum height of the canvas based on the current height of the canvas View and the offset of the boundary line View. If so, execute S5116. If not, the process ends and the currently calculated offset does not need to be calibrated.

[0232] The sum of the current height of the canvas View and the offset of the boundary line View is the height of the offset canvas.

[0233] S5116, the JS end thread recalculates the offset of the boundary line View according to the lower limit position to which the bottom boundary line of the canvas is allowed to be offset.

[0234] If the height of the canvas after the offset does not reach the minimum height of the canvas, it means that the offset bottom boundary line of the canvas will cover or block the handwriting on the canvas, especially the lowest point area of ​​the handwritten notes. At this time, the JS thread can recalculate the offset of the boundary line View according to the lower limit position to which the bottom boundary line of the canvas is allowed to offset, that is, replace the current finger position with the lower limit position to which the bottom boundary line of the canvas is allowed to offset, and recalculate the offset of the canvas boundary line View. At this time, the offset of the canvas boundary line View = the lower limit position to which the bottom boundary line of the canvas is allowed to offset - the current position of the bottom boundary line of the canvas.

[0235] S5117, the JS thread determines whether the height of the canvas after the offset exceeds the maximum height of the canvas based on the current height of the canvas View and the offset of the boundary line View. If so, execute S5118. If not, the process ends and the currently calculated offset does not need to be calibrated.

[0236] S5118, the JS thread calibrates the offset of the boundary line View according to the maximum canvas height.

[0237] If the canvas height exceeds the maximum canvas height after offset, the offset of the boundary line View needs to be calibrated.

[0238] Exemplarily, the JS thread calibrates the offset of the boundary line View according to the maximum canvas height, so that after the offset is performed according to the calibrated offset, the canvas height is the maximum canvas height. Wherein, the calibrated offset = the offset before calibration - (the current height of the canvas View + the offset before calibration - the maximum canvas height).

[0239] S512, the JS end thread updates the canvas boundary line View according to the canvas boundary line View offset, and updates and saves the current canvas View height.

[0240] After the JS-side thread calculates and calibrates the canvas boundary line View offset each time, the JS-side thread updates the canvas boundary line View according to the offset, and updates and saves the current canvas View height according to the offset.

[0241] It can be understood that after the JS-side thread updates the canvas boundary line View according to the canvas boundary line View offset, the position of the editing line at the bottom of the canvas displayed on the editing interface is adjusted accordingly.

[0242] It is understandable that the form of the canvas boundary line View is different, and the way to update it according to the offset is also different, and this embodiment does not limit this.

[0243] For example, refer to Fig.12a As shown, the canvas boundary line View may include the canvas top boundary line View and the canvas bottom boundary line View. The canvas top boundary line View corresponds to the canvas top boundary line displayed in the editing interface, and the canvas bottom boundary line View corresponds to the canvas bottom boundary line displayed in the editing interface. When the user drags the bottom boundary line of the canvas downward, the JS-side thread calculates and calibrates the offset of the canvas bottom boundary line View, and updates the canvas bottom boundary line View according to the offset. At this time, the JS-side thread updates the canvas bottom boundary line View according to the offset, and specifically, the JS-side thread moves the canvas bottom boundary line View downward according to the offset.

[0244] It should be pointed out that after the JS-side thread updates the bottom boundary line View of the canvas according to the offset, the display position of the bottom boundary line View of the canvas in the View layout may be the same as the display position of the bottom boundary line of the canvas in the editing interface, or may be mapped according to certain rules, which is not limited in this embodiment. For example, based on the offset determined by the JS-side thread, the bottom boundary line View of the canvas moves downward from position 1 to position 2. After the bottom boundary line View of the canvas moves downward from position 1 to position 2, the display position of the bottom boundary line of the canvas in the editing interface moves downward from position 1' to position 2'. Among them, the expressions of position 2 and position 2' may be the same or different, but there is a certain mapping relationship between the two. The same is true for position 1 and position 1', which will not be repeated.

[0245] For example, refer to Figure 12bAs shown, the size of the canvas boundary line View is larger than the canvas View, and the layer where the canvas boundary line View is located (such as Layer 2) is below the layer where the canvas View is located (such as Layer 1). Among them, the top boundary line of the canvas and the bottom boundary line of the canvas displayed in the editing interface correspond to one canvas boundary line View. That is, no matter whether the user drags the top boundary line of the canvas or the bottom boundary line of the canvas, the JS-side thread updates this canvas boundary line View. Still taking the example of the user dragging the bottom boundary line of the canvas downward, the JS-side thread updates this canvas boundary line View according to the offset. Specifically, the JS-side thread can adjust the size and / or position of the canvas boundary line View according to the offset, so that after the update of this canvas boundary line View is completed, the display position of the bottom boundary line of the canvas in the editing interface moves downward. Exemplarily, before the JS-side thread updates the canvas boundary line View according to the offset, the height of the canvas boundary line View is H2. After the JS-side thread updates the canvas boundary line View according to the offset, the height of the canvas boundary line View becomes H3, and H3 is greater than H2.

[0246] It should be pointed out that after the JS-side thread updates the canvas bottom boundary line View according to the offset, for example, the size and / or position of the canvas boundary line View is adjusted, the adjustment method and the downward movement of the canvas bottom boundary line displayed on the editing interface also have a certain mapping relationship, which is not limited in this embodiment.

[0247] S513, the JS-side thread generates a canvas height update task according to the canvas boundary line View offset, and adds the canvas height update task to the task queue.

[0248] Each time the JS thread calculates and calibrates a canvas boundary line View offset, it generates a canvas height update task based on the canvas boundary line View offset and adds the canvas height update task to the task queue.

[0249] Exemplarily, the parameter or variable of the canvas height update task is the canvas boundary line View offset.

[0250] For example, the JS thread can pass the canvas height update task to the Android side through the updateHandWriteImageMarginBottom method, for example, to the web page fragment editor, and the web page fragment editor adds the received canvas height update task to the task queue. The web page fragment editor does not block the transmission of the canvas height update task from the JS side to the Android side.

[0251] S514, the webpage fragment editor updates the height of the canvas View according to the canvas height update tasks that are continuously obtained.

[0252] In this embodiment, the JS end thread is the producer of the canvas height update task in the task queue, and the web page fragment editor is the consumer of the canvas height update task in the task queue.

[0253] The webpage fragment editor continuously obtains canvas height update tasks in the task queue, and updates the height of the canvas View according to the canvas height update tasks obtained in sequence. It can be understood that the webpage fragment editor takes out the canvas height update task from the task queue and updates the height of the canvas View according to the task, which is the consumption of the canvas height update task by the webpage fragment editor.

[0254] When the user drags the bottom border of the canvas, especially when the finger slides sharply or quickly, the speed at which the JS end generates canvas height update tasks is higher than the speed at which the Android end consumes canvas height update tasks. In this process, in order to increase the consumption speed of canvas height update tasks, the Android end can obtain tasks from the task queue at a certain interval for consumption based on the current accumulated number of tasks in the task queue.

[0255] Optionally, when the webpage fragment editor obtains the canvas height update task, it can first determine whether the state of the target flag is state 1. If so, it indicates that the Android side has passed the current height of the canvas View and the canvas boundary line offset limit value to the JS side. If not, it indicates that the Android side may not pass the current height of the canvas View and the canvas boundary line offset limit value to the JS side. In order to ensure the reliability of the canvas height update task generated by the JS side, the webpage fragment editor can again pass the current height of the canvas View and the canvas boundary line offset limit value to the JS side thread by calling restrictHandWriteRange on the JS side, and set the state of the target flag to state 1.

[0256] S515, in response to the user raising his hand, the JS-side thread calculates the offset of the canvas boundary line View according to the canvas bottom boundary line offset limit value and the latest sampled finger position.

[0257] For example, when the user raises his hand on the bottom border of the canvas, the JS thread listens to the move event for the bottom border of the canvas. The handleBottomLineUp method on the JS side is triggered, and the JS thread can obtain the position where the user raised his hand, that is, the latest sampled finger position. Based on this, the JS thread can calculate the offset of the canvas border View again. Similarly, the JS thread can refer to Fig.11The process shown calibrates this offset.

[0258] S516, the JS end thread updates the canvas boundary line View according to the canvas boundary line offset, and updates and saves the current canvas View height.

[0259] Similar to S512, the JS-side thread updates the canvas boundary line View according to the canvas boundary line View offset determined when the user raises his hand, and updates and saves the current canvas View height according to the offset.

[0260] In this way, the movement of the bottom border line of the canvas can be consistent with the user's dragging operation, and the offset of the bottom border line of the canvas meets the user's expectations.

[0261] S517, the JS end thread generates the latest canvas height update task according to the canvas boundary line View offset, and sends the canvas height update task to the web page fragment editor.

[0262] Exemplarily, the JS-side thread may call the updateElementAttributeInfo method of Android to pass the latest canvas height update task to the webpage fragment editor.

[0263] S518, the web page fragment editor updates the height of the canvas View according to the latest canvas height update task received.

[0264] In this way, the webpage fragment editor updates the height of the canvas View again according to the latest canvas height update task received, so that the final canvas height can meet the user's expectations and will not cause similar Figure 5c The problem of not being able to accurately adjust the canvas height can also optimize the tracking performance of the canvas boundary position changes.

[0265] For details not explained in this process, please refer to the previous text and I will not repeat them here.

[0266] The following is a specific example to explain the solution provided in the embodiment of the present application.

[0267] Reference Fig.13As shown in (1), the tablet computer displays the twenty-second editing interface 601 of the note application. As described above, the canvas View corresponds to the canvas displayed in the twenty-second editing interface 601; the canvas boundary line View corresponds to the bottom boundary line and the top boundary line of the canvas displayed in the twenty-second editing interface 601, or the bottom boundary line View included in the canvas boundary line View corresponds to the bottom boundary line of the canvas displayed in the twenty-second editing interface 601, and the top boundary line View included in the canvas boundary line View corresponds to the top boundary line of the canvas displayed in the twenty-second editing interface 601. Continue to refer to Fig.13 As shown in (1), in response to the user dragging the bottom border line of the canvas and sliding the finger position from position 7 to position 8, the JS thread calculates and calibrates the offset corresponding to the canvas border line View, and updates the canvas border line View according to the offset. For an explanation of how the JS thread updates the canvas border line View according to the offset, please refer to the previous text, which will not be repeated here.

[0268] After the JS thread updates the canvas boundary line View according to the offset, the tablet computer displays the twenty-third editing interface 602 of the note application. In the twenty-third editing interface 602, the bottom boundary line of the canvas is displayed at position 8. Fig.13 After the JS thread updates the canvas boundary line View according to the offset, the JS thread generates a canvas height update task according to the offset and adds the canvas height update task to the task queue. Then, the Android main thread can obtain the canvas height update task in the task queue and update the height of the canvas View according to the canvas height update task.

[0269] That is to say, in this embodiment, the update of the canvas boundary line View no longer depends on the update of the canvas View height. The update of the canvas boundary line View is performed independently on the JS side, and the Android side main thread updates the height of the canvas View according to the update result of the canvas boundary line View on the JS side.

[0270] The above examples all take the method of adjusting the height of the canvas by dragging the bottom boundary line of the canvas as an example. The process and logic of adjusting the height of the canvas by dragging the top boundary line of the canvas are similar and will not be repeated here.

[0271] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the display method in the above-mentioned embodiment.

[0272] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the display method in the above-mentioned embodiment.

[0273] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor may execute the computer execution instructions stored in the memory so that the chip executes the display method in the above-mentioned method embodiments.

[0274] Among them, the electronic device (such as a mobile phone, etc.), computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0275] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0276] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0277] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display method, characterized in that: include: Displaying a first interface of a target application, the first interface is displayed based on a first view and a second view, the first view corresponds to a target boundary line of a handwriting canvas, the second view corresponds to the canvas, and the second view is transparent; the target boundary line is a top boundary line or a bottom boundary line; In response to a finger position sliding from a first position to a second position when dragging the target boundary line, updating the first view; After updating the first view, displaying a second interface of the target application; In the second interface, the target boundary line is displayed at the second position; After the first view is updated, the height of the second view is updated according to the update result of the first view.

2. The method according to claim 1, characterized in that Updating the first view includes: The first view is updated through the child thread of the target application.

3. The method according to claim 1 or 2, characterized in that: Updating the height of the second view according to the update result of the first view includes: The height of the second view is updated according to the update result of the first view through the main thread of the target application.

4. The method according to claim 3, characterized in that Also includes: In the process of dragging the target boundary line, a plurality of canvas update tasks are sequentially generated through the subthread of the target application, and the plurality of canvas update tasks are sequentially added to the task queue; A canvas update task is obtained from a task queue through the main thread of the target application, and the height of the second view is updated according to the canvas update task.

5. The method according to claim 4, characterized in that Obtaining a canvas update task from a task queue, and updating the height of the second view according to the canvas update task, including: A plurality of canvas update tasks are obtained from the task queue each time, and the height of the second view is updated according to the last canvas update task among the plurality of canvas update tasks.

6. The method according to claim 4, characterized in that Also includes: In response to the operation of stopping dragging the target boundary line and raising the hand, updating the first view according to the hand raising position through the subthread of the target application, and generating a target canvas update task; Passing the target canvas update task to the main thread of the target application through the sub-thread of the target application; The target canvas update task is executed through the main thread of the target application.

7. The method according to claim 1, characterized in that Updating the first view includes: Calculating an offset of the first view according to a current position of the finger; calibrating an offset of the first view according to an offset limit of the first view; The first view is updated according to the calibrated offset of the first view.

8. The method according to claim 7, characterized in that The target boundary line is the bottom boundary line; The second position is higher than the first position; The method further comprises: calibrating an offset of the first view according to an offset limit of the first view, comprising: Calculate the cumulative sum of the offset of the first view and the current height of the second view; Determine that the accumulated sum is less than a minimum height value of the second view; the minimum height value is a difference between a current height of the second view and a first height value, the first height value is a difference between a position of the target boundary line and a third position; the third position is a lower limit position to which the bottom boundary line of the canvas is allowed to be offset; The offset of the first view is recalculated according to the third position as the offset of the first view after calibration.

9. The method according to claim 8, characterized in that The third position is the lowest point of the handwriting track on the canvas; Alternatively, the third position is the fourth position, the fourth position is located below the lowest point position, and a difference between the fourth position and the lowest point position is smaller than a preset threshold.

10. The method according to claim 7, characterized in that The target boundary line is the bottom boundary line; the second position is lower than the first position; The method further comprises: calibrating an offset of the first view according to an offset limit of the first view, comprising: Calculate the cumulative sum of the offset of the first view and the current height of the second view; Determine that the accumulated sum is less than a maximum height value of the second view; The offset of the first view is recalculated according to the maximum height value of the second view as the offset of the first view after calibration.

11. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device executes the display method as described in any one of claims 1-10.

12. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is enabled to execute the display method according to any one of claims 1 to 10.

Images