Display method, electronic device and storage medium

Abstract

Provided in the present application are a display method, an electronic device and a storage medium. In the method, a display interface of a target application comprises a canvas and a canvas boundary line, the display interface is displayed on the basis of a canvas view and a canvas boundary line view, and the canvas view is transparent. In response to a finger of a user sliding from a first position to a second position when the user drags the canvas boundary line, the canvas boundary line view is updated. After the canvas boundary line view is updated, the canvas boundary line is displayed at a second position on an application interface. After the canvas boundary line view is updated, the canvas view is updated on the basis of an update situation of the canvas boundary line view. In this way, the update of the canvas boundary line view no longer depends on the update of the canvas view, thereby improving the responsiveness when the position of the canvas boundary line changes.

Description

Display methods, electronic devices and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411542918.9, filed on October 31, 2024, entitled "Display Method, Electronic Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of smart terminal technology, and in particular to a display method, electronic device and storage medium. Background Technology

[0003] As note-taking apps become more powerful, their user experience is also becoming richer. For example, users can type text in their notes, paste content copied to the clipboard, add hyperlinks, images, audio, and even input text and graphics via handwriting. Summary of the Invention

[0004] This application provides a display method, an electronic device, and a storage medium. In this method, when a user drags the canvas boundary line in the note-taking application interface to adjust the canvas height, the update of the canvas boundary line does not depend on the update of the canvas height, thereby improving the responsiveness when the position of the canvas boundary line changes.

[0005] Firstly, embodiments of this application provide a display method. The method includes:

[0006] The electronic device displays a first interface of the target application. The first interface is based on a first view and a second view. The first view corresponds to the target boundary line of the handwriting canvas, and the second view corresponds to the canvas. The second view is transparent. The target boundary line is either the top boundary line or the bottom boundary line. The electronic device updates the first view in response to the finger position sliding from the first position to the second position when the target boundary line is dragged. After updating the first view, the electronic device displays the second interface of the target application. In the second interface, the target boundary line is displayed in the second position. After updating the first view, the electronic device updates the height of the second view according to the update result of the first view.

[0007] For example, the target application could be a note-taking application, and the first interface could be the editing interface of the note-taking application, which includes a handwriting area (also called a canvas). The first interface could include a first view and a second view, where the first view displays the canvas boundary lines and the second view displays the canvas itself.

[0008] Updating the first view can be understood as updating at least one of the following: the size, position, etc. of the first view.

[0009] In this way, since the canvas is transparent and invisible in the display interface, what the user sees directly in the display interface is the canvas boundary line. After decoupling the two, the update of the canvas boundary line no longer depends on the update of the canvas height. This allows the canvas boundary line to smoothly follow the finger's movement when the user quickly drags the canvas boundary line, and the canvas boundary line can slide and stop at the edge of the handwritten stroke.

[0010] According to the first aspect, updating the first view by the electronic device includes: updating the first view by the electronic device through a sub-thread of the target application.

[0011] For example, the child thread could be a thread on the target application's JavaScript side. This way, updates to the canvas boundaries are no longer handled by the heavily loaded main thread of the target application, improving the responsiveness to user dragging of the boundaries.

[0012] According to the first aspect, or any implementation of the first aspect above, the electronic device updates the height of the second view based on the update result of the first view, including: updating the height of the second view based on the update result of the first view through the main thread of the target application.

[0013] The main thread can be the main thread of the target application's Android version.

[0014] In this way, the canvas boundary lines and the canvas are decoupled. Updates to the canvas boundary lines are controlled by the JavaScript thread of the note-taking application, while updates to the canvas are controlled by the Android thread of the note-taking application.

[0015] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the process of dragging the target boundary line, generating multiple canvas update tasks sequentially through a sub-thread of the target application, and adding the multiple canvas update tasks to the task queue in order; and retrieving the canvas update tasks from the task queue through the main thread of the target application, and updating the height of the second view according to the canvas update tasks.

[0016] As the user drags the canvas boundaries, a sub-thread of the note-taking application continuously generates canvas height update tasks. If each canvas height update task were directly passed to the main thread for processing, it would overload the main thread. Furthermore, the production rate of canvas height update tasks exceeds the consumption rate, potentially causing task blocking and increasing the probability of task loss. Therefore, a task queue is added to the note-taking application. After generating a canvas height update task, the sub-thread places it in the task queue. The main thread, acting as the consumer of the task queue, reads canvas height update tasks from the queue and executes them.

[0017] According to the first aspect, or any implementation of the first aspect above, the main thread of the target application retrieves a canvas update task from the task queue and updates the height of the second view according to the canvas update task, including:

[0018] The main thread of the target application retrieves multiple canvas update tasks from the task queue each time, and updates the height of the second view based on the last canvas update task among these multiple canvas update tasks.

[0019] In this way, when canvas update tasks need to be executed frequently and continuously, and when these tasks are time-consuming, executing them sequentially will affect other UI update operations. The main thread can improve the efficiency of the main thread and enhance the system's responsiveness to user actions by checking if old canvas update tasks are waiting to be executed.

[0020] According to the first aspect, or any implementation of the first aspect above, the method further includes:

[0021] In response to the action of stopping dragging the target boundary line and raising the hand, the electronic device updates the first view based on the hand-raising position through a sub-thread of the target application, generating a target canvas update task; the electronic device passes the target canvas update task to the main thread of the target application through the sub-thread of the target application; the electronic device executes the target canvas update task through the main thread of the target application.

[0022] In this way, when the user raises their hand, the child thread updates the canvas boundary view based on the position of the raised finger, and the main thread can also update the canvas view height based on the newly generated canvas height update task, so that the position change of the canvas boundary and the canvas height are more in line with the user's expectations.

[0023] According to the first aspect, or any implementation of the first aspect above, the electronic device updates the first view based on the hand-raising position through a sub-thread of the target application, including: the sub-thread of the target application calculates the offset of the first view based on the current position of the finger; the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view; and the sub-thread of the target application updates the first view based on the calibrated offset of the first view.

[0024] This embodiment calibrates the canvas boundary line offset calculated based on the finger sampling position to avoid the canvas boundary line offset exceeding the limit, and also to avoid the problem that the user cannot accurately drag the canvas boundary line to the lowest or highest point of the handwriting.

[0025] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the bottom boundary line; the second position is higher than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including:

[0026] The target application's sub-thread calculates the sum of the offset of the first view and the current height of the second view; the target application's sub-thread determines that the sum is less than the minimum height value of the second view; the minimum height value is the difference between the current height of the second view and the first height value, the first height value is the difference between the position of the target boundary line and the third position; the third position is the lower limit position to which the bottom boundary line of the canvas is allowed to offset; the target application's sub-thread recalculates the offset of the first view based on the first position, as the calibrated offset of the first view.

[0027] In this way, when dragging the bottom edge of the canvas, the bottom edge of the canvas can be prevented from stopping beyond the lowest point of the handwriting.

[0028] According to the first aspect, or any of the above implementations of the first aspect, the third position is the lowest point of the handwritten trajectory on the canvas; or, the third position is the fourth position, the fourth position is below the lowest point, and the difference between the fourth position and the lowest point is less than a preset threshold.

[0029] This can prevent the lowest point of the stroke from being covered due to the stroke being too thick.

[0030] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the bottom boundary line; the second position is lower than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including: the sub-thread of the target application calculates the sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the sum is less than the maximum height value of the second view; the sub-thread of the target application recalculates the offset of the first view according to the maximum height value of the second view, and uses it as the calibrated offset of the first view.

[0031] In this way, in a scenario where the bottom edge of the canvas is dragged down, the final position of the bottom edge of the canvas is limited by the maximum height of the canvas.

[0032] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the top boundary line; the second position is lower than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including:

[0033] The target application's sub-thread calculates the sum of the offset of the first view and the current height of the second view; the target application's sub-thread determines that the sum is less than the minimum height value of the second view; the minimum height value is the difference between the current height of the second view and the second height value, and the second height value is the difference between the fifth position and the position of the target boundary line; the fifth position is the upper limit position to which the top boundary line of the canvas is allowed to offset; the target application's sub-thread recalculates the offset of the first view based on the fifth position, as the calibrated offset of the first view.

[0034] In this way, when dragging the top edge of the canvas downwards, it can prevent the bottom edge of the canvas from stopping above the height of the handwriting.

[0035] According to the first aspect, or any of the above implementations of the first aspect, the fifth position is the highest point of the handwritten trajectory on the canvas; or, the fifth position is the sixth position, the sixth position is above the highest point, and the difference between the sixth position and the highest point is less than a preset threshold.

[0036] This can prevent the lowest point of the stroke from being covered due to the stroke being too thick.

[0037] According to the first aspect, or any implementation of the first aspect above, the target boundary line is the top boundary line; the second position is higher than the first position; correspondingly, the sub-thread of the target application calibrates the offset of the first view according to the offset limit of the first view, including: the sub-thread of the target application calculates the sum of the offset of the first view and the current height of the second view; the sub-thread of the target application determines that the sum is less than the maximum height value of the second view; the sub-thread of the target application recalculates the offset of the first view according to the maximum height value of the second view, and uses it as the calibrated offset of the first view.

[0038] In this way, in a scene where the top edge of the canvas is dragged and slid down, the final position of the top edge of the canvas is limited by the maximum height of the canvas.

[0039] Secondly, embodiments of this application provide an electronic device. The electronic device includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when executed by the one or more processors, the electronic device performs the display method of the first aspect and any one thereof.

[0040] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0041] Thirdly, embodiments of this application provide a computer-readable storage medium. This computer-readable storage medium includes a computer program that, when executed on an electronic device, causes the electronic device to perform the display method of the first aspect and any one thereof.

[0042] The third aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the third aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0043] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when run, causes a computer to perform a display method as described in the first aspect or any one of the first aspects.

[0044] The fourth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the fourth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0045] Fifthly, this application provides a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes a display method as described in the first aspect or any one thereof, to control the receiving pin to receive signals and to control the transmitting pin to transmit signals.

[0046] The fifth aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the fifth aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here. Attached Figure Description

[0047] Figure 1 is an exemplary illustration of a note-taking application usage scenario;

[0048] Figure 2 is an exemplary illustration of a note-taking application usage scenario;

[0049] Figure 3a is an exemplary illustration of a note-taking application usage scenario;

[0050] Figure 3b is an exemplary illustration of a note-taking application usage scenario;

[0051] Figure 3c is an exemplary illustration of a note-taking application usage scenario;

[0052] Figure 4a is an exemplary illustration of a note-taking application usage scenario;

[0053] Figure 4b is an exemplary illustration of a note-taking application usage scenario;

[0054] Figure 5a is an exemplary illustration of a note-taking application usage scenario;

[0055] Figure 5b is an exemplary illustration of a note-taking application usage scenario;

[0056] Figure 5c is an exemplary illustration of a note-taking application usage scenario;

[0057] Figure 6a is a flowchart illustrating a problem scenario in a note-taking application;

[0058] Figure 6b is an exemplary layout diagram of the canvas view and the canvas boundary line view.

[0059] Figure 7a is a schematic diagram of the hardware structure of an exemplary electronic device;

[0060] Figure 7b is a schematic diagram of the software structure of an exemplary electronic device;

[0061] Figure 8a is an exemplary schematic diagram of module interaction;

[0062] Figure 8b is an exemplary schematic diagram of module interaction;

[0063] Figure 9 is an exemplary schematic diagram of module interaction;

[0064] Figure 10 is an exemplary schematic diagram illustrating the dragging restrictions of the canvas boundary lines;

[0065] Figure 11 is an exemplary flowchart illustrating the process of calculating and calibrating the canvas boundary line offset on the JS side.

[0066] Figure 12a is an exemplary schematic diagram of the update of the canvas boundary line view;

[0067] Figure 12b is an exemplary schematic diagram of the update of the canvas boundary line view;

[0068] Figure 13 is an exemplary illustration of a note-taking application usage scenario. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0071] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0072] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0074] The display method provided in this application can be applied to electronic devices that support handwriting input. For example, the electronic device can be a terminal device. The aforementioned terminal devices may include mobile phones, tablets, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, netbooks, smart home devices (such as smart TVs, smart screens, large screens, smart speakers, smart air conditioners, etc.), personal digital assistants (PDAs), wearable devices (such as smartwatches, smart bracelets, etc.), in-vehicle devices, virtual reality devices, etc., and this application does not impose any limitations on these.

[0075] In one example, a note-taking application is installed on the terminal device. For instance, taking a tablet computer as an example, Figure 1 shows a schematic diagram of a user interface for a note-taking application on a tablet. It is understood that the note-taking application may have different names on different terminal devices, such as notebook, notepad, etc.

[0076] Referring to Figure 1(1), the tablet computer displays a first user interface 101 of the note-taking application, which includes a first "Add" control 1012. In response to the user's click on the first "Add" control 1012, the tablet computer displays a second user interface 102 of the note-taking application, as shown in Figure 1(2). Among them, the "Text Note" option 1021, the "Handwritten Note" option 1022, etc. are displayed on the second user interface 102. The user can click on the "Text Note" option 1021 to create a new text note. The user can input text, symbols, pictures, audio, hyperlinks, etc. on the text note editing interface. The user can also input content (such as text, graphics, etc.) by handwriting on the text note editing interface. The user can also click on the "Handwritten Note" option 1022 to create a handwritten note. The user can input text, graphics, etc. by handwriting on the handwritten note user interface (canvas).

[0077] For example, referring to Figure 1(2), in response to a click on the “Text Notes” option 1021, the tablet computer displays a first editing interface 103 for text notes. For example, the first editing interface 103 includes a text area where the user can input rich text content such as text, symbols, images, hyperlinks, and audio. In one implementation, the first editing interface 103 includes a virtual keyboard 1031, where the user can input rich text content such as text, numbers, and symbols into the text area of ​​the first editing interface 103 by clicking virtual keys in the virtual keyboard 1031; in another implementation, the terminal device includes an input device, such as a keyboard, where the user can input rich text content such as text, numbers, and symbols into the text area of ​​the first editing interface 103 by using the keyboard.

[0078] In one implementation, the first editing interface 103 includes a first handwriting option 1032. After the user clicks the first handwriting option 1032, they can perform handwriting input in the handwriting area of ​​the first editing interface 103. After the tablet computer receives the user's handwriting input in the handwriting area, the handwriting area will display the user's handwriting trajectory (text, numbers, symbols, graphics, etc.). The handwriting trajectory generated by the user's handwriting input will be retained in the handwriting area and will not disappear when the handwriting trajectory is finished being drawn.

[0079] For example, in some implementations, the first editing interface 103 may also include other controls. For instance, the first editing interface 103 may also include a "collapse" control 1033, in response to a click on the "collapse" control 1033, the virtual keyboard 1031 is no longer displayed on the editing interface. For instance, the first editing interface 103 may also include a text tool 1034, which allows the user to modify the font of the text in the text area. For instance, the first editing interface 103 may also include a second "add" control 1035, which, in response to a click on the second "add" control 1035, displays options such as "Gallery," "Table," and "Hyperlink" on the editing interface. The user can add images to the text area of ​​the editing interface by clicking the "Gallery" option; add tables to the text area of ​​the editing interface by clicking the "Table" option; and add hyperlinks to the text area of ​​the editing interface by clicking the "Hyperlink" option. For instance, the editing interface may also include a "record" control 1036, which allows the user to add recording files to the text area of ​​the editing interface by clicking the "record" control 1036.

[0080] In one implementation, the tablet displays a text note editing interface where the user can input rich text content such as text and audio. When the user needs to use handwriting input, the tablet displays a handwriting input interface for the text note. This handwriting input interface does not include rich text content, and the user can input handwritten content. After the user stops handwriting input, the tablet no longer displays the handwriting input interface but instead displays an updated editing interface, which includes an image generated based on the handwritten input.

[0081] For example, referring to FIG2, the tablet computer displays a second editing interface 201 for text notes. The second editing interface 201 includes a first image 2011 and a first text area 2012. The content of the first image 2011 is generated by handwriting traces, and the first text area 2012 includes text content. In one example, in response to a user's click operation on the first image 2011, the tablet computer displays a handwriting input interface 202, which includes a first handwriting area 2021. The user can perform handwriting input in the first handwriting area 2021 of the handwriting input interface 202. As shown in FIG2 (2), the area of ​​the first handwriting area 2021 is large, almost occupying the entire display area of ​​the handwriting input interface 202, which does not include the text area.

[0082] After the user finishes handwriting input, for example, upon receiving a click on the "Save" control 2022 in the handwriting input interface 202, the tablet displays the third editing interface 203. The third editing interface 203 includes a second image 2023, which is generated based on the handwriting input content in the first handwriting area 2021. It should be noted that the dashed boxes in the second editing interface 201 and the handwriting input interface 202 shown in Figure 2 are only used to indicate the area range; in actual applications, these dashed boxes will not be displayed. Some dashed boxes in the following figures are also only used to indicate the area range; in actual applications, these dashed boxes will not be displayed, and will not be elaborated further.

[0083] In this implementation, when the user selects the handwriting input function, the tablet's display interface jumps from the second text note editing interface 201 to the handwriting input interface 202, allowing the user to complete the handwriting input. As mentioned earlier, the handwriting input interface 202 only includes the user's handwritten input content and does not include the text content included in the first text area 2012. That is, when the user performs a handwriting input operation in the handwriting input interface 202, they cannot directly view the existing content in the text note (i.e., the text content included in the first text area 2012). If the user needs to see the existing content in the text note, they need to click the "Save" control 2022, causing the tablet to display the third editing interface 203. In this case, if the user still needs to perform handwriting input based on the original handwritten input content, they can click the second image 2023 to cause the tablet to display the handwriting input interface (which includes the handwritten input content corresponding to the second image 2023), and then the user can adjust their handwritten input content. This makes the user operation relatively cumbersome, especially when the user needs to adjust the handwritten input multiple times based on the existing content in the text notes, resulting in a poor user experience.

[0084] To address the aforementioned issues, this application provides a method for displaying input content that eliminates the need for users to perform handwriting input on a dedicated handwriting input interface. Instead, users can perform handwriting input within the handwriting area of ​​a text note editing interface. After the user finishes handwriting input, the handwritten content within the handwriting area is saved as an image and displayed in the updated editing interface.

[0085] In one example, the tablet displays a fourth editing interface for text notes. This fourth interface includes a second text area where the user can input rich text content such as text, symbols, images, tables, hyperlinks, and audio. In response to the user adding a handwriting area (e.g., clicking a handwriting control), the tablet displays a fifth editing interface for text notes. This fifth interface includes a third text area and a handwriting area. The user can input rich text content such as text, symbols, images, tables, hyperlinks, and audio in the third text area, and can also write handwritten content into the handwriting area. The third text area corresponds to the second text area; the rich text content displayed in the third text area is the same as that displayed in the second text area. The starting positions of the third and second text areas can differ, and their displayed screen sizes can also differ. Optionally, the initial size (e.g., width and height) of the handwriting area in the fifth editing interface is a preset value, and its size can be dynamically adjusted based on the handwritten input. The size of the handwriting area can also be changed in response to the user adjusting its boundaries. In one example, with the horizontal direction as the X-axis and the direction perpendicular to the ground as the Y-axis, the width of the handwriting area represents the length of the handwriting area in the X-axis direction, and the height of the handwriting area represents the length of the handwriting area in the Y-axis direction.

[0086] For example, referring to Figure 3a (1), the tablet computer displays a fourth editing interface 301 for text notes. The fourth editing interface 301 includes a first title bar 3011, a first editing area 3012, a first toolbar 3013, etc. The first title bar 3011 includes the title of the text notes, editing time, category options, etc. In the fourth editing interface 301, the entire first editing area is a text area, such as the second text area 31. Users can input rich text content such as text, symbols, pictures, tables, hyperlinks, and audio in the second text area 31. For example, the second text area 31 includes text a. The first toolbar 3013 includes multiple controls, such as a second handwriting option 30131, text tools, an "add" control, a "record" control, etc.

[0087] In one example, as shown in Figure 3a (2), in response to the user's click on the second handwriting option 30131, the tablet displays the fifth editing interface 302 of the text notes. The fifth editing interface 302 includes a second title bar 3021, a second editing area 3022, and a bottom toolbar 3023. The second title bar 3021 can be the same as the first title bar 3011; the second editing area 3022 includes a second handwriting area 32 and a third text area 31'; the user can write handwritten input content into the second handwriting area 32 by handwriting, and the user can input rich text content such as text, symbols, pictures, tables, hyperlinks, and audio in the third text area 31'; the bottom toolbar 3023 includes a handwriting tool control, and the user can click the handwriting tool control to select the line thickness, pen shape, color, etc. used for handwriting input in the second handwriting area 32. In one implementation, after the user clicks the second handwriting option 30131, the tablet computer adds a View of the second handwriting area 32 within the View corresponding to the editing area, thus adding a handwriting area to the editing area of ​​the editing interface. Correspondingly, the display area of ​​the text area within the editing area becomes smaller. For example, the display area of ​​the third text area 31' in Figure 3a (2) is smaller than the display area of ​​the second text area 31 in Figure 3a (1). The rich text content included in the third text area 31' is the same as the rich text content included in the second text area 31. In one implementation, the View corresponding to the third text area 31' is the View of the second text area 31. Optionally, in the editing interface, the handwriting tool control in the bottom toolbar appears simultaneously with the handwriting area; that is, when the editing area of ​​the editing interface includes the handwriting area, the bottom toolbar includes the handwriting tool control.

[0088] For example, referring to the sixth editing interface 303 shown in Figure 3a (3), the user can input text, numbers, graphics, etc. by handwriting in the second handwriting area 32. After the user finishes handwriting input in the second handwriting area 32, a save operation can be performed. For example, the save operation can be clicking an area outside the handwriting area in the editing interface, such as the user clicking the third text area 31'.

[0089] In one implementation, as shown in Figure 3b, in response to a user's save operation on the second handwriting area 32 (e.g., a click operation within the third text area 31'), the tablet displays a seventh editing interface 304 for the text notes. The editing area of ​​the seventh editing interface 304 includes a third image 33 and a fourth text area 31'. The third image 33 is generated based on the second handwriting area 32. In one example, the third image 33 and the second handwriting area 32 have the same size and are displayed in the same position on the screen; the fourth text area 31' and the third text area 31' can both have the same display area and display position. The elements in the third image 33 are generated based on the handwritten input content (handwriting trajectory) within the second handwriting area 32, and the fourth text area 31' includes the text 'a'.

[0090] In another implementation, as shown in Figure 3c, in response to a user's save operation on the second handwriting area 32 (e.g., a click operation within the third text area 31'), the tablet displays an eighth editing interface 304' for the text notes. The editing area of ​​the eighth editing interface 304' includes a fourth image 33' and a fifth text area 31"'. The fourth image 33' has the same starting display position (top left corner of the image) as the second handwriting area 32, but its display area differs from that of the second handwriting area 32. For example, the height of the fourth image 33' differs from that of the second handwriting area 32. For instance, the handwritten input content within the second handwriting area 32 is relatively small, and the height covered by the handwriting trajectory within the handwriting area is less than the height of the handwriting area. The height of the fourth image 33' is determined based on the height of the area covered by the handwritten input content within the handwriting area. Elements in the fourth image 33' are generated based on the handwritten input content within the second handwriting area 32. Correspondingly, the height of the fifth text area 31"' differs from that of the third text area 31', and the fifth text area 31"' includes the text 'a'.

[0091] Referring to Figures 3b and 3c, the seventh editing interface 304 and the eighth editing interface 304' do not include a handwriting area, and correspondingly, they do not include handwriting tool controls. For example, as shown in Figure 3b, the seventh editing interface 304 includes handwriting options, text tools, an "Add" control, a "Record" control, etc., and the eighth editing interface 304' is similar, so it will not be described again.

[0092] For ease of description, in this embodiment, the image generated based on the handwritten input content (handwriting trajectory) of the handwriting area is referred to as a handwritten image. For example, the third image 33 and the fourth image 33' are handwritten images. In this embodiment, the display area occupied by the handwritten image is referred to as the handwriting display area.

[0093] Optionally, in one implementation, in response to the user's save operation on the second handwriting area 32, the tablet computer saves information about the handwriting trajectory within the second handwriting area 32, for example, the information about the handwriting trajectory within the second handwriting area 32 is saved in a binary file; the tablet computer also determines the display position and size of the third image 33 (or the fourth image 33') based on the display position and size of the second handwriting area 32; the tablet computer also generates elements in the third image 33 (or the fourth image 33') based on the handwriting trajectory within the second handwriting area 32.

[0094] Understandably, the boundary frame of the third image 33 (or the fourth image 33') is only used to indicate the extent of the image. In practical applications, this boundary frame may not be displayed.

[0095] In this example, the text note editing interface is always displayed while the user is handwriting inputting, and the rich text content of the text area is always displayed on the editing interface, so that the user can always see the original content in the text area while handwriting inputting, bringing the user a smooth user experience.

[0096] After the handwriting area is saved, that is, after a handwritten image corresponding to the handwriting area is generated, the user can still adjust the initially entered content. For example, the user can continue to perform handwriting input based on the existing handwritten input, or delete or modify the existing handwritten input.

[0097] In some embodiments, when a user performs an editing operation on a handwritten image, the tablet displays a handwriting area corresponding to the handwritten image, and the original handwritten input content is displayed within the handwriting area. For example, the editing operation could be a click operation on the handwritten image.

[0098] For example, referring to Figure 4a (1), the tablet computer displays a seventh editing interface 304 for text notes, which includes a third handwritten image 33. In response to a user's click on the third handwritten image 33, the tablet computer displays a ninth editing interface 305 for text notes, as shown in Figure 4a (2). The third image 33 in the seventh editing interface 304 corresponds to the second handwritten area 32 in the ninth editing interface 305. Optionally, in one implementation, after receiving a user's click on the third image 33, the tablet computer determines the position and size of the corresponding handwritten area (i.e., the second handwritten area 32) based on the display position and size of the third image 33, and also generates the handwritten trajectory of the corresponding handwritten area (i.e., the second handwritten area 32) based on the handwritten trajectory information of the handwritten area saved when the third image 33 was generated. For example, the handwritten trajectories "1", "2", "3", and "4" are displayed in the second handwritten area 32. The user can also continue to input handwriting within the second handwritten area 32. For example, in response to a user's handwriting input within the second handwriting area 32, the tablet displays an updated tenth editing interface 306, as shown in Figure 4a (3). On the updated tenth editing interface 306, the handwriting trace "5" is displayed within the second handwriting area 32.

[0099] In some embodiments, users do not need to edit the handwritten image; instead, they can directly begin handwriting input on the image. The tablet receives the user's handwriting input on the image, deletes the image, generates a corresponding handwriting area, and displays the original handwriting input within that area. This way, users do not need to trigger the generation of the handwriting area through editing operations (such as clicking on the image) but can directly begin handwriting input on the image, making it convenient for them to write.

[0100] For example, referring to Figure 4b (1), the tablet displays a seventh editing interface 304 for text notes, which includes a third image 33. In response to the user's handwriting input on the third image 33, the tablet displays an eleventh editing interface 307 for text notes, which includes a third handwriting area 34, as shown in Figure 4b (2). The third image 33 corresponds to the third handwriting area 34. In one example, the display position and size of the third handwriting area 34 may be consistent with the display position and size of the third image 33. The handwriting input content (handwriting trajectory) displayed in the third handwriting area 34 includes two parts; one part of the handwriting trajectory is the same as the elements in the third image 33, for example, the handwriting trajectories "1", "2", "3", "4", which is generated based on the information of the handwriting trajectory saved when the third image 33 was generated; the other part of the handwriting trajectory is generated based on the user's handwriting input on the third image 33.

[0101] For ease of description, the handwriting area will be referred to as the canvas or handwriting canvas in this embodiment. In this embodiment, the user can adjust the height of the canvas by dragging (or pulling) the canvas boundary lines to meet the user's usage needs. It is understood that the canvas boundary lines may include a top boundary line and a bottom boundary line, and the user can adjust the height of the canvas, i.e., adjust the size of the canvas, by dragging the top boundary line and / or the bottom boundary line.

[0102] The following explanation uses the example of a user dragging the bottom edge of the canvas.

[0103] Referring to Figure 5a (1), the tablet displays the twelfth editing interface 401 of the note-taking application. The twelfth editing interface 401 includes a canvas 41, a bottom boundary line 4101, and a top boundary line 4102. It can be understood that the area between the bottom boundary line 4101 and the top boundary line 4102 is the canvas 41 that supports user handwriting input. In the twelfth editing interface 401, both the bottom boundary line 4101 and the top boundary line 4102 are in their initial states. Users can adjust the height of the canvas 41, i.e., adjust the size of the canvas 41, by long-pressing and dragging the bottom boundary line 4101 and / or the top boundary line 4102.

[0104] For example, as shown in Figure 5a(1), the user long-presses the bottom boundary line 4101 of the canvas, specifically by long-pressing the hot area of ​​the bottom boundary line 4101. In response to this user operation, the tablet displays the thirteenth editing interface 402 of the note-taking application, as shown in Figure 5a(2). In the thirteenth editing interface 402, the bottom boundary line 4101 of the canvas is in an adjustment state. For example, when the state of the bottom boundary line 4101 of the canvas changes from the initial state to the adjustment state, the lines of the bottom boundary line 4101 become thicker and / or the color becomes darker, thereby prompting the user of the change in the state of the bottom boundary line of the canvas.

[0105] Referring again to Figures 5a(2) and (3), the user slowly drags the bottom border line 4101 of the canvas downwards. In response to this user action, the bottom border line 4101 of the canvas moves downwards, and the height of the canvas 41 is adjusted according to the position of the bottom border line 4101. It is understood that as the bottom border line 4101 of the canvas moves downwards, the height of the canvas 41 increases. The user can release their hand after dragging the bottom border line 4101 of the canvas to the desired position. For example, referring to Figure 5a(3), the tablet displays the fourteenth editing interface 403 of the note-taking application, and the user has dragged the bottom border line 4101 of the canvas to the desired position. In response to the user's release action, the tablet displays the fifteenth editing interface 404 of the note-taking application, as shown in Figure 5a(4). In the fifteenth editing interface 404, the state of the bottom border line 4101 of the canvas is restored to the initial state. At this point, the user has completed the adjustment of the canvas size.

[0106] Thus, as the user slowly drags the canvas boundary line, its displayed position in the editing interface updates accordingly (i.e., the canvas boundary line slides within the editing interface), and the canvas height also changes. Understandably, in the scenario shown in Figure 5a where the user drags the bottom boundary line of the canvas, the movement of the canvas boundary line and the finger's movement are essentially synchronized, providing good responsiveness. Furthermore, the user can drag the bottom boundary line to the lower boundary of the handwritten stroke (i.e., the position corresponding to the lowest point of the handwritten trajectory). However, in practical applications, if the user drags the canvas boundary line too quickly or with too large an amplitude, the canvas boundary line may experience lag or fail to stop accurately at the desired position (e.g., the bottom boundary line cannot be dragged to the lower boundary of the handwritten stroke).

[0107] Figure 5b illustrates an exemplary scenario of a lag issue with the canvas boundary line sliding update. For example, as shown in Figure 5b(1), the tablet displays the sixteenth editing interface 405 of a note-taking application, and the user drags the bottom boundary line 4101 of the canvas downwards with their finger. When the user's finger slides to position 1 of the editing interface, the bottom boundary line 4101 of the canvas is also displayed at position 2 of the editing interface. The difference between position 1 and position 2 is greater than or equal to a preset threshold, and the difference can be perceived by the user, even clearly. As another example, as shown in Figure 5b(2), the tablet displays the seventeenth editing interface 406 of a note-taking application, and the user drags the bottom boundary line 4101 of the canvas upwards with their finger. When the user's finger slides to position 3 of the editing interface, the bottom boundary line 4101 of the canvas is also displayed at position 4 of the editing interface. The difference between position 3 and position 4 is greater than or equal to a preset threshold, and the difference can be perceived by the user, even clearly.

[0108] The same applies to scenarios where users drag the top edge of the canvas excessively, so I won't elaborate further.

[0109] Therefore, when users drag the canvas boundary line significantly to adjust the canvas height, the sliding update of the canvas boundary line is prone to lag, resulting in poor responsiveness of the canvas boundary line sliding and a poor user experience.

[0110] Figure 5c exemplarily illustrates a scenario where the canvas boundary line cannot be accurately slid to the target position. For example, as shown in Figure 5c (1), the tablet displays the eighteenth editing interface 407 of the note-taking application, and the user drags the bottom boundary line 4101 of the canvas upwards with their finger. As the user's finger slides upwards, the bottom boundary line 4101 of the canvas slides upwards accordingly, and the height of the canvas 41 decreases accordingly, as shown in the nineteenth editing interface 408 in Figure 5c (2). As mentioned earlier, when the user drags the bottom boundary line 4101 of the canvas, the sliding update of the canvas boundary line is sluggish. Continuing to refer to the twentieth editing interface 409 of the note-taking application shown in Figure 5c (3), the user's finger quickly slides to position 5 and releases. Here, position 5 can be the lower boundary position of the handwritten mark on the canvas, or a position above the lower boundary position of the handwritten mark on the canvas. In response to the user's operation, the bottom boundary line 4101 of the canvas stops at position 6 as the finger slides. The bottom boundary line 4101 of the canvas no longer slides upwards, and the state of the bottom boundary line 4101 of the canvas is restored to the initial state. The twenty-first editing interface 410 can be seen in Figure 5c (4). As shown in Figure 5c (3), position 6 is below position 5, and the bottom boundary line of the canvas has not accurately slid to the user's ideal position.

[0111] The same applies to scenarios where users quickly drag the top edge of the canvas, so I won't elaborate further.

[0112] Thus, when a user quickly drags the bottom edge of the canvas upwards and releases their finger after it reaches the lower edge of the handwriting, the bottom edge of the canvas cannot accurately stop at the lower edge of the handwriting. Instead, it stops at an intermediate position (such as position 6) and cannot continue to slide upwards. Similarly, when a user quickly drags the top edge of the canvas downwards and releases their finger after it reaches the upper edge of the handwriting (i.e., the position corresponding to the highest point of the handwriting trajectory), the top edge of the canvas cannot accurately stop at the upper edge of the handwriting. Instead, it stops at an intermediate position and cannot continue to slide downwards.

[0113] In the design of note-taking applications, the control of the canvas and its boundaries is entirely handled by the main thread of the application. This means that the main thread not only handles the updating of the entire application's user interface (UI), but also processes some logical tasks, which also depend on the execution of the main thread. However, when the user swipes the canvas boundaries sharply or quickly, the number of tasks that the main thread needs to perform becomes exceptionally large, leading to a prolonged time for updating the canvas and its boundaries, thus causing a stuttering issue when the canvas boundaries are swiped.

[0114] Referring to Figure 6a(1), the editing interface (or view in the editing interface) of the note-taking application includes a canvas view and a canvas boundary line view. This can also be understood as the editing interface being displayed based on the canvas view and the canvas boundary line view. The canvas view corresponds to the canvas (i.e., the handwriting area) in the editing interface, and the canvas boundary line view corresponds to the canvas boundary lines (top and / or bottom boundary lines) displayed in the editing interface. In this embodiment, the canvas view is transparent, meaning the canvas in the editing interface is transparent.

[0115] In one implementation, referring to Figure 6b (1), the canvas boundary line View may include a canvas top boundary line View and a canvas bottom boundary line View. The canvas top boundary line View and the canvas bottom boundary line View are located on the same layer, and the canvas View is also located on this layer. Specifically, the canvas top boundary line View corresponds to the canvas top boundary line displayed in the editing interface, and the canvas bottom boundary line View corresponds to the canvas bottom boundary line displayed in the editing interface. The widths of the canvas top boundary line View and the canvas bottom boundary line View can be the same. The widths of the canvas top boundary line View and the canvas bottom boundary line View can be the same as or different from the width of the canvas View; this embodiment does not limit this.

[0116] In one implementation, referring to Figure 6b(2), the size of the canvas boundary line View is larger than that of the canvas View, and the layer containing the canvas boundary line View (e.g., Layer 2) is below the layer containing the canvas View (e.g., Layer 1). The top and bottom boundary lines of the canvas displayed in the editing interface correspond to a single canvas boundary line View. Continuing to refer to Figure 6b(2), the height of the canvas View is H1, and the height of the canvas boundary line View is H2, where H2 is greater than H1. The width of the canvas boundary line View can be the same as or different from the width of the canvas View; this embodiment does not limit this.

[0117] Referring to the process shown in Figure 6a (2) and the scenario shown in Figure 5b, for example, in response to the user dragging the bottom boundary line of the canvas from position x1 to position x2, the note-taking application adjusts the size of the canvas View according to position x2, and adjusts the position of the corresponding canvas boundary line View according to the offset of the adjusted canvas View. When the user drags the bottom boundary line of the canvas significantly, such as dragging the bottom boundary line of the canvas from position x2 to position x3, from position x3 to position x4, etc., the main thread has to perform an unusually large number of tasks, which prolongs the time for updating the canvas View, and consequently prolongs the time for updating the corresponding canvas boundary line View according to the update result of the canvas View, thus causing the problem of lag in the sliding update of the bottom boundary line of the canvas.

[0118] Referring to the process shown in Figure 6a (3) and the scenario shown in Figure 5c, for example, the user quickly drags the bottom boundary line of the canvas to position xn (such as the lower boundary of the handwriting or a position above the lower boundary of the handwriting) and then releases their hand. Due to the user's excessive dragging speed, there may be incomplete sampling of finger swipe data, causing the canvas View update and the canvas boundary line View update to stop prematurely. That is, the note-taking application cannot update the canvas View based on position xn, and therefore cannot update the corresponding canvas boundary line View based on the update result, making the user perceive that the bottom boundary line of the canvas cannot be accurately dragged to the lower boundary of the handwriting.

[0119] To address the aforementioned issues, this application provides a display method. In this method, the canvas boundary lines and the canvas itself are decoupled. Updates to the canvas boundary lines are controlled by the JavaScript thread of the note-taking application, while updates to the canvas are controlled by the Android thread of the note-taking application. Since the canvas is transparent and invisible in the display interface, the user directly sees the canvas boundary lines. By decoupling the two, updates to the canvas boundary lines are no longer handled by the demanding Android thread. This allows the canvas boundary lines to smoothly follow the user's finger movement when the user quickly drags them, and the canvas boundary lines can slide and stop at the edge of the handwritten stroke.

[0120] It's important to note that the Android thread of the note-taking application can be understood as the main thread, responsible for updating the user interface and handling some logical tasks. The JavaScript thread of the note-taking application can be understood as a sub-thread of the main thread, implemented using JavaScript.

[0121] It should be noted that the canvas boundary line and the canvas are decoupled, which can also be understood as the update of the canvas boundary line view being decoupled from the update of the canvas view. The update of the canvas boundary line view is implemented on the JS side of the note-taking application, while the update of the canvas view is implemented on the Android side of the note-taking application. The update of the canvas boundary line view does not need to wait for the update of the canvas view to be completed before it is executed.

[0122] Figure 7a shows a schematic diagram of the structure of an electronic device. For example, the electronic device could be the tablet computer mentioned earlier. It should be understood that the electronic device shown in Figure 7a is merely an example of an electronic device, and an electronic device may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in Figure 7a can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0123] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0124] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0125] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0126] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0127] In some embodiments, the processor 110 may include one or more interfaces. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0128] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0129] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0130] Mobile communication module 150 can provide wireless communication solutions for electronic devices, including 2G / 3G / 4G / 5G. Wireless communication module 160 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0131] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology.

[0132] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0133] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0134] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0135] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0136] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc., which will not be listed here, and this application does not limit them.

[0137] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 194. When force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device detects the intensity of the touch operation based on the pressure sensor. The electronic device may also calculate the touch location based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities may correspond to different operation commands.

[0138] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor and display screen 194 together form a touchscreen, also called a "touch screen." The touch sensor detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor may also be located on the surface of the electronic device, in a different position than the display screen 194.

[0139] In this embodiment, the display screen 194 can receive and display handwriting input. A touch sensor can be used to receive click operations and collect the trajectory of handwriting input. A pressure sensor can detect the pressure applied to the display screen 194, thereby using the magnitude of the pressure to determine whether it is handwriting input; for example, higher pressure indicates handwriting input. For instance, assuming a first pressure value is detected, it is considered that the screen is being wiped, and a second pressure value is detected, it is considered that handwriting input is being performed, where the first pressure value is less than the second pressure value.

[0140] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0141] Figure 7b is a software structure block diagram of an electronic device according to an embodiment of this application.

[0142] The layered architecture of electronic devices divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: the application layer, the application framework layer, the Android Runtime and system libraries, and the kernel layer.

[0143] The application layer can include a series of application packages.

[0144] As shown in Figure 7b, the application package may include applications such as notes, camera, and gallery. The application package may also include applications such as calling, calendar, maps, navigation, music, video, SMS, WLAN (wireless local area networks), and Bluetooth.

[0145] In this embodiment, the note-taking application includes an Android client and a JavaScript client. The Android client may include a Web View, an Edit Web Fragment, and a Hand Write View Manager. In the note-taking application, the JavaScript thread continuously generates canvas height update tasks as the user swipes their finger and places these tasks into a task queue. The Android thread (such as the main thread) acts as a consumer of the task queue, retrieving canvas height update tasks from the queue and executing them. When the user lifts their finger, the JavaScript thread directly passes the latest canvas height update task to the Android thread, which then executes the latest task to update the canvas view height.

[0146] On the Android side of the note-taking application, it can include Web View, Edit Web Fragment, and Hand Write View Manager.

[0147] A Web View is a user interface element used to display web page content within an application. It has a built-in browser engine that can load and display web pages, web applications, or other online content. Users can browse web pages directly within the application without leaving it. Web Views are primarily used to display web page content, supporting functions such as loading, rendering, and interactivity, enabling a complete interactive experience with the web page content. In this embodiment, a Web View can be used to display note content, especially when the note content contains complex layouts, images, or requires online content support; in such cases, a Web View can achieve sophisticated interface effects.

[0148] Edit Web Fragment is used to implement editing functionality within a Web View, especially for editing handwritten notes. Edit Web Fragment encapsulates the components that manage the Web View, providing specific interfaces or methods, and also includes logic for handling handwritten input, such as recognizing handwritten text and converting it to text format.

[0149] Hand Write View Manager is the component responsible for managing handwriting input. It handles all handwriting-related operations, such as recognition, saving, and editing. Hand Write View Manager can interact with Edit Web Fragment and other related components. When a user makes a handwriting input, Hand Write View Manager captures it and converts it into editable text using recognition algorithms or other services. This text can then be passed to Edit Web Fragment for display or further editing. Additionally, Hand Write View Manager may provide other functionalities such as an eraser and undo / redo.

[0150] On the Android side of the note-taking application, ihandwrite can also be included. ihandwrite is an interface that defines the methods and properties related to handwriting input processing. The Hand Write View Manager and other components that need to use handwriting functionality will interact with the specific handwriting processing implementation through the ihandwrite interface. This means that a class that implements the ihandwrite interface can be used as a handwriting processing component.

[0151] In summary, the Web View displays the note content, the Edit Web Fragment provides editing functionality, the Hand Write View Manager manages handwriting input and processing, and the ihand write interface defines the relevant methods and properties for handwriting processing. These components and interfaces work together to implement the creation, editing, and display of handwritten notes.

[0152] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0153] As shown in Figure 7b, the application framework layer may include window manager service, input manager service, window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0154] The input manager service manages input devices such as touchscreens, keyboards, and mice. It ensures that input events (such as touch events and key events) are correctly received and processed by various applications in the system, and it is also responsible for handling the configuration and policies related to input devices.

[0155] The Window Manager Service manages all windows and views in the system. It ensures that each application's windows are displayed correctly on the screen and handles various window-related events, such as window creation, destruction, movement, and resizing. The Window Manager Service also works closely with other system services (such as the rendering engine) to ensure that content on the screen is updated and displayed efficiently and smoothly.

[0156] The window manager manages windowed applications. Content providers store and retrieve data, making that data accessible to applications. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0157] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0158] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0159] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0160] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), input flinger, surface flinger, and OpenGL for embedded systems.

[0161] The input flinger is responsible for passing input events to the appropriate window or application. When the input manager service determines that an input event should be passed to a certain application or window, it transmits the event to the input flinger. The input flinger then passes the input event to the correct window or application based on the current window layout and focus information. When there are multiple handwriting input events (referred to as handwriting input), the input flinger can combine the events and then pass them to the appropriate window or application.

[0162] In handwritten notes, the input flinger ensures that stylus input events distributed by the input manager service are accurately delivered to the handwritten note-taking application. This allows the application to process and display information accordingly based on the user's writing actions.

[0163] The surface flinger is responsible for combining the content of multiple applications or windows into a single final image and displaying it on the screen. The surface flinger receives drawing requests from each application or window and, based on the window's layout, size, position, and other information, combines this content into a final image. Then, it displays this image on the screen.

[0164] In handwritten notes, as the user writes, the handwritten note app draws its content (including handwriting, background, etc.) onto a surface and transmits it to a surface flinger for compositing. The surface flinger then combines the content from the handwritten note app with content from other applications or windows into a final image, which is then displayed on the screen so the user can see their handwritten notes.

[0165] OpenGL is a high-performance graphics rendering library that supports complex graphics and image rendering, as well as various special effects and animations.

[0166] The kernel layer is the layer between hardware and software. It includes at least display drivers, audio drivers, Wi-Fi drivers, and sensor drivers. The hardware includes at least a processor, display screen, Wi-Fi module, and sensors.

[0167] It is understood that the layers in the software structure shown in Figure 7b and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer layers than shown in the figure, and each layer may include more or fewer components, or combine some components, or split some components, or arrange the components differently, which is not limited by this application.

[0168] It is understood that, in order to implement the display method in the embodiments of this application, the electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0169] In this embodiment, the canvas boundary line View and the canvas View are decoupled. Updates to the canvas boundary line View are controlled by the JavaScript thread of the note-taking application, while updates to the canvas View are controlled by the Android main thread of the note-taking application. The Android main thread's updates to the canvas View are based on the update results (or update scenarios) of the canvas boundary line View by the JavaScript thread. Therefore, when the JavaScript thread updates the canvas boundary line View, it generates a canvas height update task based on the update status and transmits this task to the Android main thread for execution.

[0170] In this embodiment, the parameter for the canvas height update task can be the offset of the canvas boundary line View dragged by the user, that is, the offset of the canvas boundary line View in the height direction. The Android main thread can adjust the height of the canvas View based on the offset of the canvas boundary line View dragged by the user. Taking the user dragging the bottom boundary line of the canvas as an example, if the user drags the bottom boundary line of the canvas downwards and the offset of the canvas boundary line View is a positive value, then the Android main thread adjusts the height of the canvas View based on the offset of the canvas boundary line View, that is, it adds the offset value to the original height of the canvas View.

[0171] Understandably, as the user drags the canvas boundary, the JavaScript thread of the note-taking application continuously generates canvas height update tasks. If each canvas height update task is directly passed to the Android main thread for processing, it will cause the Android main thread to become overloaded. Moreover, the production rate of canvas height update tasks exceeds the consumption rate, which may lead to task blocking and increase the probability of task loss. Therefore, this embodiment adds a task queue to the Android side of the note-taking application. Optionally, the task queue is a first-in, first-out queue. After the JavaScript thread of the note-taking application generates a canvas height update task, it puts the canvas height update task into the task queue. The Android main thread, as the consumer of the task queue, reads the canvas height update task from the task queue and executes the canvas height update task.

[0172] In this way, the update of the canvas boundary view no longer depends on the update of the canvas view, but is updated directly based on the sampling results of finger swipes. In other words, in the note-taking application's editing interface, changes in the position of the canvas boundary line no longer require waiting for the canvas height to adjust, thereby optimizing the responsiveness of changes in the position of the canvas boundary line and improving the user experience.

[0173] In one implementation, referring to Figure 8a, as the user drags the canvas boundary line, the JS thread of the note-taking application continuously updates the canvas boundary line View based on the sampling results of the finger dragging. Then, based on the position changes of the canvas boundary line View, it continuously generates canvas height update tasks and writes these tasks sequentially into a task queue. As the consumer of the task queue, the Android main thread of the note-taking application sequentially retrieves canvas height update tasks from the task queue and executes them. In this way, the canvas height can be updated according to the task order.

[0174] When the user drags the canvas boundary line and releases their hand (or raises their hand), the JavaScript thread of the note-taking application updates the canvas boundary line View based on the position of the finger lifted. Then, based on the change in the position of the canvas boundary line View, it generates a canvas height update task _New and writes it to the task queue. When the user releases their hand, the Android main thread of the note-taking application reads the latest canvas height update task _New from the task queue and executes it.

[0175] In another implementation, referring to Figure 8b, as the user drags the canvas boundary line, the JS thread of the note-taking application continuously updates the canvas boundary line View based on the sampling results of the finger dragging. Then, based on the position changes of the canvas boundary line View, it continuously generates canvas height update tasks and writes these tasks sequentially into a task queue. As the consumer of the task queue, the Android main thread of the note-taking application sequentially retrieves canvas height update tasks from the task queue and executes them.

[0176] Unlike the method shown in Figure 8a, when the user drags the canvas boundary line and releases their hand (or lifts their hand), the JS thread of the note-taking application updates the canvas boundary line View based on the position of the finger lifted. Then, based on the change in the position of the canvas boundary line View, it generates a canvas height update task _New and submits it directly to the Android main thread so that the Android main thread can execute it directly. Understandably, while the JS thread submits the canvas height update task _New directly to the Android main thread, there may still be other canvas height update tasks cached in the task queue, but the Android main thread no longer reads canvas height update tasks from the task queue.

[0177] In this way, when the user raises their hand, the JS thread updates the canvas boundary line View according to the position of the finger raised, and the Android main thread can also update the canvas View height according to the newly generated canvas height update task, so that the position change of the canvas boundary line and the canvas height are more in line with the user's expectations.

[0178] Optionally, when the user presses the canvas boundary line, the Android main thread creates a new task queue to cache canvas height update tasks generated by the JS thread. Optionally, when the JS thread directly submits the canvas height update task_New to the Android main thread, the task queue can be destroyed since the Android main thread no longer needs to read tasks from it. Optionally, when the user releases their hand, the note-taking application's Android main thread reads the latest canvas height update task_New from the task queue and then destroys the task queue.

[0179] When a user drags the canvas boundary, especially during large or rapid finger swipes, the JavaScript thread generates canvas height update tasks faster than the Android main thread consumes them. To improve the consumption speed of canvas height update tasks, the Android main thread can retrieve tasks from the task queue at regular intervals based on the current cumulative number of tasks.

[0180] The Android main thread retrieves tasks from the task queue at regular intervals for consumption. This can be understood as the Android main thread reading n (where n is an integer) tasks sequentially from the task queue each time, but only executing the last of these n tasks, while the first n-1 tasks are not processed (e.g., discarded). The value of n can be dynamically set by the Android main thread, and its value is related to the current number of tasks in the task queue. The larger the current number of tasks in the task queue, the larger the Android main thread can set n. For example, with n set to 3, the Android main thread reads 3 tasks sequentially from the task queue each time, discards the first two tasks, and only executes the third task.

[0181] Thus, when canvas update tasks need to be executed frequently and continuously, and when these tasks are time-consuming, executing them sequentially will affect other UI update operations. The Android main thread can improve the efficiency of the main thread and enhance the system's responsiveness to user actions by checking if older canvas update tasks are waiting to be executed.

[0182] It should be noted that the Android main thread retrieves tasks from the task queue at certain intervals for consumption, which can be applied to the implementation shown in Figure 8a or Figure 8b, and will not be described in detail here.

[0183] The following example, using the user dragging the bottom boundary line of the canvas, illustrates the method provided in the embodiments of this application.

[0184] Figure 9 illustrates an exemplary schematic diagram of module interaction. As shown in Figure 9, the flow of the display method provided in this embodiment may include the following steps:

[0185] S501, in response to user actions, displays the canvas in the web view, along with the top and bottom borders of the canvas.

[0186] For example, a user action could be creating a new canvas, as shown in Figure 3a.

[0187] For example, a user action could be clicking on a handwritten image, as shown in Figure 4a.

[0188] For example, the user operation could be a handwriting input operation on a handwritten image, as shown in Figure 4b.

[0189] In this embodiment, the canvas is the handwriting area on the boundary interface of the note-taking application, and the canvas is transparent. From the user's perspective, the top and bottom boundary lines of the canvas are used to define the area of ​​the canvas.

[0190] S502, in response to the user completing the handwriting input, the web page view generates an instruction to calculate the canvas boundary line offset limit value and sends the instruction to the web page fragment editor.

[0191] The user's handwriting input can be understood as the action of the user leaving the canvas when using their finger or stylus for handwriting input.

[0192] Canvas boundary line offset limits can include: the upper limit to which the top edge of the canvas can be offset (top line_upper limit), the lower limit to which the bottom edge of the canvas can be offset (bottom line_lower limit), and the maximum canvas height (Height_max). The top line_upper limit and Height_max define the allowed range of offset for the top edge of the canvas, while the bottom line_lower limit and Height_max define the allowed range of offset for the bottom edge of the canvas.

[0193] Understandably, when the top edge of the canvas shifts upward, it indicates an increase in canvas height; when it shifts downward, it indicates a decrease in canvas height; when the bottom edge of the canvas shifts downward, it indicates an increase in canvas height; and when it shifts upward, it indicates a decrease in canvas height.

[0194] It should be noted that, in the embodiments of this application, for example, when the canvas boundary line is shifted upwards, the position coordinate decreases; when the canvas boundary line is shifted downwards, the position coordinate increases. It is understood that the reverse settings are similar, and will not be elaborated further here.

[0195] The maximum canvas height, Height_max, can be preset; however, this embodiment does not limit the maximum canvas height. It is understood that when the top edge of the canvas is dragged upwards to increase the canvas height to Height_max, the top edge of the canvas shifts upwards to the lower limit of the allowed top edge offset (top line_lower limit); when the bottom edge of the canvas is dragged downwards to increase the canvas height to Height_max, the bottom edge of the canvas shifts downwards to the upper limit of the allowed bottom edge offset (bottom line_upper limit).

[0196] In this embodiment, the upper limit position to which the top edge line of the canvas is allowed to offset is determined based on the position corresponding to the highest point of the handwritten trajectory in the canvas. Referring to Figure 10(1), when the user drags the top edge line of the canvas downwards so that the top edge line reaches or exceeds the position corresponding to the highest point of the handwritten trajectory, the top edge line of the canvas offsets to the top line_upper limit position. Similarly, the lower limit position to which the bottom edge line of the canvas is allowed to offset is determined based on the position corresponding to the lowest point of the handwritten trajectory in the canvas. Continuing to refer to Figure 10(1), when the user drags the bottom edge line of the canvas upwards so that the bottom edge line reaches or exceeds the position corresponding to the lowest point of the handwritten trajectory, the bottom edge line of the canvas offsets to the bottom line_lower limit position.

[0197] Considering that users may use different pen types for handwriting input, such as ballpoint pens, markers, and highlighters, the thickness of the handwriting traces displayed on the canvas will vary. To avoid obscuring the highest or lowest point of the handwriting due to dragging the canvas boundary lines, this embodiment modifies the positions corresponding to the highest and lowest points of the handwriting traces. Refer to Figure 10 (2) for the positions corresponding to the highest and lowest points of the handwriting traces. The position corresponding to the highest point is located above the position corresponding to the highest point, and the interval between them can be less than a preset value, for example, a first preset number (e.g., 8) pixels. The position corresponding to the lowest point is located below the position corresponding to the lowest point, and the interval between them can be a second preset number (e.g., 8 pixels). The values ​​of the first and second preset numbers can be equal or unequal; this embodiment does not limit this. Therefore, referring to Figure 10(2), when the user drags the top edge line of the canvas downwards so that the top edge line of the canvas reaches or exceeds the position corresponding to the highest point of the handwriting trajectory, the top edge line of the canvas shifts to the top line_upper limit position. Continuing to refer to Figure 10(2), when the user drags the bottom edge line of the canvas upwards so that the bottom edge line of the canvas reaches or exceeds the position corresponding to the lowest point of the handwriting trajectory, the bottom edge line of the canvas shifts to the bottom line_lower limit position.

[0198] S503, the handwriting view manager obtains the canvas boundary line offset limit value and returns the canvas boundary line offset limit value to the web page fragment editor.

[0199] Specifically, the handwriting view manager uses the position corresponding to the highest point of the handwriting trajectory in the canvas, or position 'L_top_1', as the top line_upper limit, and uses the position corresponding to the lowest point of the handwriting trajectory in the canvas, or position 'L_bottom_1', as the bottom line_lower limit.

[0200] For example, the handwriting view manager can use the getHandWriteMaxHeight function to calculate the top line_upper limit, bottom line_lower limit, and the maximum canvas height Height_max.

[0201] S504, the webpage fragment editor sends the current height of the canvas view and the offset limit value of the canvas boundary line to the JS thread.

[0202] For example, webpage fragment editing can pass the current height of the canvas view and the canvas boundary offset limit values ​​to the JS thread by calling the restrictHandWriteRange function on the JS side.

[0203] S505, the JS-side thread saves the current height of the canvas view and the offset limit value of the canvas boundary line.

[0204] In this way, when the user initiates handwriting input or drawing, the current height of the canvas view and the canvas boundary line offset limit values ​​saved on the JS side will be updated. Specifically, after the user finishes handwriting input, if the highest and / or lowest point of the handwriting changes, the top line_upper limit and bottom line_lower limit will also change, and the JS thread needs to save the latest canvas boundary line offset limit values.

[0205] In this embodiment, when a user initiates handwriting input or drawing, the JavaScript side generates a placeholder image based on the current height of the canvas view. A placeholder image typically refers to a temporary image or placeholder used to represent actual content. These images usually have simple geometric shapes or blurred patterns, used to fill blank areas where content has not yet been determined, allowing for a preview of the layout and typography. In this embodiment, a placeholder image can be used to represent locations or elements where content cannot be determined temporarily; specifically, it can be used to represent handwriting areas or handwritten images. For example, when a user leaves a blank area in their handwritten notes but has not yet determined which handwritten image to fill it with, a placeholder image can be used to represent this location, helping the user maintain the integrity and logic of the interface layout during handwriting input.

[0206] S506, in response to the user pressing and holding the bottom edge of the canvas, the JS thread determines that the dragged object is the bottom edge of the canvas and passes the dragged object to the web page fragment editor.

[0207] Drag events targeting the canvas boundary can be categorized into down, move, and up events. These events typically describe the pressing, moving, and releasing of an object on a UI element. A down event can refer to an event acting on the bottom boundary of the canvas, a move event can refer to a movement event adjusting the position of the bottom boundary, and an up event can be an event indicating the completion of the position adjustment. Similarly, a move event can be understood as a movement event adjusting the canvas height, and an up event can be an event indicating the completion of the height adjustment.

[0208] For example, when a user presses (or long-presses) the bottom edge of the canvas, the JS thread listens for the down event of the bottom edge of the canvas, and the JS's handleBottomLineDown method is triggered, confirming that the dragged object is the bottom edge of the canvas. Subsequently, the JS thread calls the Android's startDragging method to pass the information that the dragged object is the bottom edge of the canvas to the web page fragment editor.

[0209] S507, Web page fragment editing generates an indicator for calculating the offset limit value of the bottom boundary line of the canvas.

[0210] The canvas bottom boundary line offset limit value can include the lower limit position to which the canvas bottom boundary line is allowed to offset, and the maximum canvas height, Height_max.

[0211] S508, the handwriting view manager calculates the canvas bottom boundary line offset limit value and returns the canvas bottom boundary line offset limit value to the web page fragment editor.

[0212] The calculation method for the bottom line_lower limit can be found in the previous text and will not be repeated here.

[0213] S509, the webpage fragment editor sends the canvas bottom boundary line offset limit value to the JS-side thread.

[0214] Optionally, the maximum canvas height (Height_max) can be pre-stored on the JS side. The handwritten view manager only needs to calculate the lower limit position (bottom line_lower limit) to which the bottom boundary line of the canvas is allowed to offset, and return the bottom line_lower limit to the web page fragment editor so that the bottom line_lower limit can be passed to the JS thread through the web page fragment editor.

[0215] For example, webpage fragment editing can pass the current height of the canvas view and the canvas boundary offset limit values ​​to the JS thread by calling the restrictHandWriteRange function on the JS side.

[0216] Optionally, after the webpage fragment editor passes the current height of the canvas view and the canvas boundary offset limit value to the JS thread by calling the restrictHandWriteRange function on the JS side, it sets the target flag to state 1, indicating that the current height of the canvas view and the canvas boundary offset limit value have been passed to the JS side.

[0217] S510, the JS-side thread saves the offset limit value of the bottom boundary line of the canvas.

[0218] S511, in response to the user's dragging of the bottom edge of the canvas, the JS thread calculates and calibrates the offset of the bottom edge of the canvas based on the offset limit value of the bottom edge of the canvas and the sampled position of each finger.

[0219] For example, when a user drags the bottom edge of the canvas, the JS thread listens for the move event for the bottom edge of the canvas. The handleBottomLineMove method on the JS side is triggered, and then the JS thread continuously generates canvas height update tasks based on the offset of the bottom edge of the canvas. The canvas height update tasks can be passed to the Android side through the updateHandWriteImageMarginBottom method to add these canvas height update tasks to the task queue on the Android side.

[0220] As the user drags the bottom edge of the canvas, the position of the user's finger is sampled. For each sampled finger position, the difference between that finger position and the current position of the bottom edge of the canvas is calculated and used as the offset of the bottom edge of the canvas for that time.

[0221] In this embodiment, for example, a positive offset value indicates that the boundary line shifts downwards, and a negative offset value indicates that the boundary line shifts upwards. It can be understood that a negative offset value for the top boundary line of the canvas indicates an increase in canvas height; a positive offset value indicates a decrease in canvas height; a positive offset value for the bottom boundary line of the canvas indicates an increase in canvas height; and a negative offset value for the bottom boundary line of the canvas indicates a decrease in canvas height.

[0222] This embodiment also calibrates the offset of the bottom boundary line of the canvas calculated based on the finger sampling position to avoid the problem of the bottom boundary line of the canvas offset exceeding the limit, and also to avoid the problem of the user being unable to accurately drag the canvas boundary line to the lowest or highest point of the handwriting.

[0223] Referring to Figure 11, the process of the JS-side thread calculating and calibrating the offset of the bottom edge of the canvas in response to the user dragging the bottom edge of the canvas may include:

[0224] S5111, the JS-side thread obtains the current height of the saved canvas View, the lower limit position to which the bottom boundary line of the canvas can be offset, and the maximum height of the canvas.

[0225] S5112, the JS-side thread calculates the minimum allowed canvas height based on the current height of the canvas View and the lower limit position to which the bottom boundary line of the canvas can be offset.

[0226] Wherein, the minimum allowed canvas height = the current height of the canvas view - (the position of the bottom edge of the canvas - the lower limit position to which the bottom edge of the canvas is allowed to be offset).

[0227] S5113, the JS-side thread obtains the current finger position and calculates the offset of the canvas boundary line View based on the current finger position.

[0228] Here, the current finger position is the currently sampled finger position.

[0229] In this process, the offset of the canvas boundary line view refers to the offset in the height direction relative to the bottom boundary line of the canvas. Specifically, the offset of the canvas boundary line view = current finger position - current position of the bottom boundary line of the canvas.

[0230] S5114: The JS thread determines whether the bottom edge of the animation canvas is being dragged upwards or downwards. If the bottom edge of the animation canvas is being dragged upwards, then S5115 is executed; if the bottom edge of the animation canvas is being dragged downwards, then S5117 is executed.

[0231] When the offset of the canvas boundary line view is positive, it indicates that the bottom boundary line of the canvas is being dragged downwards to increase the canvas height; when the offset of the canvas boundary line view is negative, it indicates that the bottom boundary line of the canvas is being dragged upwards to decrease the canvas height.

[0232] In S5115, the JS thread determines whether the canvas height after offset has not reached the minimum canvas height based on the current height of the canvas view and the offset of the boundary line view. If yes, then execute S5116; otherwise, the process ends, and the currently calculated offset does not need to be calibrated.

[0233] The sum of the current height of the canvas view and the offset of the boundary line view is the height of the canvas after the offset.

[0234] S5116, the JS-side thread recalculates the offset of the boundary line View based on the lower limit position to which the bottom boundary line of the canvas is allowed to offset.

[0235] If the canvas height after offset does not reach the minimum canvas height, it means that the offset bottom edge of the canvas will cover or obscure the handwriting on the canvas, especially the lowest point of the handwritten note. In this case, the JavaScript thread can recalculate the offset of the canvas boundary view based on the allowed lower limit position of the canvas bottom edge. That is, it replaces the current finger position with the allowed lower limit position of the canvas bottom edge and recalculates the offset of the canvas boundary view. At this point, the offset of the canvas boundary view = the allowed lower limit position of the canvas bottom edge - the current position of the canvas bottom edge.

[0236] S5117: The JS thread determines whether the canvas height exceeds the maximum canvas height after offset based on the current height of the canvas view and the offset of the boundary line view. If yes, S5118 is executed; otherwise, the process ends and the currently calculated offset does not need to be calibrated.

[0237] S5118, the JS-side thread calibrates the offset of the boundary line View based on the maximum height of the canvas.

[0238] If the canvas height exceeds the maximum canvas height after offset, the offset of the boundary view needs to be calibrated.

[0239] For example, the JS-side thread calibrates the offset of the boundary view based on the maximum canvas height, so that after offsetting according to the calibrated offset, the canvas height is the maximum canvas height. Wherein, the calibrated offset = the offset before calibration - (current canvas view height + offset before calibration - maximum canvas height).

[0240] In S512, the JS-side thread updates the canvas boundary line View based on the canvas boundary line View offset and updates and saves the current canvas View height.

[0241] After each calculation and calibration of the canvas boundary line View offset by the JS-side thread, the JS-side thread updates the canvas boundary line View based on the offset and updates and saves the current canvas View height based on the offset.

[0242] Understandably, after the JS thread updates the canvas boundary line view based on the canvas boundary line view offset, the position of the bottom editing line of the canvas displayed on the editing interface will be adjusted accordingly.

[0243] It is understandable that the way the canvas boundary line View is updated according to the offset will also be different depending on the form of the View. This embodiment does not limit this.

[0244] For example, referring to FIG12a, the canvas boundary line View may include a canvas top boundary line View and a canvas bottom boundary line View. The canvas top boundary line View corresponds to the canvas top boundary line displayed in the editing interface, and the canvas bottom boundary line View corresponds to the canvas bottom boundary line displayed in the editing interface. When the user drags the canvas bottom boundary line downwards, the JS-side thread calculates and calibrates the offset of the canvas bottom boundary line View, and updates the canvas bottom boundary line View according to the offset. At this time, the JS-side thread updates the canvas bottom boundary line View according to the offset, specifically, the JS-side thread may move the canvas bottom boundary line View downwards according to the offset.

[0245] It should be noted that after the JavaScript thread updates the bottom border line view of the canvas based on this offset, the display position of the bottom border line view in the View layout and the display position of the bottom border line in the editing interface can be the same, or they can be mapped according to certain rules. This embodiment does not limit this. For example, based on the offset determined by the JavaScript thread, the bottom border line view of the canvas moves down from position 1 to position 2. After the bottom border line view of the canvas moves down from position 1 to position 2, the display position of the bottom border line in the editing interface moves down from position 1' to position 2'. The representation of position 2 and position 2' may be the same or different, but there is a certain mapping relationship between them. The same applies to position 1 and position 1', which will not be elaborated further.

[0246] For example, referring to Figure 12b, the size of the canvas boundary line View is larger than the canvas View, and the layer containing the canvas boundary line View (e.g., Layer 2) is below the layer containing the canvas View (e.g., Layer 1). The top and bottom boundary lines of the canvas displayed in the editing interface correspond to a single canvas boundary line View. That is, regardless of whether the user drags the top or bottom boundary line, the JS thread updates this canvas boundary line View. Taking the user dragging the bottom boundary line downwards as an example, the JS thread updates this canvas boundary line View based on this offset. Specifically, the JS thread can adjust the size and / or position of the canvas boundary line View based on this offset, so that after updating this canvas boundary line View, the display position of the bottom boundary line in the editing interface shifts downwards. For example, before the JS thread updates the canvas boundary view based on the offset, the height of the canvas boundary view is H2. After the JS thread updates the canvas boundary view based on the offset, the height of the canvas boundary view becomes H3, and H3 is greater than H2.

[0247] It should be noted that after the JS thread updates the bottom boundary line view of the canvas based on this offset, for example, if the size and / or position of the canvas boundary line view is adjusted, the adjustment method is also related to the downward movement of the bottom boundary line of the canvas displayed on the editing interface. This embodiment does not limit this.

[0248] In S513, the JS-side thread generates a canvas height update task based on the View offset of the canvas boundary line and adds the canvas height update task to the task queue.

[0249] Each time the JS thread calculates and calibrates a canvas boundary line View offset, it generates a canvas height update task based on the canvas boundary line View offset and adds the canvas height update task to the task queue.

[0250] For example, the parameter or variable for the canvas height update task is the canvas boundary line View offset.

[0251] For example, the JavaScript thread can pass the canvas height update task to the Android side via the `updateHandWriteImageMarginBottom` method, such as passing it to a webpage clip editor. The webpage clip editor will add the received canvas height update task to its task queue. The webpage clip editor will not block the passing of the canvas height update task from the JavaScript side to the Android side.

[0252] S514, the webpage fragment editor updates the height of the canvas view based on the continuously acquired canvas height update task.

[0253] In this embodiment, the JS-side thread is the producer of the canvas height update task in the task queue, and the web page fragment editing is the consumer of the canvas height update task in the task queue.

[0254] The webpage clip editor continuously retrieves canvas height update tasks from the task queue and updates the height of the canvas view according to each of these tasks. It's understandable that the webpage clip editor's consumption of canvas height update tasks involves retrieving the task from the task queue and updating the canvas view's height accordingly.

[0255] When a user drags the bottom edge of the canvas, especially during large or rapid finger swipes, the JavaScript side generates canvas height update tasks faster than the Android side consumes them. To improve the consumption speed of canvas height update tasks, the Android side can retrieve tasks from the task queue at regular intervals based on the current cumulative number of tasks.

[0256] Optionally, when the webpage fragment editor receives a canvas height update task, it can first check if the target flag is in state 1. If yes, it indicates that the Android side has passed the current height of the canvas view and the canvas boundary offset limit values ​​to the JS side; if no, it indicates that the Android side may not have passed the current height of the canvas view and the canvas boundary offset limit values ​​to the JS side. To ensure the reliability of the canvas height update task generated by the JS side, the webpage fragment editor can again pass the current height of the canvas view and the canvas boundary offset limit values ​​to the JS thread by calling the JS side's `restrictHandWriteRange` function, and set the target flag to state 1.

[0257] In response to the user's hand-raising action, the JS-side thread calculates the offset of the canvas boundary line View based on the offset limit value of the bottom boundary line of the canvas and the latest sampled finger position.

[0258] For example, when a user raises their hand along the bottom edge of the canvas, the JavaScript thread listens for a move event related to the bottom edge. The JavaScript's handleBottomLineUp method is triggered, allowing the JavaScript thread to obtain the position of the user's raised hand, i.e., the most recently sampled finger position. Based on this, the JavaScript thread can recalculate the offset of a canvas edge view. Similarly, the JavaScript thread can calibrate this offset using the process shown in Figure 11.

[0259] In S516, the JS-side thread updates the canvas boundary line View based on the canvas boundary line offset and updates and saves the current canvas View height.

[0260] Similar to S512, the JS-side thread updates the canvas boundary line View based on the offset of the canvas boundary line View determined when the user raises their hand, and updates and saves the current canvas View height based on this offset.

[0261] In this way, the movement of the bottom edge of the canvas can be consistent with the user's dragging operation, and the offset of the bottom edge of the canvas meets the user's expectations.

[0262] In S517, the JS-side thread generates the latest canvas height update task based on the view offset of the canvas boundary line, and sends the canvas height update task to the web page fragment editor.

[0263] For example, a JS-side thread can call Android's updateElementAttributeInfo method to pass the latest canvas height update task to the web page fragment editor.

[0264] S518, the webpage fragment editor updates the height of the canvas view based on the latest canvas height update task received.

[0265] In this way, the webpage clip editor updates the height of the canvas view again based on the latest canvas height update task received, so that the final determined canvas height can meet the user's expectations, and will not have problems such as the inability to accurately adjust the canvas height as shown in Figure 5c. It can also optimize the responsiveness of changes in the position of the canvas boundary line.

[0266] For any parts of this process that are not explained in detail, please refer to the previous text; they will not be repeated here.

[0267] The following explanation, using a specific example, illustrates the solution provided in the embodiments of this application.

[0268] Referring to Figure 13(1), the tablet computer displays the twenty-second editing interface 601 of the note-taking application. As mentioned earlier, the canvas view corresponds to the canvas displayed in the twenty-second editing interface 601; the canvas boundary line view corresponds to the bottom boundary line and the top boundary line of the canvas displayed in the twenty-second editing interface 601, or the canvas boundary line view includes the bottom boundary line view corresponding to the bottom boundary line of the canvas displayed in the twenty-second editing interface 601, and the canvas boundary line view includes the top boundary line view corresponding to the top boundary line of the canvas displayed in the twenty-second editing interface 601. Continuing to refer to Figure 13(1), in response to the user dragging the bottom boundary line of the canvas and sliding the finger position from position 7 to position 8, the JS thread calculates and calibrates the offset corresponding to the canvas boundary line view, and updates the canvas boundary line view according to the offset. For an explanation of how the JS thread updates the canvas boundary line view according to the offset, please refer to the previous text, which will not be repeated here.

[0269] After the JavaScript thread updates the canvas boundary view based on the offset, the tablet displays the twenty-third editing interface 602 of the notes application. In the twenty-third editing interface 602, the bottom boundary line of the canvas is displayed at position 8. Continuing to refer to Figure 13, after the JavaScript thread updates the canvas boundary view based on the offset, the JavaScript thread generates a canvas height update task based on the offset and adds the canvas height update task to the task queue. Subsequently, the Android main thread can obtain the canvas height update task from the task queue and update the height of the canvas view according to the canvas height update task.

[0270] In other words, in this embodiment, the update of the canvas boundary line view no longer depends on the update of the canvas view height. The update of the canvas boundary line view is performed independently on the JS side, and the Android main thread updates the height of the canvas view based on the update result of the canvas boundary line view on the JS side.

[0271] The above examples all use the method of adjusting the canvas height by dragging the bottom edge of the canvas. The process and logic of adjusting the canvas height by dragging the top edge of the canvas are similar and will not be repeated here.

[0272] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned related method steps to implement the display method in the above embodiment.

[0273] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the display method described in the above embodiment.

[0274] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the display methods in the above-described method embodiments.

[0275] In this embodiment, the electronic devices (such as mobile phones), computer storage media, computer program products, or chips are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0276] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0278] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A display method, characterized in that, include: The first interface of the target application is displayed. The first interface is based on a first view and a second view. The first view corresponds to the target boundary line of the handwritten canvas, and the second view corresponds to the canvas. The second view is transparent. The target boundary line is the top boundary line or the bottom boundary line. In response to the finger position sliding from a first position to a second position when the target boundary line is dragged, the first view is updated; After updating the first view, the second interface of the target application is displayed; In the second interface, the target boundary line is displayed at the second position; After updating the first view, the height of the second view is updated based on the update result of the first view.

2. The method according to claim 1, characterized in that, Updating the first view includes: The first view is updated through a sub-thread of the target application.

3. The method according to claim 1 or 2, characterized in that, Updating the height of the second view based on the update result of the first view includes: The height of the second view is updated based on the update result of the first view via the main thread of the target application.

4. The method according to claim 3, characterized in that, Also includes: During the process of dragging the target boundary line, multiple canvas update tasks are generated sequentially through the sub-thread of the target application, and the multiple canvas update tasks are added to the task queue in order. The main thread of the target application retrieves a canvas update task from the task queue and updates the height of the second view according to the canvas update task.

5. The method according to claim 4, characterized in that, Retrieve a canvas update task from the task queue, and update the height of the second view according to the canvas update task, including: Each time, multiple canvas update tasks are retrieved from the task queue, and the height of the second view is updated according to the last canvas update task among these multiple canvas update tasks.

6. The method according to claim 4, characterized in that, Also includes: In response to the operation of stopping dragging the target boundary line and raising the hand, the first view is updated according to the raised hand position through the sub-thread of the target application, and a target canvas update task is generated. The target canvas update task is passed to the main thread of the target application through a sub-thread of the target application; The target canvas update task is executed through the main thread of the target application.

7. The method according to claim 1, characterized in that, Updating the first view includes: Calculate the offset of the first view based on the current position of the finger; The offset of the first view is calibrated according to the offset limit of the first view; The first view is updated based on the offset of the calibrated first view.

8. The method according to claim 7, characterized in that, The target boundary line is the bottom boundary line; The second position is higher than the first position; Based on the offset limit of the first view, the offset of the first view is calibrated, including: Calculate the sum of the offset of the first view and the current height of the second view; Determine a minimum height value that the accumulated sum is less than the second view; the minimum height value is the difference between the current height of the second view and a first height value, the first height value is the difference between the position of the target boundary line and a third position; the third position is the lower limit position to which the bottom boundary line of the canvas is allowed to offset. Based on the third position, the offset of the first view is recalculated as the offset of the calibrated first view.

9. The method according to claim 8, characterized in that, The third position is the lowest point of the handwritten trajectory on the canvas; Alternatively, the third position may be the fourth position, which is located below the lowest point, and the difference between the fourth position and the lowest point is less than a preset threshold.

10. The method according to claim 7, characterized in that, The target boundary line is the bottom boundary line; the second position is lower than the first position; Based on the offset limit of the first view, the offset of the first view is calibrated, including: Calculate the sum of the offset of the first view and the current height of the second view; The sum is determined to be less than the maximum height value of the second view; The offset of the first view is recalculated based on the maximum height value of the second view, and used as the offset of the first view after calibration.

11. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, cause the electronic device to perform the display method as described in any one of claims 1-10.

12. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the display method as described in any one of claims 1-10.

Images