A method for updating a mark and an electronic device

Abstract

The application provides a marking updating method and an electronic device, and relates to the technical field of terminals. The method comprises the following steps: displaying a first interface of a first application program, and displaying a first text generated based on a handwriting operation on the first interface, wherein the first text is composed of at least one first paragraph, each first paragraph is composed of at least one line, and each line is composed of at least one first character; in response to a marking operation performed on at least one first target character in the first characters, displaying a first mark on the first target character; in a target paragraph of the first paragraph, in the case that a deletion operation is performed on at least one second target character in the first characters, moving the position of a first part of characters to fill a blank area formed by deleting the second target character; and if the first part of characters includes at least part of the first target character, updating the position of the first mark to display a first updated mark on the first target character after moving the position of the first part of characters. According to the scheme shown in the application, after a user performs a deletion operation on part of the handwritten characters in the handwritten text, the electronic device can automatically reformat the remaining handwritten characters, and update the mark corresponding to the handwritten character synchronously in the automatic formatting process. In this way, the problem that the handwritten character does not correspond to the mark after the electronic device automatically formats is avoided.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a tag update method and an electronic device. Background Technology

[0002] With the improvement of electronic device functions, electronic devices have brought great convenience to people's daily life and work. For example, information recording applications (such as note-taking applications) allow users to easily record information elements (such as image elements, text elements, formula elements, handwritten elements, etc.) anytime and anywhere, freeing users from relying on traditional paper-based recording methods and improving the efficiency of information recording.

[0003] If a user has recorded multiple handwritten characters through a message recording application, and the user needs to delete some of the handwritten characters from the handwritten text, the electronic device can reformat the remaining handwritten characters.

[0004] In some cases, the remaining handwritten characters may have markings. If, during the layout process of the remaining handwritten characters, the electronic device only layouts the remaining characters without updating the markings simultaneously, a mismatch between the remaining handwritten characters and the markings will occur, affecting the user experience. Summary of the Invention

[0005] This application provides a tag updating method and an electronic device that, after a user deletes some handwritten characters from handwritten text, automatically reformats the remaining handwritten characters and synchronously updates the tags corresponding to the handwritten characters. This avoids the problem of mismatched handwritten characters and tags after automatic reformatting by the electronic device.

[0006] In a first aspect, embodiments of this application provide a marker update method applied to an electronic device, comprising: displaying a first interface of a first application and displaying first text generated based on handwriting operations on the first interface, wherein the first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character; in response to a marker operation performed on at least one first target character among the first characters, displaying a first marker on the first target character; in the target paragraph of the first paragraph, in the case of a deletion operation on at least one second target character among the first characters, moving the position of a first portion of characters to fill the blank area formed by deleting the second target character, wherein the first portion of characters includes at least a portion of the first characters in the target paragraph other than the second target characters; if the first portion of characters includes at least a portion of the first target characters, updating the position of the first marker to display a first update marker on the first target character after moving the position of the first portion of characters.

[0007] The tag update method provided in this application can divide handwritten text based on characters, lines, and paragraphs. After a user deletes some handwritten characters from the text, the remaining handwritten characters are automatically reformatted within the paragraph, and the tags corresponding to the handwritten characters are updated synchronously during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and tags after automatic reformatting on electronic devices.

[0008] In one implementation, in the target paragraph of the first paragraph, after deleting at least one second target character from the first characters, and moving the position of the first part of the characters to fill the blank area formed by the deletion of the second target characters, the method further includes: if the first part of the characters does not include the first target character, then after moving the position of the first part of the characters, the first update mark is not displayed on the first text. This implementation demonstrates a specific method by which an electronic device synchronously updates the marks corresponding to handwritten characters when the remaining handwritten characters do not include previously marked handwritten characters. This avoids the problem of mismatch between handwritten characters and marks after automatic typesetting by the electronic device.

[0009] In one implementation, displaying a first interface of a first application and displaying first text generated based on a handwriting operation on the first interface includes: in response to a handwriting operation performed on the first interface, displaying at least one first character corresponding to the handwriting operation; obtaining at least one character rectangle based on the at least one first character, wherein each character rectangle corresponds to at least one first character, and the character rectangle is the smallest bounding rectangle corresponding to a first character representing a single word, or the character rectangle is the smallest bounding rectangle corresponding to multiple first characters used to represent a single word; determining at least one line rectangle based on the at least one character rectangle, wherein each line rectangle corresponds to a line in the first text; determining at least one paragraph rectangle based on the at least one line rectangle, wherein each paragraph rectangle corresponds to a first paragraph; and, when displaying the first text, performing the division of the first character based on the character rectangle, and performing the division of the line in the first text based on the line rectangle, and performing the division of the first paragraph based on the paragraph rectangle. Using this implementation method, the electronic device can recognize each character corresponding to the handwriting operation in real time during the user's handwriting operation, so as to realize the division of character rectangles, line rectangles and segment rectangles, so as to automatically reformat the remaining handwritten characters later, and synchronously update the marks corresponding to the handwritten characters during the automatic reformatization process.

[0010] In one implementation, determining at least one row rectangle based on at least one character rectangle includes: creating a first row rectangle based on the first obtained first character rectangle, and determining the first row center line corresponding to the first row rectangle, wherein the first row center line passes through the center point of the first character rectangle and is parallel to the horizontal direction; traversing second character rectangles, and when traversing to the current second character rectangle, traversing all currently created row rectangles to determine whether the current second character rectangle intersects with any row center line corresponding to any of the current row rectangles, wherein the second character rectangles include character rectangles after the first character rectangles obtained in the first order; if the current second character rectangle intersects with a target row center line, adding the current second character rectangle to the target row rectangle corresponding to the target row center line; if the current second character rectangle does not intersect with any row center line, creating a new row rectangle based on the current second character rectangle, and traversing the next second character rectangle; after all second character rectangles have been traversed, all row rectangles are obtained. This implementation demonstrates how row rectangles are generated, enabling automatic rearrangement of the remaining handwritten characters and synchronous updating of the corresponding markers during the automatic rearrangement process.

[0011] In one implementation, determining at least one segment rectangle based on at least one row rectangle includes: creating a first segment rectangle based on a first row rectangle; traversing the second row rectangles along a first direction, and when traversing to the current second row rectangle, obtaining the line spacing between the current second row rectangle and the previous row rectangle, wherein the first direction is the generation direction of the second row rectangle, and the second row rectangle includes row rectangles created after the first row rectangle; if the line spacing is greater than a preset line spacing threshold, or if the line spacing is greater than the average line spacing, creating a new segment rectangle and marking the current second row rectangle as belonging to the newly created segment rectangle, and then traversing the next second row rectangle; if the line spacing is less than or equal to the line spacing threshold, and if the line spacing is less than or equal to the average line spacing, marking the current second row rectangle as belonging to the last created segment rectangle, and then traversing the next second row rectangle; after all second row rectangles have been traversed, all segment rectangles are obtained. This implementation demonstrates how segment rectangles are generated, facilitating subsequent automatic reformatting of remaining handwritten characters, and synchronously updating the markings corresponding to the handwritten characters during the automatic reformatting process.

[0012] In one implementation, displaying a first mark on a first target character in response to a marking operation performed on at least one first target character in the first characters includes: acquiring a first mark trajectory in response to the marking operation performed on the first target character; and displaying the first mark on the first target character based on the first mark trajectory. This implementation illustrates a specific display method for the first mark, facilitating subsequent automatic reformatting of the remaining handwritten characters, and synchronously updating the marks corresponding to the handwritten characters during the automatic reformatting process.

[0013] In one implementation, in response to a marking operation performed on a first target character, after obtaining the first marking trajectory, the method further includes: determining at least one first target character rectangle corresponding to the first marking trajectory, wherein one first target character rectangle corresponds to one first target character; setting a first flag bit as a valid flag bit in the first character data structure corresponding to each first target character rectangle, so as to identify the correspondence between the first target character and the first marking trajectory based on the valid flag bit, wherein the first character data structure includes text information corresponding to the first target character, the first target character rectangle, and the first flag bit. This implementation demonstrates the setting method of the valid flag bit. In this way, the electronic device can update the markings corresponding to handwritten characters based on the setting of the valid flag bit, avoiding the problem of mismatch between handwritten characters and markings after automatic typesetting by the electronic device.

[0014] In one implementation, after deleting at least one second target character from the first character in the target paragraph of the first paragraph, the method further includes: determining at least one first target row rectangle, which includes at least one row rectangle containing the first part of the characters, and at least one row rectangle corresponding to the deletion operation; and deleting the first marker in the first target row rectangle. This implementation first deletes the original first marker to avoid the problem of mismatch between the marker and the character during the subsequent movement of the first part of the characters.

[0015] In one implementation, moving the position of the first set of characters to fill the blank area formed by deleting the second target character includes: determining the target paragraph corresponding to the blank area; traversing line rectangles along a first direction in the target paragraph to determine a starting line rectangle, an ending line rectangle, a first offset region, and a first index position, wherein the starting line rectangle is the first obtained line rectangle with at least a portion of blank area, the ending line rectangle is the next obtained line rectangle with at least a portion of blank area, the first offset region corresponds to the target blank area corresponding to the starting line rectangle, and the first index position corresponds to the starting position of the first character rectangle in the ending line rectangle; traversing character rectangles in the starting line rectangle and filling the blank area located in the first offset region. The subsequent word rectangles undergo an offset operation to move them to the first offset region, and the first offset region is updated. Within the ending line rectangle, word rectangles are traversed starting from the first index position, and an offset operation is performed on each traversed word rectangle to move it to the updated first offset region. The offset operation ends when the word rectangle traversal in the ending line rectangle is complete, or when the word rectangles in the ending line rectangle have filled the first offset region. The steps of traversing the line rectangles along the first direction in the target paragraph to determine the starting line rectangle, ending line rectangle, first offset region, and first index position are executed again. When all line rectangles in the target paragraph have been traversed, the movement of the first part of the characters is completed to fill the blank area. This implementation demonstrates the specific process by which an electronic device fills the blank area corresponding to a deletion operation. In this way, the electronic device reformatts the remaining handwritten characters within the paragraph and synchronously updates the markers corresponding to the handwritten characters during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and markers after automatic reformatting by the electronic device.

[0016] In one implementation, if the first part of the characters includes at least a portion of the first target characters, the position of the first marker is updated so that after moving the position of the first part of the characters, a first update marker is displayed on the first target character. This includes: after moving the position of the first part of the characters, displaying the first update marker on the first target character based on a valid flag bit. This implementation illustrates the specific process of updating the first marker. In this way, the electronic device can reformat the remaining handwritten characters in a paragraph and synchronously update the markers corresponding to the handwritten characters during the automatic reformatting process, avoiding the problem of mismatch between handwritten characters and markers after automatic reformatting by the electronic device.

[0017] In one implementation, after moving the position of the first part of the characters, a first update mark is displayed on the first target character based on a valid flag. This includes: traversing the first target line rectangle and determining the first target character rectangle corresponding to the valid flag within the current first target line rectangle; and setting the height and width of the first update mark on the first target character rectangle corresponding to the valid flag, based on the line height and line center of the current first target line rectangle, to display the first update mark. This implementation demonstrates how the first update mark is displayed. In this way, the electronic device can reformat the remaining handwritten characters within a paragraph, and synchronously update the marks corresponding to the handwritten characters during the automatic reformatting process, avoiding the problem of mismatch between handwritten characters and marks after automatic reformatting by the electronic device.

[0018] In one implementation, obtaining at least one character rectangle based on at least one first character includes: a first recognition engine acquiring handwriting data corresponding to the first character and recognizing the handwriting data to obtain at least one full recognition result, and then obtaining the character rectangle based on the full recognition result. Here, one character rectangle corresponds to one first character used to represent a single character, or one character rectangle corresponds to multiple first characters used to represent a single word. This implementation can accurately divide character rectangles for single characters or single words, avoiding layout errors caused by incorrect character rectangle division during subsequent re-layout, and thus preventing the problem of mismatch between handwritten characters and markings after automatic layout on electronic devices.

[0019] In one implementation, after obtaining at least one character rectangle based on at least one first character, the method further includes: a first recognition engine sending the full recognition results to a text layout module; the text layout module sorting the full recognition results according to a preset first order to ensure the character rectangles have a first order. This implementation illustrates the interaction between the first recognition engine and the text layout module. Based on this process, the remaining handwritten characters can be automatically reformatted, and the corresponding markers for the handwritten characters are updated synchronously during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and markers after automatic reformatting on electronic devices.

[0020] In one implementation, determining at least one row rectangle based on at least one character rectangle includes: the text layout module calling the row layout module to sequentially concatenate character rectangles with a first order to determine the row rectangle. This implementation demonstrates how the row layout module generates the row rectangle. Based on this process, the remaining handwritten characters can be automatically reformatted, and the corresponding markers for the handwritten characters are updated synchronously during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and markers after automatic reformatting on electronic devices.

[0021] In one implementation, determining at least one segment rectangle based on at least one row rectangle includes: the text layout module calling the segment layout module to concatenate the row rectangles to determine the segment rectangles. This implementation demonstrates how the segment layout module generates segment rectangles. Based on this process, the remaining handwritten characters can be automatically reformatted, and the corresponding markers for the handwritten characters are updated synchronously during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and markers after automatic reformatting on electronic devices.

[0022] In one implementation, when displaying first text, the segmentation of the first character is completed based on character rectangles, the segmentation of lines in the first text is completed based on line rectangles, and the segmentation of the first paragraph is completed based on paragraph rectangles. This includes: a segment layout module sending a first typesetting result to a first recognition engine, so that the first recognition engine determines the segmentation of the first character based on character rectangles, the segmentation of lines in the first text based on line rectangles, and the segmentation of the first paragraph based on paragraph rectangles. The first typesetting result includes character rectangles, line rectangles, and paragraph rectangles. This implementation illustrates the interaction between the segment layout module and the first recognition engine, enabling the first recognition engine to obtain the current paragraph segmentation state in the first interface.

[0023] In one implementation, in response to a marking operation on a first target character, obtaining a first marking trajectory includes: a view processing module obtaining the first marking trajectory in response to the marking operation on the first target character. This implementation illustrates, based on a software architecture, the specific method by which an electronic device obtains the first marking trajectory. Based on this process, subsequent automatic reformatting of the remaining handwritten characters can be implemented, and during the automatic reformatting process, the markings corresponding to the handwritten characters are updated synchronously. This avoids the problem of mismatch between handwritten characters and markings after automatic reformatting by the electronic device.

[0024] In one implementation, in response to the marking operation performed on the first target character, after obtaining the first marking trajectory, the method further includes: the view processing module sending the first marking trajectory to the first recognition engine; the first recognition engine determining at least one first target character rectangle corresponding to the first marking trajectory; and the first recognition engine setting a first flag bit in each first character data structure to a valid flag bit, so as to identify the correspondence between the first target character and the first marking trajectory based on the valid flag bit. This implementation illustrates the interaction process between the first recognition engine and the view processing module, enabling the first recognition engine to set the valid flag bit. This allows the electronic device to update the markings corresponding to handwritten characters based on the setting of the valid flag bit, avoiding the problem of mismatch between handwritten characters and markings after automatic layout by the electronic device.

[0025] In one implementation, before determining the target paragraph corresponding to the blank area, the method further includes: a view processing module triggering a reformatting process based on the deletion operation to send a reformatting instruction set to the first recognition engine; upon receiving the first reformatting result, the first recognition engine obtaining the deletion trajectory corresponding to the blank area based on the reformatting instruction set; and the first recognition engine sending the deletion trajectory to the text layout module. This implementation demonstrates the interaction between the first recognition engine and the text layout module. By sending the deletion trajectory of the blank area to the text layout module, the first recognition engine enables the text layout module to accurately locate the target paragraph to be processed based on the deletion trajectory corresponding to the blank area, thereby reformatting the target paragraph. During the automatic reformatting process, the markers corresponding to the handwritten characters are updated synchronously.

[0026] In one implementation, determining the target paragraph corresponding to the blank area includes: the text layout module determining the segment rectangle corresponding to the deletion trajectory as the target segment rectangle for the target paragraph. Using this implementation, the text layout module can accurately locate the target paragraph to be processed based on the deletion trajectory corresponding to the blank area, and then reformat the target paragraph.

[0027] In one implementation, after the text layout module determines the segment rectangle corresponding to the deletion trajectory as the target segment rectangle for the target paragraph, it further includes: the text layout module sending a first move instruction to the segment layout module, so that the segment layout module performs an offset operation on the row rectangles in each target segment rectangle based on the first move instruction. This implementation illustrates the interaction between the text layout module and the segment layout module, allowing the text layout module to notify the segment layout module that an offset operation needs to be performed.

[0028] In one implementation, within the target paragraph, line rectangles are traversed along a first direction to determine a starting line rectangle, an ending line rectangle, a first offset region, and a first index position. This includes: in response to a first move instruction, the segment layout module traverses line rectangles along the first direction within each target segment rectangle to determine the starting line rectangle, the ending line rectangle, the first offset region, and the first index position, and sends a second move instruction to the line layout module, causing the line layout module to perform an offset operation on the character rectangles within each line rectangle based on the second move instruction. This implementation illustrates the specific method by which the segment layout module reformatts the target paragraph based on the first move instruction, and the interaction process between the segment layout module and the line layout module. Thus, the segment layout module can fill line rectangles with blank areas during the traversal of each line rectangle in the target paragraph, achieving automatic layout.

[0029] In one implementation, within the starting row rectangle, word rectangles are traversed, and offset operations are performed on word rectangles located after the first offset region to move the word rectangles to the first offset region, and the first offset region is updated. This includes: in response to a second move instruction, the row layout module traverses the word rectangles within the starting row rectangle, and performs offset operations on word rectangles located after the first offset region to move the word rectangles to the first offset region, and updates the first offset region. This implementation illustrates the specific process of the row layout module performing offset operations on word rectangles. In this way, the row layout module can fill row rectangles with blank areas to achieve automatic layout.

[0030] In one implementation, within the ending row rectangle, the word rectangles are traversed starting from the first index position, and an offset operation is performed on each traversed word rectangle to move it to the updated first offset area. This includes: the line layout module traversing the word rectangles within the ending row rectangle starting from the first index position and performing an offset operation on each traversed word rectangle to move it to the updated first offset area. This implementation illustrates the specific process of the line layout module performing the offset operation on the word rectangles. In this way, the line layout module can fill row rectangles with blank areas to achieve automatic layout.

[0031] In one implementation, the offset operation ends when the word rectangles within the ending row rectangle have been traversed, or when the word rectangles within the ending row rectangle have filled the first offset region. This includes: when the word rectangles within the ending row rectangle have been traversed, or when the word rectangles within the ending row rectangle have filled the first offset region, the row layout module ends the offset operation and sends a third move instruction to the segment layout module. The third move instruction is used to notify the segment layout module that the current in-row offset operation has been completed. This implementation illustrates the interaction process between the row layout module and the segment layout module. In this way, the segment layout module can obtain the current operation state of the row layout module to further execute subsequent steps.

[0032] In one implementation, the step of traversing the line rectangles along a first direction within the target paragraph to determine the starting line rectangle, ending line rectangle, first offset region, and first index position is performed again. This includes: in response to a third move instruction, the paragraph layout module again performs the step of traversing the line rectangles along the first direction within the target paragraph to determine the starting line rectangle, ending line rectangle, first offset region, and first index position. This implementation illustrates a specific method by which the paragraph layout module reformatts the target paragraph based on the third move instruction. Thus, the paragraph layout module can fill line rectangles with blank areas during the traversal of each line rectangle in the target paragraph, achieving automatic layout.

[0033] In one implementation, after traversing all row rectangles within the target segment rectangle, the first part of the characters is moved to fill the blank area. This includes: the segment layout module sending a fourth move instruction to the text layout module after traversing all row rectangles within the target segment rectangle; the text layout module obtaining the current character rectangle, current row rectangle, and current segment rectangle based on the fourth move instruction, and sending a fifth move instruction to the first recognition engine; the first recognition engine obtaining the current character rectangle, current row rectangle, and current segment rectangle based on the fifth move instruction to complete the movement of the first part of the characters and triggering an update of the first interface to fill the blank area. Using this implementation, the text layout display component triggers an update of the first interface after confirming the completion of the offset operation, achieving automatic layout.

[0034] In one implementation, after deleting at least one second target character from the first characters in the target paragraph of the first paragraph, the method further includes: the view processing module sending first query information to the first recognition engine; in response to the first query information, the first recognition engine sending a first target row rectangle to the view processing module; and the view processing module deleting the first marker within the first target row rectangle. Using this implementation, the view processing module first deletes the existing first marker to avoid a mismatch between the marker and the character during subsequent movement of the first part of the characters.

[0035] In one implementation, if the first part of the characters includes at least a portion of the first target characters, the position of the first marker is updated so that after moving the position of the first part of the characters, a first update marker is displayed on the first target character. This includes: the view processing module displaying the first update marker on the first target character based on a valid flag bit. Using this implementation, the view processing module displays the first update marker based on a valid flag bit, ensuring the compatibility between the first update marker and the first target character, avoiding the problem of mismatch between the remaining handwritten characters and the marker, which would negatively impact the user experience.

[0036] Secondly, embodiments of this application provide a tag updating device applied to an electronic device. The tag updating device includes a first display module for displaying a first interface of a first application and displaying first text generated based on handwriting operations on the first interface. The first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character. A second display module is used to display a first tag on the first target character in response to a tagging operation performed on at least one first target character among the first characters. A moving module is used to move the position of a first portion of characters within the target paragraph of the first paragraph, in the case of a deletion operation on at least one second target character among the first characters, to fill the blank area formed by deleting the second target character. The first portion of characters includes at least a portion of the first characters in the target paragraph other than the second target characters. An updating module is used to update the position of the first tag if the first portion of characters includes at least a portion of the first target characters, so that after moving the position of the first portion of characters, a first update tag is displayed on the first target character.

[0037] The tag updating device provided in this application can divide handwritten text based on characters, lines, and paragraphs. After a user deletes some handwritten characters from the text, the device automatically reformatts the remaining handwritten characters within the paragraph, and synchronously updates the tags corresponding to the handwritten characters during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and tags after automatic reformatting by electronic devices.

[0038] Thirdly, embodiments of this application provide an electronic device, which includes a touch screen, a memory, and one or more processors; the touch screen, the memory, and the processors are coupled; wherein the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the tag update method provided by the first aspect and any implementation thereof.

[0039] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform a tag update method as provided in the first aspect and any implementation thereof.

[0040] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the tag update method provided by the first aspect and any of its implementations.

[0041] It is understood that the beneficial effects that the technical solutions provided in the third to fifth aspects above can achieve can be referred to the beneficial effects in the first aspect and any of its possible implementations, and will not be repeated here. Attached Figure Description

[0042] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is an illustration of a user writing handwritten characters in a note-taking application on a tablet.

[0044] Figure 2 This is a schematic diagram illustrating a scenario where a user marks certain handwritten characters.

[0045] Figure 3 This is a schematic diagram illustrating a scenario where a user deletes some handwritten characters.

[0046] Figure 4 This is a schematic diagram illustrating a scenario where an electronic device automatically typesets the remaining handwritten characters.

[0047] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0048] Figure 6 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application;

[0049] Figure 7 This is a first flowchart illustrating the tag update method provided in this application embodiment;

[0050] Figure 8 This is a schematic diagram of the first application interface provided in an embodiment of this application;

[0051] Figure 9 This is a schematic diagram of the first note interface provided in an embodiment of this application;

[0052] Figure 10 This is a schematic diagram of a user performing handwriting operations on a first interface, provided in an embodiment of this application.

[0053] Figure 11 This is the first interactive schematic diagram of the tag update method provided in the embodiments of this application;

[0054] Figure 12 This is a schematic diagram of a scenario where an electronic device acquires a word rectangle, as provided in an embodiment of this application.

[0055] Figure 13 This is a schematic diagram of the first scenario in which the line layout module provided in this application performs splicing of character rectangles;

[0056] Figure 14 This is a second scenario diagram illustrating the line layout module's splicing of character rectangles provided in this application embodiment;

[0057] Figure 15 This is a schematic diagram of the third scenario in which the line layout module provided in this application performs splicing processing on character rectangles;

[0058] Figure 16 This is a schematic diagram of the fourth scenario in which the line layout module provided in this application performs splicing processing on character rectangles;

[0059] Figure 17 This is a schematic diagram of the first scenario in which the segment layout module provided in this application performs splicing of row rectangles;

[0060] Figure 18 This is a second scenario diagram illustrating the segment layout module provided in this application for splicing row rectangles;

[0061] Figure 19 This is a schematic diagram of the third scenario where the segment layout module provided in this application performs splicing of row rectangles;

[0062] Figure 20 This is a schematic diagram of the fourth scenario in which the segment layout module provided in this application performs splicing processing on row rectangles;

[0063] Figure 21 This is the first schematic diagram of the first text obtained based on different writing habits provided in the embodiments of this application;

[0064] Figure 22 This is a second schematic diagram of the first text obtained based on different writing habits provided in the embodiments of this application;

[0065] Figure 23 The second interactive schematic diagram of the tag update method provided in the embodiments of this application;

[0066] Figure 24 This is a schematic diagram illustrating the formation method of the first mark provided in an embodiment of this application;

[0067] Figure 25 This is a schematic diagram of a scenario in which an electronic device, according to an embodiment of this application, determines a first target character rectangle;

[0068] Figure 26 This is a schematic diagram of the data storage structure provided in an embodiment of this application;

[0069] Figure 27This is a schematic diagram of the first scenario provided by the embodiments of this application, in which a user performs a deletion operation on a first interface;

[0070] Figure 28 This is a schematic diagram of a second scenario where a user performs a deletion operation on a first interface, as provided in this application embodiment.

[0071] Figure 29 This is a schematic diagram of the first part of the characters provided in the embodiments of this application;

[0072] Figure 30 This is a schematic diagram of a scenario in which an electronic device, according to an embodiment of this application, determines a first target row rectangle;

[0073] Figure 31 This is a schematic diagram of a scenario where an electronic device deletes a first marker, as provided in an embodiment of this application.

[0074] Figure 32 The third interactive schematic diagram of the tag update method provided in the embodiments of this application;

[0075] Figure 33 This is a schematic diagram of a scenario where the text layout module of this application determines the target segment rectangle;

[0076] Figure 34 This is a schematic diagram of a scenario where the segment layout module provided in this application determines the start row rectangle, the end row rectangle, the first offset region, and the first index position;

[0077] Figure 35 This is a schematic diagram illustrating a scenario where the row layout module provided in this application performs an offset operation;

[0078] Figure 36 This is a schematic diagram of a scenario where an electronic device displays a first update marker, as provided in an embodiment of this application.

[0079] Figure 37 This is the second flowchart of the marker update method provided in the embodiments of this application;

[0080] Figure 38 This is a schematic diagram of the structure of the marker update device provided in the embodiments of this application;

[0081] Figure 39 This is a schematic diagram of the structure of a marker updating device provided in another embodiment of this application;

[0082] Figure 40 This is a schematic diagram of the chip system provided in the embodiments of this application. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0084] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0085] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0086] The application scenarios of the embodiments of this application will be described below with reference to the accompanying drawings.

[0087] Electronic devices can typically add information elements to certain information recording applications (such as note-taking applications, notepad applications, memo applications, sticky note applications, etc.) to achieve information recording functions.

[0088] Information elements can include, for example, image elements, text elements, formula elements, and handwritten elements. Among them, handwritten elements are elements written using a handwriting device (such as a stylus) or by using one's finger, and include text elements, number elements, character elements, and line elements.

[0089] Users typically use the information recording function of electronic devices for learning and work. When users use the information recording function of electronic devices, there are scenarios where handwriting elements are added to information recording applications, such as writing handwritten characters in the note-taking interface of a note-taking application using a stylus.

[0090] For ease of explanation, the following description uses a tablet computer as the electronic device and a note-taking application as the information recording application. In practice, the electronic device can also be a mobile phone or other types of electronic device, and the information recording application can be a memo application or other types of application. The examples in this application do not constitute a limitation on the type of electronic device or the type of application.

[0091] Figure 1 This is an illustration of a user writing handwritten characters in a note-taking application on a tablet.

[0092] like Figure 1 As shown in the example, when the tablet computer 1 displays the first note-taking interface 2 of the note-taking application, the user can click on the pen control 3 in the first note-taking interface 2. In response to the click on the pen control 3, the tablet computer 1 enters the pen control mode. The user can perform handwriting operations on the first note-taking interface 2 using a stylus 4. In response to the handwriting operation, the tablet computer 1 displays at least one handwritten character 5 composed of pen lines on the first note-taking interface 2.

[0093] After the user performs a long handwriting operation, the tablet computer 1 displays multiple handwritten characters 5 on the first note interface 2.

[0094] Users may have a need to mark certain handwritten characters, such as the number 5.

[0095] Figure 2 This is a schematic diagram illustrating a scenario where a user marks certain handwritten characters.

[0096] like Figure 2 As shown, further based on the aforementioned example, if a user needs to mark multiple handwritten characters 5 within the K1 area, the user can click on the highlighter control 6. In response to the click on the highlighter control 6, the tablet computer 1 enters highlighter control mode. The user can then use the stylus 4 to mark multiple handwritten characters 5 within the K1 area using the highlighter control 6 on the first note-taking interface 2. In response to the marking operation, the tablet computer 1 displays highlighter marks 7 on the handwritten characters 5 in the K1 area of ​​the first note-taking interface 2. In this way, the tablet computer 1 highlights the handwritten characters 5 in the K1 area, making it easier for the user to quickly locate the K1 area while viewing the first note-taking interface 2.

[0097] Furthermore, users may also have the need to delete certain handwritten characters, such as the character 5.

[0098] Figure 3 This is a schematic diagram illustrating a scenario where a user deletes some handwritten characters.

[0099] like Figure 3 As shown, further based on the aforementioned example, if a user needs to delete multiple handwritten characters 5 in areas A1 and A2, the user can click on the erase control 8. In response to the click on the erase control 8, the tablet computer 1 enters the erase control mode. The user can then use the stylus 4 to erase the multiple handwritten characters 5 in areas A1 and A2 using the erase control 8 on the first note-taking interface 2. In response to the erase operation, the tablet computer 1 cancels the display of the handwritten characters 5 in areas A1 and A2 of the first note-taking interface 2.

[0100] To ensure the aesthetics of the first note interface 2, or to ensure the readability of the handwritten text, the user may need to reformat the remaining handwritten characters 5.

[0101] Currently, users typically need to manually move the remaining handwritten characters 5 multiple times using the lasso control 9 to allow the tablet 1 to reformat the handwritten text. This results in low editing efficiency for the electronic device and negatively impacts the user experience. Therefore, users generally hope that the tablet 1 can automatically reformat the remaining handwritten characters 5.

[0102] Figure 4 This is a schematic diagram illustrating a scenario where an electronic device automatically typesets the remaining handwritten characters.

[0103] like Figure 4 As shown, based on the aforementioned example, after the tablet computer 1 cancels the display of the handwritten character 5 in areas A1 and A2 of the first note interface 2, the user expects the tablet computer 1 to automatically move the handwritten character 5 in area A3 so as to fill the blank areas A1 and A2 with the handwritten character 5 in area A3.

[0104] After tablet computer 1 moves the handwritten character 5 in area A3, the handwritten character 5 in area K1 moves to area K2. If tablet computer 1 does not move the highlighter mark 7 in area K1 synchronously, a mismatch occurs between the highlighter mark 7 in area K1 and the handwritten character 5 in area K2. This forces the user to manually delete the highlighter mark 7 in area K1 and re-mark the handwritten character 5 in area K2, resulting in low editing efficiency and negatively impacting the user experience.

[0105] In order to enable electronic devices to automatically format handwritten characters and synchronously update the corresponding marks of handwritten characters during the automatic formatting process, thereby solving the problem of low editing efficiency of electronic devices, this application provides a mark updating method.

[0106] The tag update method provided in this application can be applied to electronic devices, including but not limited to mobile phones, tablets, personal computers, personal digital assistants (PDAs), workstations, large-screen devices (e.g., smart screens, smart TVs), wearable devices (e.g., smart bracelets, smartwatches), handheld game consoles, home game consoles, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality devices, and in-vehicle smart terminals. This application does not limit the specific technology or form of the electronic device. The electronic devices involved in this application can be equipped with... Harmony This application does not restrict the use of other operating systems.

[0107] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0108] like Figure 5 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, an antenna 01, an antenna 02, a mobile communication module 150, a wireless communication module 160, a sensor module 180, a camera 192, a display screen 193, etc. The sensor module 180 may include a touch sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a geomagnetic sensor 180D, an accelerometer sensor 180E, a proximity sensor 180F, a proximity light sensor 180G, etc. The gyroscope sensor 180B, barometric pressure sensor 180C, geomagnetic sensor 180D, and accelerometer sensor 180E can all be used to detect the motion state of the electronic device; therefore, they can also be called motion sensors.

[0109] Processor 110 may include one or more processing units, such as a central processing unit (CPU), application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0110] The controller can generate operation control signals based on the instruction opcode and timing signals, thereby controlling the process of acquiring and executing instructions.

[0111] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0112] DSPs are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency point, a digital signal processor is used to perform Fourier transforms on the frequency point energy, etc.

[0113] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0114] In some embodiments, processor 110 may include one or more interfaces. Interfaces may include Inter-Integrated Circuit (I2C) interfaces, Inter-Integrated Circuit Sound (I2S) interfaces, Pulse Code Modulation (PCM) interfaces, Universal Asynchronous Receiver / Transmitter (UART) interfaces, Mobile Industry Processor Interface (MIPI) interfaces, General-Purpose Input / Output (GPIO) interfaces, Subscriber Identity Module (SIM) interfaces, and / or Universal Serial Bus (USB) interfaces, etc. These interfaces are used to connect to other components in electronic device 100.

[0115] In this embodiment, the electronic device 100 can be connected to an external input device via a USB interface. The external input device can be, for example, a mouse or a stylus.

[0116] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0117] The memory may include internal memory 121. Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0118] Random access memory (RAM) can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data. Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into RAM for direct read and write by the processor 110.

[0119] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions.

[0120] In this embodiment, the electronic device 100 can first store files such as pictures, documents, and notes in an external non-volatile memory. When the electronic device 100 runs a note-taking application, it can retrieve information elements such as pictures, documents, and notes from the storage path of the external non-volatile memory.

[0121] The wireless communication function of electronic device 100 can be implemented through antenna 01, antenna 02, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0122] Antennas 01 and 02 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 01 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0123] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 01, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 01. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0124] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device or displays an image or video through the display screen 193. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0125] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. The wireless communication module 160 receives electromagnetic waves via antenna 02, modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 02.

[0126] In some embodiments, the antenna 01 of the electronic device 100 is coupled to the mobile communication module 110, and the antenna 02 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0127] Electronic device 100 implements display functions through a GPU, a display screen 193, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0128] The display screen 193 is used to display images, videos, etc. The display screen 193 includes a display panel. In some embodiments, the electronic device may include one or N display screens 193, where N is a positive integer greater than 1.

[0129] In this embodiment, the display screen 193 can be used to display various interfaces of the note-taking application.

[0130] Electronic device 100 can perform shooting functions through ISP, camera 192, video codec, GPU, display 193 and application processor.

[0131] The ISP (Image Signal Processor) is used to process data fed back from the camera 192. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set within the camera 192.

[0132] Camera 192 is used to capture still images or videos. An object passes through the lens to generate an optical image that is projected onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, the electronic device 100 may include one or N cameras 192, where N is a positive integer greater than 1.

[0133] Touch sensor 180A, also known as a "touch device," can be located on display screen 193. The touch sensor 180A and display screen 193 together form a touchscreen, also known as a "touchscreen." Touch sensor 180A is used to detect touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event.

[0134] In this embodiment, the electronic device 100 can determine the control, element, etc. selected by the user based on the data collected by the touch sensor 180A. The electronic device 100 can also determine the handwritten elements input by the user based on the data collected by the touch sensor 180A.

[0135] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100.

[0136] The 180C barometric pressure sensor is used to measure barometric pressure.

[0137] The geomagnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the geomagnetic sensor 180D to detect the opening and closing of the flip cover.

[0138] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0139] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0140] The proximity light sensor 180G may include, for example, a light-emitting diode and a light detector, such as a photodiode.

[0141] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0142] Figure 6 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0143] like Figure 6 As shown, the layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0144] The application layer can include a series of application packages.

[0145] like Figure 6 As shown, the application package can include applications such as camera, gallery, calendar, call, map, navigation, music, video, and SMS.

[0146] In this embodiment of the application, the application package may include information recording applications such as note-taking applications, memo applications, notebook applications, and sticky note applications.

[0147] In this embodiment, the information recording application may include a view processing module. When the information recording application is a note-taking application, the view processing module may be, for example, a canvas module.

[0148] The view processing module can be set at the application layer as a core component of the note-taking application, enabling operations such as drawing graphics, drawing images, drawing handwritten characters, or other drawing-related tasks. The view processing module is mainly used to implement specific business logic and user interaction functions.

[0149] In some implementations, the view processing module can also be part of the application framework layer, providing consistent drawing functions and interfaces for different applications. This application embodiment only illustrates the example of the view processing module being set up in the application layer; the specific setup of the view processing module is not limited in this application embodiment.

[0150] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0151] like Figure 6 As shown, the application framework layer may include a window manager, an input manager, a sensor manager, a phone manager, a resource manager, a notification manager, etc.

[0152] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0153] The input manager can be used to listen for user input events, such as clicks and swipes performed by the user's finger on the display screen 193 of the electronic device 100. By listening to input events, the electronic device 100 can determine whether it is in use.

[0154] In this embodiment of the application, the input manager can listen to the Action_Down event, which indicates that the stylus is pressed on the display screen 193, and the Action_Up event, which indicates that the stylus is lifted off the display screen 193.

[0155] The sensor manager is used to monitor data returned by various sensors in the electronic device 100, such as motion sensor data, proximity sensor data, and temperature sensor data. Using the data returned by each sensor, the electronic device 100 can determine whether it is shaking or whether the display screen 193 is obstructed.

[0156] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0157] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0158] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0159] In this embodiment of the application, the application framework layer may further include a stylus service layer, which can provide various interfaces and component services to the application layer to support handwriting input and recognition functions.

[0160] The stylus service layer may include a real-time recognition engine, which can be used to convert the user's handwritten characters into text.

[0161] Specifically, the real-time recognition engine may include a text layout presentation component (TextLayoutPresenter), a recognition manager (RecognizeManager), and a recognition engine (RecognizeEngine).

[0162] The text layout display component can be used to obtain the layout of handwritten text, such as obtaining the coordinates, width, and height of each handwritten character.

[0163] The recognition manager manages the handwritten character recognition process, including initiating handwritten character recognition, monitoring the recognition status, and processing the recognition results. The recognition manager ensures the recognition process runs smoothly and interacts with the recognition engine to obtain the results.

[0164] A recognition engine can be used to perform handwritten character recognition tasks. It can analyze handwritten characters based on machine learning algorithms or other recognition technologies and convert them into readable text format.

[0165] The stylus service layer may also include a text layout module (TextLayout), a line layout module (TextRow), and a paragraph layout module (TextParagraph).

[0166] The text layout module can be used for overall text layout calculations and text presentation. It can handle text stylization, such as font, color, and alignment. The text layout module can also handle text wrapping, alignment, and other formatting issues.

[0167] In this embodiment of the application, the text layout module can chain calls to the line layout module and the paragraph layout module, so that the text layout module calls the line layout module to perform line-by-line layout processing of the text, and the text layout module calls the paragraph layout module to perform paragraph-by-paragraph layout processing of the text.

[0168] The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0169] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0170] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0171] System libraries can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing libraries (e.g., OpenGL, EGL), 2D graphics engines (e.g., SGL), game engines, scene recognition modules, etc.

[0172] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0173] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0174] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0175] A 2D graphics engine is a graphics engine for 2D drawing. The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

[0176] The kernel layer can include the base kernel, extended kernel, hardware abstraction layer (HAL), and driver layer.

[0177] The basic kernel can implement core functions such as memory management, task management, inter-process communication, and interrupt management based on the memory (Mem) interface, task (Task) interface, inter-process communication (IPC) interface, and interrupt interface.

[0178] The Hardware Abstraction Layer (HAL) can abstract the hardware operation interface, encapsulating the underlying driver interface into a unified Application Programming Interface (API). This simplifies the complexity of hardware operations for applications.

[0179] It should be noted that the embodiments of this application are only illustrated by taking the HAL layer as part of the kernel layer. In fact, the HAL layer can also be independent of the kernel layer, and the embodiments of this application do not limit this.

[0180] In this embodiment, the HAL layer may have a supplier service layer. The supplier service layer can provide corresponding hardware and software support according to the needs of the stylus service layer. Specifically, it can provide handwriting and recognition capabilities based on hardware and software support.

[0181] The driver layer includes at least the display driver, camera driver, audio driver, sensor driver, and graphics driver.

[0182] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0183] Figure 7 This is the first flowchart of the tag update method provided in the embodiments of this application.

[0184] like Figure 7 As shown, in some embodiments, the method includes the following steps S11-S18.

[0185] In step S11, in response to the user's click on the first icon, the electronic device 100 displays the home screen of the first application.

[0186] Wherein, the first icon is the application icon of the first application. The electronic device 100 may be a tablet computer. The first application may be an information recording application, such as a note-taking application. In the following embodiments of this application, the electronic device 100 is a tablet computer, the first application is a note-taking application, and the first icon is a note icon, which are used as examples for illustrative purposes. The embodiments of this application do not limit the device type of the electronic device 100, the program type of the first application, or the icon type of the first icon.

[0187] Figure 8 This is a schematic diagram of the first application interface provided in the embodiments of this application.

[0188] like Figure 8 As shown, in one implementation, the electronic device 100 responds to a user's click on the note icon 10 by running a note application and displaying the note application's home screen 11.

[0189] Since the electronic device 100 is a tablet computer with a large screen size, the main interface 11 may include, for example, a first interface area 12 and a second interface area 13.

[0190] The first interface area 12 may include multiple controls. These controls may include, for example, a new note control 14, all notes control 15, text note control 16, handwritten note control 17, favorite notes control, card center control, and recently deleted control. Other controls may include, for example, folder controls and a new folder control.

[0191] The second interface area 13 may include the content corresponding to the currently selected control in the first interface area 12. For example, if the electronic device 100 can select all note controls 15 by default, then the electronic device 100 can display all notes created by the note-taking application in the second interface area 13.

[0192] The following is an exemplary description of how the electronic device 100 displays all notes created by the note-taking application in the second interface area 13.

[0193] For example, in the second interface area 13, the electronic device 100 can display all notes by displaying the cover of each note. These cover pages can be user-defined or generated by default.

[0194] Furthermore, below each note cover, the electronic device 100 can display the note's name and creation time (or last edit time). For example, the electronic device 100 can display the note's name as "Name 1" and the note's creation time as "October 1st" below the note cover.

[0195] It should be noted that notes created by note-taking applications include different note types, such as handwritten notes and text notes. Handwritten notes are notes that include handwritten elements in the recorded information, while text notes are notes that do not include handwritten elements in the recorded information.

[0196] To facilitate the differentiation of different note types, the electronic device 100 can display a stylus icon 18 below the cover of the handwritten note, allowing the user to quickly distinguish the handwritten note in the second interface area 13. This application embodiment can be applied to both the marking and updating process of historically stored handwritten notes and the marking and updating process of newly created notes. This application embodiment is not limited in this respect.

[0197] The following embodiments of this application illustrate the tag update method using the process of creating a new handwritten note on an electronic device 100 as an example.

[0198] In step S12, in response to the click operation on the newly created control in the main interface, the electronic device 100 displays the first interface.

[0199] The "Create New" control is used to create the interface for recording information elements. In a note-taking application, the "Create New" control can be, for example, like... Figure 8 The newly created note control 14 is shown. In a memo application, the newly created control can be, for example, a newly created memo control (not shown in the figure). This application embodiment does not limit the specific type of the newly created control.

[0200] When the newly created control is the new note control 14, the first interface is, for example, as described in the aforementioned embodiment. Figure 1 The first note interface shown is 2.

[0201] In other words, in one implementation, the electronic device 100 displays the first note interface 2 in response to the user's click operation on the new note control 14.

[0202] The following describes the interface format of the first note interface 2.

[0203] Figure 9 This is a schematic diagram of the first note interface provided in the embodiments of this application.

[0204] like Figure 9 As shown, the first note interface 2 may include a first floating button (Bar) 19, an undo control 20, a reverse undo control 21, a save control 22, an editing bar 23, a record area 24, a page number control 25, a copy control 26, a search control 27, etc.

[0205] The first floating button 19 is used to provide control functions for the first note interface 2, such as controlling the first note interface 2 to be displayed in full screen, or in split screen, or in floating window, or to minimize the floating window, or to close the floating window.

[0206] The undo control 20 is used to undo the user's operation of placing an information element in the recording area 24. Specifically, when an information element exists in the recording area 24, each time the user clicks the undo control 20, the electronic device 100 performs an operation to undo the information element corresponding to the previous placement operation.

[0207] The undo control 21 is used to restore the information element corresponding to the previous undo operation. Specifically, each time the user clicks the undo control 21, the electronic device 100 performs an operation to restore the information element corresponding to the previous undo operation.

[0208] It should be noted that when there are no recoverable information elements in the recording area 24, the undo control 21 is set to an unavailable state. Specifically, the electronic device 100 can gray out the undo control 21 and not respond to click operations on the undo control 21.

[0209] The save control 22 is used to save the information elements currently existing in the record area 24.

[0210] The editing bar 23 includes AI pen control 28, hard pen control 29, highlighter control 30, eraser control 31, lasso control 32, graphic placement control 33, image placement control 34, text placement control 35, formula placement control 36, marker pen control 37, ruler placement control 38, zoom control 39, more function addition control 40, and settings control 41.

[0211] More controls can be set in the editing bar 23, and this application embodiment does not limit the types of controls in the editing bar 23. Subsequent embodiments of this application will further explain how to use the controls in the editing bar 23.

[0212] Recording area 24 is used to record information elements recorded by the user through various input methods.

[0213] Page number control 25 is used to display the total number of pages in the current note, as well as the page number corresponding to the current note page. For example, "1 / 1" indicates that the current note has a total of 1 page, and the current note page is page 1. It should be noted that the recording area 24 can be divided into multiple note pages.

[0214] The copy control 26 is used to copy all the content of the current note page and enable the electronic device 100 to automatically generate a 1-page note identical to the current note page.

[0215] Search control 27 is used to search for a user-specified note page or information element.

[0216] It should be noted here that, in the embodiments of this application, Figure 9 The interface form of the first note-taking interface 2 shown is for illustrative purposes only. In actual applications, the first note-taking interface 2 may include more or fewer controls than shown in the figure, and the style of the controls is not limited to the style shown in this application. The embodiments of this application do not limit the interface form of the first note-taking interface 2.

[0217] In step S13, in response to a handwriting operation performed on the first interface, the electronic device 100 displays a first text corresponding to the handwriting operation, wherein the first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character.

[0218] In one implementation, step S13 includes steps S131-S135.

[0219] In step S131, in response to a click operation on the handwriting control in the first interface, the electronic device 100 enters the handwriting input state. In response to the handwriting operation performed on the first interface, the electronic device 100 displays at least one first character corresponding to the handwriting operation.

[0220] like Figure 9 As shown, further based on the aforementioned example, when the first interface is the first note interface 2, the handwriting controls include, for example, a pen control 29, a highlighter control 30, etc. This application embodiment does not limit the specific control type of the handwriting controls. The first character is, for example, a handwritten character 5 composed of pen lines.

[0221] In one implementation, the recording area 24 of the first note interface 2 may include an editing layer, a top element layer, a handwriting layer, and a bottom element layer. In this embodiment, the handwritten character 5 may be drawn on the handwriting layer.

[0222] Figure 10 This is a schematic diagram of a user performing handwriting operations on a first interface, provided in an embodiment of this application.

[0223] like Figure 10 As shown, in one implementation, the user can click on the pen control 29. In response to the click on the pen control 29, the electronic device 100 can highlight the pen control 29 and enter the pen handwriting input state.

[0224] Furthermore, when a user performs a handwriting operation in the recording area 24, the electronic device 100 displays at least one handwritten character 5 composed of hard pen lines corresponding to the handwriting operation.

[0225] It should be noted that the pen handwriting input state determines the display format of the lines corresponding to the user's current handwriting operation. Specifically, in pen handwriting input state, the electronic device 100 can only display preset pen lines for handwriting operations. For example, the default pen line setting in pen handwriting state can be a black pencil line with a thickness of 1 pixel. Then, when the user slides the stylus 4 in the recording area 24, the electronic device 100 will display the user's sliding trajectory based on this pencil line to generate the handwritten character 5.

[0226] If the user needs to adjust the current pen lines, the user needs to click the pen control 29 again to make the electronic device 100 display the pen settings window. In this way, the user can set the pen lines. After the settings are completed, the electronic device 100 can display the re-set pen lines.

[0227] It should also be noted that if a user makes a mistake during the handwriting operation, the user can use the erase control 31 to change the handwritten character 5.

[0228] This application embodiment is only an example of a user performing handwriting operations based on the pen control 29. For methods of operation based on other handwriting controls, please refer to this embodiment. This application will not elaborate further on the comparison.

[0229] It should be noted that in this embodiment, users typically perform handwriting operations based on the natural language writing order (hereinafter referred to as the first order). For example, users typically write from left to right (i.e., along the positive x-axis) on the first note interface 2, and after completing one line, they begin writing on the next line along the positive y-axis.

[0230] The following embodiments of this application are all illustrated by the scenario in which the user performs handwriting operations in the first order. In fact, this application also supports the typesetting of text with other types of language writing order. Other embodiments can be referred to the embodiments of this application, and this application will not elaborate on them.

[0231] In step S132, the electronic device 100 acquires at least one character rectangle based on at least one first character.

[0232] Each character rectangle corresponds to at least one first character. A character rectangle is the smallest bounding rectangle corresponding to a first character of a single character, or a character rectangle is the smallest bounding rectangle corresponding to multiple first characters of a single word.

[0233] Figure 11 This is the first interactive schematic diagram of the tag update method provided in the embodiments of this application.

[0234] Figure 12 This is a schematic diagram of a scenario where an electronic device acquires a character rectangle, as provided in an embodiment of this application.

[0235] like Figure 11 and Figure 12 As shown, in one implementation, step S132 includes steps S1321-S1323.

[0236] In step S1321, the first recognition engine obtains the handwriting data corresponding to the first character and recognizes the handwriting data to obtain at least one full recognition result, and then obtains the character rectangle based on the full recognition result.

[0237] One full recognition result corresponds to one first character used to represent a single character, or one full recognition result corresponds to multiple first characters used to represent a single word.

[0238] In this embodiment, the first recognition engine is, for example, a real-time recognition engine. The real-time recognition engine may include a text layout display component, a recognition manager, and the recognition engine itself.

[0239] The following further explains step S1321 based on the interaction process of each component in the first recognition engine.

[0240] In one implementation, step S1321 includes steps S1321a-S1321c.

[0241] In step S1321a, the text layout display component obtains the handwriting data corresponding to the first character and sends the handwriting data to the recognition manager.

[0242] In one implementation, the input manager can receive touch signals acquired by the touch sensor and convert the touch signals into standard MotionEvent objects.

[0243] MotionEvent is a class provided by Android that encapsulates specific information about touch events. A MotionEvent object is a concrete representation of the MotionEvent class. A MotionEvent object can encapsulate event type, touch point coordinates, pressure information, tilt angle information, multi-touch information, etc. The event types encapsulated in a MotionEvent object include Action_Down, Action_Move, and Action_Up events. The Action_Down event is used to indicate that the stylus is pressed on the display screen. The Action_Move event is used to indicate that the stylus is swiped on the display screen. The Action_Up event is used to indicate that the stylus is released from the display screen.

[0244] Furthermore, the input manager can send MotionEvent objects to the view processing module in the note-taking application. The view processing module can be a functional module associated with the handwriting layer, which can encapsulate the onTouchEvent method to receive and parse MotionEvent objects.

[0245] After parsing the MotionEvent object, the view processing module can record the coordinates of the touch point according to the type of touch event and update the path. The view processing module can further invoke the rendering thread to draw the updated path on the display screen. During this process, the specific content of the MotionEvent object parsed by the view processing module is the handwriting data corresponding to the first character. The view processing module can send the handwriting data corresponding to the first character to the text layout display component, so that the text layout display component can send the handwriting data to the recognition manager.

[0246] In one implementation, the view processing module can send handwriting data to the text layout display component once after obtaining a MotionEvent object including the Action_Up event. That is, the text layout display component can obtain handwriting data corresponding to one stroke at a time and send the handwriting data corresponding to each stroke to the recognition manager.

[0247] In step S1321b, the recognition manager calls the recognition engine to recognize the handwriting data in order to obtain the full recognition results.

[0248] The Recognition Manager is the main business class used for recognition. Based on the Recognition Manager, all recognition results can be maintained and integrated.

[0249] The recognition engine is a tool class for recognition, which can process recognition tasks in real time. Specifically, the recognition engine may include a stroke engine, a language resource library, and a recognition switch.

[0250] After receiving the handwriting data, the recognition manager may initiate a recognition task to the recognition engine. The recognition engine may respond to the recognition task based on the recognition switch, recognize the handwriting data based on the stroke engine, then match the recognized handwriting data with the language resource library to obtain a full recognition result, and further, the recognition engine sends the full recognition result to the recognition manager.

[0251] The full recognition result recognized by the recognition engine is specifically described below.

[0252] The character "Jin" is taken as an example for illustration, and the full recognition result corresponding to the character "Jin" is for example:

[0253] ("label": "Jin",

[0254] "line_box": [{"height": 99.40732, "x": 255.14925, "y": 263.9456, "width": 69.37886)],

[0255] "single_word_scope": [0, 1],

[0256] "type": "text",

[0257] "words": [{"bounding box": {"height": 99.40732, "x": 255.14925, "y": 263.9456, "width": 69.37886}, "candidates": ["Jin", "Ling", "Ling"], "label": "Jin"}])

[0258] Herein, "label": "Jin" indicates that the first recognized character is the character "Jin", and "x" and "y" in "line_box" represent the coordinates of the top-left vertex (i.e. Figure 12 point Q therein) of the rectangular area where the character "Jin" is located, "height" represents the height of the rectangular area where the character "Jin" is located, and "width" represents the width of the rectangular area where the character "Jin" is located.

[0259] "single_word_scope" indicates that the number of characters corresponding to the character "jin" is 1. It should be noted that when the recognition engine recognizes a word, for example, when recognizing "Hello", it will take "Hello" as a full recognition result, and the number of characters corresponding to this full recognition result is 5.

[0260] "type": "text" indicates that the character "jin" corresponds to the "text" type.

[0261] "bounding box" represents the minimum circumscribed rectangle corresponding to the character "jin", that is, the character rectangle. The character rectangle corresponds to the rectangular area where the character "jin" is located. "candidates": ["jin", "ling", "ling"] represents candidate recognition results, "label":

[0262] "jin" may be the final result selected from the candidate recognition results.

[0263] In this way, after obtaining the full recognition result, the recognition engine may send the full recognition result to the recognition manager, so that the recognition manager performs integration processing on the full recognition result.

[0264] In step S1321c, the recognition manager sends the full recognition result to the text layout display component.

[0265] In step S1322, the first recognition engine sends the full recognition result to the text layout module.

[0266] In one implementation, step S1322 includes step S1322a.

[0267] In step S1322a, the text layout display component sends the full recognition result to the text layout module.

[0268] It should be noted that step S1321 and step S1322 may be performed for a single character and a single word written by the user, that is, every time the user writes a character, for example, writes "jin", or writes a word, for example, writes "Hello", the first recognition engine can complete real-time acquisition and transmission of a full recognition result. Actually, the first recognition engine also supports performing step S1321 and step S1322 after the user writes a plurality of characters or a plurality of words. This is not limited in the embodiments of the present application.

[0269] In step S1323, the text layout module sorts the full recognition results based on a preset first order, so that the character rectangles have the first order.

[0270] It can be known from the foregoing content that the first order is a natural language writing order (or a natural language reading order).

[0271] Further as Figure 12 shown, in the case where the user's written content is "Today's sky is as blue as a wash", the text layout module will sort the character rectangles corresponding to each character such as "今", "天", "的", "天", "空", "湛", "蓝", "如", and "洗", so as to ensure that in the subsequent processing, after processing the character rectangle corresponding to the character "今", the character rectangle corresponding to the character "天", the character rectangle corresponding to the character "的", and the subsequent character rectangles will be processed sequentially.

[0272] In step S133, the electronic device 100 determines at least one line rectangle based on the at least one character rectangle.

[0273] Each line rectangle corresponds to one line in the first text.

[0274] In one implementation, step S133 includes step S1331.

[0275] In step S1331, the text layout module invokes the line layout module to sequentially perform splicing processing on the character rectangles with the first order, so as to determine the line rectangles.

[0276] The process in which the text layout module invokes the line layout module to perform splicing processing on the character rectangles is specifically described below.

[0277] In one implementation, step S1331 includes steps S1331a to S1331f.

[0278] In step S1331a, the line layout module creates a first line rectangle based on the first obtained first character rectangle, and determines a first line midline corresponding to the first line rectangle, where the first line midline passes through the center point of the first character rectangle and is parallel to the horizontal direction.

[0279] Figure 13 is a schematic diagram of the first scenario for the line layout module to perform splicing processing on character rectangles provided by the embodiments of the present application.

[0280] As Figure 13 shown, for example, the first character rectangle 42 first obtained by the line layout module may be the character rectangle corresponding to the character "今". Further, the line layout module creates the first line rectangle 43 based on the first character rectangle 42. Further, the line layout module may determine the first line midline L1 based on the connection line of the center point coordinate P1 of the first character rectangle 42, the midpoint P2 of the first side edge of the first character rectangle 42, and the midpoint P3 of the second side edge of the first character rectangle 42.

[0281] In step S1331b, the line layout module traverses the second character rectangles, and when traversing to the current second character rectangle, traverses all currently created line rectangles to determine whether the current second character rectangle intersects with any line midline corresponding to all the current line rectangles, wherein the second character rectangles include character rectangles obtained after the first character rectangle along the first order.

[0282] Figure 14 is a second schematic scenario diagram of splicing processing of character rectangles by the line layout module provided in the embodiments of the present application.

[0283] As shown in Figure 13 and Figure 14 , further based on the foregoing example, the line layout module may traverse the second character rectangle 44, and when traversing to the current second character rectangle 44 (for example, traversing to the character rectangle corresponding to the character "天"), traverse all currently created line rectangles (i.e., the first line rectangle 43) to determine whether the current second character rectangle 44 intersects with any line midline corresponding to all the current line rectangles.

[0284] In step S1331c, when the current second character rectangle intersects with a target line midline among the line midlines, the line layout module adds the current second character rectangle to the target line rectangle corresponding to the target line midline.

[0285] Further as shown in Figure 13 and Figure 14 , further based on the foregoing example, since the current second character rectangle 44 is the character rectangle corresponding to the character "天", all currently created line rectangles only include the first line rectangle 43, and the line midline corresponding to all the current line rectangles only includes the first line midline L1, the character rectangle corresponding to the character "天" intersects with the target line midline (i.e., the first line midline L1) among the line midlines, and the line layout module adds the character rectangle corresponding to the character "天" to the target line rectangle corresponding to the target line midline, that is, adds it to the first line rectangle 43.

[0286] In a similar manner, the line layout module may determine that the character rectangles corresponding to the characters "的", "天", "空", "湛", "蓝", "如", "洗", "几", "朵", "轻", "盈", "的", "白", "云", "悠", "闲", "地", "漂" and the like can be added to the first line rectangle 43.

[0287] In step S1331d, when the current second character rectangle does not intersect with any of the line midlines, the line layout module creates a new line rectangle based on the current second character rectangle, and traverses the next second character rectangle.

[0288] Figure 15 is a third schematic scenario diagram of splicing processing of character rectangles by the line layout module provided in the embodiments of the present application.

[0289] As Figures 13 to 15 shown, further based on the foregoing embodiment, when the line layout module traverses to the current second character rectangle 44 (for example, traverses to the character rectangle corresponding to the character "fu"), it traverses all currently created line rectangles (i.e., the first line rectangle 43), so as to determine whether the current second character rectangle 44 intersects with any line midline corresponding to all the current line rectangles.

[0290] Since the current second character rectangle 44 is the character rectangle corresponding to the character "fu", all currently created line rectangles only include the first line rectangle 43, and the line midline corresponding to all current line rectangles is only the first line midline L1. Therefore, the character rectangle corresponding to the character "fu" does not intersect with the first line midline L1. In this way, the line layout module creates a new line rectangle based on the current second character rectangle, for example, creates a first new line rectangle 45.

[0291] Figure 16 is a fourth schematic diagram of a scenario in which the line layout module performs splicing processing on character rectangles provided in the embodiments of the present application.

[0292] As Figure 16 shown, in a manner similar to that of the foregoing embodiment, the line layout module may continue to determine that the character rectangles corresponding to the characters "zai", "kong", "zhong", "fang", "fo", "shi", "yi", "fu", "jing", "xin", "hui", "zhi", "de", "shui", "cai", "hua", etc. can be added to the first new line rectangle 45.

[0293] In a manner similar to that of the foregoing embodiment, the line layout module may create a second new line rectangle 46 when traversing to the character rectangle corresponding to the character "yang".

[0294] Step S1331e: after the traversal of all second character rectangles is completed, the line layout module obtains all line rectangles.

[0295] Further as Figure 16 shown, after the line layout module traverses to the last character rectangle, for example, traverses to the character rectangle corresponding to the character "a", the line layout module may obtain the first line rectangle 43, the first new line rectangle 45, the second new line rectangle 46, the third new line rectangle 47, the fourth new line rectangle 48, and the fifth new line rectangle 49.

[0296] Step S1331f: the text layout module invokes the line layout module to determine the line rectangles.

[0297] The text layout module may obtain data corresponding to the line rectangles from the line layout module, for example, obtain data corresponding to the first line rectangle 43, the first new line rectangle 45, the second new line rectangle 46, the third new line rectangle 47, the fourth new line rectangle 48, and the fifth new line rectangle 49.

[0298] In step S134, the electronic device 100 determines at least one segment rectangle based on at least one row rectangle.

[0299] Each segment rectangle corresponds to a first paragraph.

[0300] In one implementation, step S134 includes step S1341.

[0301] Step S1341: The text layout module calls the segment layout module to splice the row rectangles to determine the segment rectangles.

[0302] The following is a detailed explanation of the process by which the text layout module calls the segment layout module to splice row rectangles.

[0303] In one implementation, step S1341 includes steps S1341a-S1346a.

[0304] Step S1341a: The segment layout module creates the first segment rectangle based on the first row rectangle.

[0305] Figure 17 This is a schematic diagram of the first scenario in which the segment layout module provided in this application splices row rectangles.

[0306] like Figure 17 As shown, based on the aforementioned example, the segment layout module can first obtain the first row rectangle 43, and then create the first segment rectangle 50 after obtaining the first row rectangle 43.

[0307] Step S1342a: Along the first direction, the segment layout module traverses the second row rectangles, and when traversing to the current second row rectangle, obtains the row spacing between the current second row rectangle and the previous row rectangle. Here, the first direction is the generation direction of the second row rectangle, and the second row rectangle includes the row rectangles created after the first row rectangle.

[0308] Figure 18 This is a second scenario diagram illustrating how the segment layout module provided in this application splices row rectangles.

[0309] like Figure 18 As shown, further based on the aforementioned example, the first second row rectangle is the first newly created row rectangle 45. When the segment layout module traverses along the positive y-axis to the first newly created row rectangle 45, it obtains the row spacing between the current second row rectangle and the previous row rectangle (i.e., the first row rectangle 43).

[0310] For example, the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is 15 pixels.

[0311] In step S1343a, if the line spacing is less than or equal to the line spacing threshold, and if the line spacing is less than or equal to the average line spacing, the segment layout module marks the current second row rectangle as belonging to the last created segment rectangle, and iterates through the next second row rectangle.

[0312] Further as Figure 18 As shown in the example, the preset line spacing threshold in the segment layout module is 25 pixels. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is less than the line spacing threshold.

[0313] It should be noted that the line spacing threshold provided in this application embodiment is only for illustrative purposes. The specific value of the line spacing threshold can be set according to the actual situation, and this application embodiment does not limit it.

[0314] Furthermore, since only the first second row rectangle has been traversed so far, the segment layout module can only obtain one line spacing. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is equal to the average line spacing.

[0315] In this way, the segment layout module can mark the current second row rectangle (i.e., the first newly created row rectangle 45) as belonging to the last created segment rectangle (i.e., the first segment rectangle 50), and the segment layout module can further traverse the next second row rectangle.

[0316] Figure 19 This is a schematic diagram of the third scenario where the segment layout module provided in this application splices row rectangles.

[0317] like Figure 19 As shown, further based on the aforementioned example, the next second row rectangle is the second newly created row rectangle 46. When the segment layout module traverses along the positive y-axis to the second newly created row rectangle 46, it obtains the row spacing between the current second row rectangle and the previous row rectangle (i.e., the first newly created row rectangle 45).

[0318] For example, the line spacing between the second new row rectangle 46 and the first new row rectangle 45 is 30 pixels.

[0319] In step S1344a, if the line spacing is greater than the preset line spacing threshold, or if the line spacing is greater than the average line spacing, the segment layout module creates a new segment rectangle, marks the current second row rectangle as belonging to the newly created segment rectangle, and then traverses the next second row rectangle.

[0320] Further as Figure 19As shown, since the preset line spacing threshold in the segment layout module is 25 pixels, the segment layout module can determine that the line spacing between the second newly created line rectangle 46 and the first newly created line rectangle 45 is greater than the line spacing threshold.

[0321] It should be noted that in step S1344a, the segment layout module can create a new segment rectangle when it determines that either the line spacing is greater than the line spacing threshold or the line spacing is greater than the average line spacing. If the segment layout module has determined that the line spacing meets the condition of being greater than the line spacing threshold, the segment layout module does not need to compare the line spacing with the average line spacing in order to reduce power consumption.

[0322] This application's embodiments are merely illustrative examples of how the segment layout module compares line spacing with a line spacing threshold. In fact, this application does not restrict the order in which the segment layout module executes any judgment condition.

[0323] In this way, the segment layout module can create new segment rectangles, such as creating the first newly created segment rectangle 51. Furthermore, the segment layout module can mark the current second row rectangle (i.e., the second newly created row rectangle 46) as belonging to the first newly created segment rectangle 51.

[0324] Figure 20 This is a schematic diagram of the fourth scenario in which the segment layout module provided in this application splices row rectangles.

[0325] like Figure 20 As shown, based on a similar approach to the aforementioned embodiments, the segment layout module can mark the current second row rectangle (i.e., the third newly created row rectangle 47) as belonging to the last created segment rectangle (i.e., the first newly created segment rectangle 51) when traversing to the third newly created row rectangle 47.

[0326] In step S1345a, after all the second row rectangles have been traversed, the segment layout module obtains all the segment rectangles.

[0327] Further as Figure 20 As shown, after the segment layout module traverses to the last row rectangle, for example, after traversing to the fifth newly created row rectangle 49, the segment layout module can obtain all the segment rectangles, that is, obtain the first segment rectangle 50, the first newly created segment rectangle 51, and the second newly created segment rectangle 52.

[0328] In step S1346a, the text layout module calls the segment layout module to determine the segment rectangle.

[0329] It should be noted that the implementation of steps S1341a-S1346a is based on the user's writing habits.

[0330] Figure 21This is the first schematic diagram of the first text obtained based on different writing habits provided in the embodiments of this application.

[0331] like Figure 21 As shown in A, users may have a habit of keeping large gaps between paragraphs. Specifically, when writing the first paragraph, users will keep a large gap between two adjacent paragraphs in order to emphasize the independence of the content of each first paragraph. This not only clearly divides the paragraphs but also improves the reading comfort of the first text.

[0332] like Figure 21 As shown in B, users may not have the habit of maintaining large spacing between paragraphs. Specifically, when writing the first paragraph, users may be accustomed to a more compact layout and will not maintain large spacing between adjacent paragraphs.

[0333] In steps S1341a-S1346a of this embodiment, differentiation can only be performed on the first text that retains a large spacing between paragraphs. Although the creation of segment rectangles can be achieved in most cases, it has certain limitations.

[0334] Therefore, in order to ensure the accuracy of paragraph division, the embodiments of this application also provide the following implementation methods.

[0335] In one implementation, step S1341 includes steps S1341b-S1346b.

[0336] In step S1341b, the segment layout module creates the first segment rectangle based on the first obtained first row rectangle.

[0337] The specific implementation of step S1341b can be found in the aforementioned step S1341a. This application's embodiments will not elaborate further on this aspect.

[0338] In step S1342b, along the first direction, the segment layout module traverses the second row rectangle, and when traversing to the current second row rectangle, obtains the row spacing between the current second row rectangle and the previous row rectangle, as well as obtains the first row height corresponding to the current second row rectangle and the second row height corresponding to the previous row rectangle.

[0339] Further as Figure 18As shown, based on the previous example, the first second-row rectangle is the first newly created row rectangle 45. When the segment layout module traverses along the positive y-axis to the first newly created row rectangle 45, it obtains the row spacing between the current second-row rectangle (i.e., the first newly created row rectangle 45) and the previous row rectangle (i.e., the first row rectangle 43). Furthermore, the segment layout module also obtains the first row height corresponding to the current second-row rectangle (i.e., the first newly created row rectangle 45) and the second row height corresponding to the previous row rectangle (i.e., the first row rectangle 43).

[0340] For example, the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is 15 pixels. The first row height corresponding to the first newly created row rectangle 45 is 90 pixels, and the second row height corresponding to the first row rectangle 43 is 100 pixels.

[0341] In step S1343b, when the line spacing is less than or equal to the line spacing threshold, the line spacing is less than the average line spacing, and the ratio of the first difference between the first line height and the second line height to the maximum value between the first line height and the second line height is less than or equal to the first ratio threshold, the segment layout module marks the current second row rectangle as belonging to the last created segment rectangle and traverses the next second row rectangle.

[0342] Further as Figure 18 As shown in the example, the preset line spacing threshold in the segment layout module is 25 pixels. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is less than the line spacing threshold.

[0343] Furthermore, since only the first second row rectangle has been traversed so far, the segment layout module can only obtain one line spacing. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is equal to the average line spacing.

[0344] Furthermore, for example, the first proportional threshold is 20%. Then, the first difference between the first row height and the second row height is the difference between 100 pixels and 90 pixels, which is 10 pixels. The maximum value between the first row height and the second row height is 100 pixels. Since the first difference accounts for 10% of the maximum value between the first row height and the second row height, the first difference is less than the first proportional threshold.

[0345] It should be noted that the first proportional threshold provided in this application embodiment is only for illustrative purposes. The specific value of the first proportional threshold can be set according to the actual situation, and this application embodiment does not limit it.

[0346] In this way, the segment layout module can mark the current second row rectangle (i.e., the first newly created row rectangle 45) as belonging to the last created segment rectangle (i.e., the first segment rectangle 50), and the segment layout module can further traverse the next second row rectangle.

[0347] Further as Figure 19 As shown, the next second row rectangle is the second newly created row rectangle 46. When the segment layout module traverses along the positive y-axis to the second newly created row rectangle 46, it obtains the row spacing between the current second row rectangle and the previous row rectangle (i.e., the first newly created row rectangle 45). Furthermore, the segment layout module also obtains the first row height corresponding to the current second row rectangle (i.e., the second newly created row rectangle 46) and the second row height corresponding to the previous row rectangle (i.e., the first newly created row rectangle 45).

[0348] For example, the line spacing between the second new row rectangle 46 and the first new row rectangle 45 is 30 pixels. The first row height corresponding to the second new row rectangle 46 is 100 pixels, and the first row height corresponding to the first new row rectangle 45 is 90 pixels.

[0349] In step S1344b, if the line spacing is greater than the line spacing threshold, or the line spacing is greater than the average line spacing, or the proportion of the first difference between the first line height and the second line height to the maximum value between the first line height and the second line height is greater than the first proportion threshold, the segment layout module creates a new segment rectangle, marks the current second line rectangle as belonging to the newly created segment rectangle, and then traverses the next second line rectangle.

[0350] Further as Figure 19 As shown, since the preset line spacing threshold in the segment layout module is 25 pixels, the segment layout module can determine that the line spacing between the second newly created line rectangle 46 and the first newly created line rectangle 45 is greater than the line spacing threshold.

[0351] It should be noted that in step S1344b, the segment layout module can create a new segment rectangle when it determines that the line spacing is greater than the line spacing threshold, or the line spacing is greater than the average line spacing, or the proportion of the first difference to the maximum value between the first row height and the second row height is greater than the first proportion threshold. If the segment layout module has determined that the line spacing is greater than the line spacing threshold, the segment layout module does not need to compare the line spacing with the average line spacing, nor does it need to compare the proportion of the first difference to the maximum value between the first row height and the second row height with the first proportion threshold, in order to reduce power consumption.

[0352] In this way, the segment layout module can create new segment rectangles, such as creating the first newly created segment rectangle 51. Furthermore, the segment layout module can mark the current second row rectangle (i.e., the second newly created row rectangle 46) as belonging to the first newly created segment rectangle 51.

[0353] In step S1345b, after all the second row rectangles have been traversed, the segment layout module obtains all the segment rectangles.

[0354] In step S1346b, the text layout module calls the segment layout module to determine the segment rectangle.

[0355] It should be noted that the implementation of steps S1341b-S1346b is set according to the user's writing habits.

[0356] Figure 22 This is a second schematic diagram of the first text obtained based on different writing habits provided in the embodiments of this application.

[0357] like Figure 22 As shown in A, if the first text is in English, users may have a habit of capitalizing the first character in a larger font size. Specifically, when writing the first text, users may capitalize the first character of the first paragraph in a larger font size to improve readability, thus enhancing the readability of the first text.

[0358] It should be noted that this embodiment is only illustrated using English text as the first text. In reality, if the first text is Chinese text, users may also have the habit of writing the first character in a larger font size at the beginning of a paragraph.

[0359] like Figure 22 As shown in B, users may also only be accustomed to using a regular font size with the first character capitalized.

[0360] If a user has a habit of capitalizing the first character in a larger font size, the line height of the line rectangle corresponding to the first line of the paragraph will be greater than the line height of the line rectangle corresponding to the inner lines of the paragraph. Based on this principle, the paragraph layout module can determine whether the current line rectangle belongs to a newly created paragraph rectangle.

[0361] In steps S1341b-S1345b of this embodiment, differentiation can be applied not only to first text that maintains a large spacing between paragraphs, but also to first text that begins with a large capital character in the first line of a paragraph. Nevertheless, this implementation still has certain limitations.

[0362] To ensure the accuracy of paragraph division, in one implementation, step S1341 can also be based on steps S1341c-S1346c.

[0363] In step S1341c, the segment layout module creates the first segment rectangle based on the first obtained first row rectangle.

[0364] The specific implementation of step S1341c can be found in the aforementioned step S1341a. This application's embodiments will not elaborate further on this aspect.

[0365] In step S1342c, along the first direction, the segment layout module traverses the second row rectangles, and when traversing to the current second row rectangle, it obtains the row spacing between the current second row rectangle and the previous row rectangle, as well as the first row height corresponding to the current second row rectangle, the second row height corresponding to the previous row rectangle, the first start coordinate and the first end coordinate corresponding to the current second row rectangle, and the second start coordinate and the second end coordinate corresponding to the previous row rectangle.

[0366] Further as Figure 18 As shown, based on the previous example, the first second-row rectangle is the first newly created row rectangle 45. When the segment layout module traverses along the positive y-axis to the first newly created row rectangle 45, it obtains the row spacing between the current second-row rectangle (i.e., the first newly created row rectangle 45) and the previous row rectangle (i.e., the first row rectangle 43). Furthermore, the segment layout module also obtains the first row height corresponding to the current second-row rectangle (i.e., the first newly created row rectangle 45), and the second row height corresponding to the previous row rectangle (i.e., the first row rectangle 43). Additionally, the segment layout module also obtains the first start coordinate and the first end coordinate corresponding to the current second-row rectangle (i.e., the first newly created row rectangle 45), and the second start coordinate and the second end coordinate corresponding to the previous row rectangle (i.e., the first row rectangle 43).

[0367] For example, the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is 15 pixels. The first row height corresponding to the first newly created row rectangle 45 is 90 pixels, and the second row height corresponding to the first row rectangle 43 is 100 pixels. The first starting coordinates corresponding to the first newly created row rectangle 45 are (100, 190), and the first ending coordinates corresponding to the first newly created row rectangle 45 are (1230, 190). The second starting coordinates corresponding to the first row rectangle 43 are (250, 265), and the second ending coordinates corresponding to the first row rectangle 43 are (1600, 265).

[0368] It should be noted that in this embodiment, the first starting coordinate can be the coordinate corresponding to the upper left corner of the current second row rectangle, the coordinate corresponding to the lower left corner of the current second row rectangle, or the coordinate corresponding to the midpoint of the left boundary of the current second row rectangle. Similarly, the first ending coordinate can be the coordinate corresponding to the upper right corner of the current second row rectangle, the coordinate corresponding to the lower right corner of the current second row rectangle, or the coordinate corresponding to the midpoint of the right boundary of the current second row rectangle. The setting method for the second starting coordinate and the second ending coordinate is similar. This embodiment will not elaborate further on this. In the implementation process of this embodiment, it is only necessary to ensure that the first starting coordinate, the first ending coordinate, the second starting coordinate, and the second ending coordinate are compared based on their corresponding positions.

[0369] In step S1343c, when the line spacing is less than or equal to the line spacing threshold, the line spacing is less than the average line spacing, the proportion of the first difference to the maximum value between the first line height and the second line height is less than or equal to the first proportion threshold, the first distance between the horizontal coordinates in the first starting coordinates and the horizontal coordinates in the second starting coordinates is less than or equal to the first distance threshold, the second distance between the horizontal coordinates in the second ending coordinates and the segment end boundary is less than or equal to the second distance threshold, and the third distance between the vertical coordinates in the first ending coordinates and the vertical coordinates in the second ending coordinates is less than or equal to the third distance threshold, the segment layout module marks the current second row rectangle as belonging to the last created segment rectangle and traverses the next second row rectangle. The segment end boundary is determined based on the end coordinates corresponding to all currently created segment rectangles.

[0370] Further as Figure 18 As shown in the example, the preset line spacing threshold in the segment layout module is 25 pixels. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is less than the line spacing threshold.

[0371] Furthermore, since only the first second row rectangle has been traversed so far, the segment layout module can only obtain one line spacing. Therefore, the segment layout module can determine that the line spacing between the first row rectangle 43 and the first newly created row rectangle 45 is equal to the average line spacing.

[0372] Furthermore, for example, the first proportional threshold is 20%. Then, the first difference between the first row height and the second row height is the difference between 100 pixels and 90 pixels, which is 10 pixels. The maximum value between the first row height and the second row height is 100 pixels. Since the first difference accounts for 10% of the maximum value between the first row height and the second row height, the first difference is less than the first proportional threshold.

[0373] Furthermore, for example, the first distance threshold can be three times the average row height; for instance, if the average row height is 100 pixels, the first distance threshold can be 300 pixels. For example, the second distance threshold can be 100 pixels. Then, the segment layout module can determine that the first distance between the horizontal coordinates in the first and second starting coordinates is the difference between 250 pixels and 100 pixels, which is 150 pixels, and thus determine that the first distance is less than the first distance threshold.

[0374] Furthermore, since only the first second-row rectangle has been traversed so far, the segment layout module can only obtain one end coordinate, namely the second end coordinate. Therefore, the segment layout module can determine that the segment end boundary is the line corresponding to x = 1600. The second distance between the x-coordinate in the second end coordinate and the segment end boundary is the difference between 1600 pixels and 1600 pixels, which is 0 pixels. Thus, the segment layout module can determine that the second distance is less than the second distance threshold.

[0375] Furthermore, for example, the third distance threshold is the distance of the average row height, which is, for example, 100 pixels. Then, the segment layout module can determine that the third distance between the ordinate in the first end coordinate and the ordinate in the second end coordinate is the difference between 265 pixels and 190 pixels, which is 75 pixels, and thus determine that the third distance is less than the third distance threshold.

[0376] In this way, the segment layout module can mark the current second row rectangle (i.e., the first newly created row rectangle 45) as belonging to the last created segment rectangle (i.e., the first segment rectangle 50), and the segment layout module can further traverse the next second row rectangle.

[0377] It should be noted that the first distance threshold, the second distance threshold, and the third distance threshold provided in this application embodiment are only for illustrative purposes. The specific values ​​of the first distance threshold, the second distance threshold, and the third distance threshold can be set according to the actual situation, and this application embodiment does not limit them.

[0378] Further as Figure 19As shown, the next second row rectangle is the second newly created row rectangle 46. When the segment layout module traverses along the positive y-axis to the second newly created row rectangle 46, it obtains the line spacing between the current second row rectangle and the previous row rectangle (i.e., the first newly created row rectangle 45). Furthermore, the segment layout module also obtains the first row height corresponding to the current second row rectangle (i.e., the second newly created row rectangle 46), and the second row height corresponding to the previous row rectangle (i.e., the first newly created row rectangle 45). Additionally, the segment layout module also obtains the first start coordinates and the first end coordinates corresponding to the current second row rectangle (i.e., the second newly created row rectangle 46), and the second start coordinates and the second end coordinates corresponding to the previous row rectangle (i.e., the first newly created row rectangle 45).

[0379] For example, the line spacing between the second newly created row rectangle 46 and the first newly created row rectangle 45 is 30 pixels. The first row height corresponding to the second newly created row rectangle 46 is 100 pixels, and the first row height corresponding to the first newly created row rectangle 45 is 90 pixels. The first starting coordinates corresponding to the second newly created row rectangle 46 are (250, 110), and the first ending coordinates corresponding to the second newly created row rectangle 46 are (1600, 110). The second starting coordinates corresponding to the first newly created row rectangle 45 are (100, 190), and the second ending coordinates corresponding to the first newly created row rectangle 45 are (1230, 190).

[0380] In step S1344c, if the line spacing is greater than the line spacing threshold, or the line spacing is greater than the average line spacing, or the proportion of the first difference to the maximum value between the first row height and the second row height is greater than the first proportion threshold, or the first distance between the horizontal coordinates in the first starting coordinates and the horizontal coordinates in the second starting coordinates is greater than the first distance threshold, or the second distance between the horizontal coordinates in the first ending coordinates and the end boundary of the segment is greater than the second distance threshold, or the third distance between the vertical coordinates in the first ending coordinates and the vertical coordinates in the second ending coordinates is greater than the third distance threshold, the segment layout module creates a new segment rectangle, marks the current second row rectangle as belonging to the newly created segment rectangle, and then traverses the next second row rectangle.

[0381] Further as Figure 19 As shown, since the end boundary of a segment can be the straight line corresponding to x = 1600, the second distance between the second end coordinate and the end boundary of the segment is the difference between 1600 pixels and 1230 pixels, which is 370 pixels. Therefore, the segment layout module can determine that the second distance is greater than the second distance threshold.

[0382] It should be noted that in step S1344, the segment layout module can create a new segment rectangle if any of the following conditions are met: the line spacing is greater than the line spacing threshold; the line spacing is greater than the average line spacing; the ratio of the first difference to the maximum value between the first and second line heights is greater than the first ratio threshold; the first distance between the horizontal coordinates of the first and second starting coordinates is greater than the first distance threshold; the second distance between the horizontal coordinate of the first and second ending coordinates and the segment end boundary is greater than the second distance threshold; or the third distance between the vertical coordinates of the first and second ending coordinates is greater than the third distance threshold. If the segment layout module has determined that the second distance is greater than the second distance threshold, the segment layout module does not need to judge other conditions to reduce power consumption.

[0383] In this way, the segment layout module can create new segment rectangles, such as creating the first newly created segment rectangle 51. Furthermore, the segment layout module can mark the current second row rectangle (i.e., the second newly created row rectangle 46) as belonging to the first newly created segment rectangle 51.

[0384] In step S1345c, after all the second row rectangles have been traversed, the segment layout module obtains all the segment rectangles.

[0385] In step S1346c, the text layout module calls the segment layout module to determine the segment rectangle.

[0386] It should be noted that the implementation of steps S1341c-S1346c is set according to the user's writing habits.

[0387] Further as Figures 18 to 22 As shown, regardless of whether the first text is Chinese or English, users are likely to have the habit of first-line indentation and paragraph margins. Therefore, if the first distance between the current second-line rectangle and the previous-line rectangle is greater than a first distance threshold, it indicates that the current second-line rectangle belongs to the beginning of a paragraph with first-line indentation, and the paragraph layout module can determine that the current rectangle belongs to a newly created paragraph rectangle. If the second distance between the previous-line rectangle and the paragraph end boundary is greater than a second distance threshold, it indicates that the previous-line rectangle belongs to the end of a paragraph with paragraph margins, and the paragraph layout module can determine that the current rectangle belongs to a newly created paragraph rectangle.

[0388] The paragraph layout module divides the first paragraph based on the implementation of steps S1341c-S1346c above, further ensuring the accuracy of paragraph division.

[0389] It should be noted that, based on different user habits, step S1341 in this embodiment can be implemented based on steps S1341a-S1346a, steps S1341b-S1346b, or steps S1341c-S1346c. Step S1341 can also be implemented based on more derivative embodiments. This embodiment does not limit the specific implementation of step S1341.

[0390] It should also be noted that the order in which the sub-steps in step S134 are executed is not limited in the embodiments of this application.

[0391] In summary, the text layout module can obtain the data corresponding to the character rectangle, line rectangle, and segment rectangle.

[0392] It should be noted that, in this embodiment, the rectangular area corresponding to the row rectangle can be determined based on the height of its corresponding first character rectangle and the distance between its corresponding first and last character rectangles. The rectangular area corresponding to the segment rectangle can be determined by the top boundary of its corresponding first row rectangle and the bottom boundary of its corresponding last row rectangle. In fact, the rectangular areas corresponding to the row rectangle and the segment rectangle can also be appropriately expanded based on the above-mentioned areas. The schematic representations of the character rectangles, row rectangles, and segment rectangles shown in the accompanying drawings of this application do not constitute a limitation on the specific area range of the character rectangles, row rectangles, and segment rectangles. This embodiment does not limit the specific area range corresponding to the row rectangles and segment rectangles.

[0393] In step S135, when the first text is displayed, the electronic device 100 divides the first character based on at least one character rectangle, divides the lines in the first text based on line rectangles, and divides the first paragraph based on segment rectangles.

[0394] In step S1351, the paragraph layout module sends the first typesetting result to the first recognition engine so that the first recognition engine can complete the division of the first character based on the character rectangle, and complete the division of the lines in the first text based on the line rectangle, and complete the division of the first paragraph based on the paragraph rectangle.

[0395] The first typesetting result includes character rectangles, line rectangles, and segment rectangles.

[0396] It should be noted that segment data can include line data and segment rectangles, line data can include word data and line rectangles, and word data can include word rectangles. Therefore, after determining the segment rectangle, the segment layout module can actually determine the line rectangle and word rectangle at the same time. That is, the segment layout module can generate the first layout result.

[0397] In one implementation, step S1351 includes step S1351a.

[0398] In step S1351a, the paragraph layout module sends the first typesetting result to the text layout display component, so that the text layout display component completes the division of the first character based on the character rectangle, and completes the division of the lines in the first text based on the line rectangle, and completes the division of the first paragraph based on the paragraph rectangle.

[0399] In this way, the text layout display component can obtain the data corresponding to the character rectangle, line rectangle, and segment rectangle respectively.

[0400] In step S14, in response to a marking operation performed on at least one first target character among the first characters, the electronic device 100 displays a first mark on the first target character.

[0401] Figure 23 This is the second interactive schematic diagram of the tag update method provided in the embodiments of this application.

[0402] like Figure 23 As shown, in one implementation, step S14 includes steps S141-S144.

[0403] In step S141, in response to a click operation on the marking control in the first interface, the electronic device 100 enters the marking input state, and in response to a marking operation on at least one first target character among the first characters, the electronic device 100 acquires the first marking trajectory.

[0404] In one implementation, step S141 includes step S1411.

[0405] In step S1411, in response to the marking operation performed on the first target character, the view processing module obtains the first marking trajectory.

[0406] Further based on the foregoing example, when the first interface is the first note interface 2, the marking controls may include, for example, a highlighter control 30, a marker control, etc. This application embodiment only uses the highlighter control 30 as an example for illustration; the specific control type of the marking control is not limited in this application embodiment.

[0407] For details on how users perform marking operations in this embodiment, please refer to [link / reference needed]. Figure 2 ,exist Figure 2 In the scenario shown, the first target character includes the handwritten character 5 within the K1 area, and the first mark is, for example, a highlighter mark 7 composed of highlighter lines.

[0408] In this embodiment of the application, the first marker trajectory obtained by the electronic device 100 may be a path formed by multiple touch points obtained by the view processing module parsing the MotionEvent object.

[0409] Specifically, Figure 2 In the scenario shown, the user slides across multiple handwritten characters 5 within the K1 area, causing the input sensor to receive touch signals acquired by the touch sensor and convert these signals into MotionEvent objects. The input manager can then send these MotionEvent objects to the view processing module. Furthermore, after parsing the MotionEvent objects, the view processing module can record the coordinates of the touch point based on the type of touch event and update the path.

[0410] In step S142, the electronic device 100 displays a first mark on the first target character based on the first mark trajectory.

[0411] Figure 24 This is a schematic diagram illustrating the formation method of the first mark provided in the embodiments of this application.

[0412] like Figure 24 As shown, after the view processing module obtains multiple touch points corresponding to the first mark trajectory, it can generate rectangular units for each touch point, using the touch point as the center of a rectangle and based on a preset rectangle length and width. In this way, multiple connected rectangular units can form a highlighter mark effect. The view processing module sends multiple rectangular units to the display via a rendering thread, thus enabling the display screen to show the first mark (e.g., highlighter mark 7).

[0413] In one implementation, step S141 is followed by steps S143-S144.

[0414] In step S143, the electronic device determines at least one first target character rectangle corresponding to the first marker trajectory, wherein one first target character rectangle corresponds to one first target character.

[0415] In one implementation, step S143 includes steps S1431-S1432.

[0416] In step S1431, the view processing module sends the first marker trajectory to the first recognition engine.

[0417] In one implementation, step S1431 includes step S1431a.

[0418] In step S1431a, the view processing module sends the first marker trajectory to the text layout display component.

[0419] In step S1432, the first recognition engine determines at least one first target character rectangle corresponding to the first marked trajectory.

[0420] In an implementation, step S1432 includes step S1432a.

[0421] In step S1432a, the text layout display component determines at least one first target character rectangle corresponding to the first marked trajectory.

[0422] Specifically, since the text layout display component has acquired the first typesetting result in the foregoing process, and the first typesetting result includes character rectangles, after acquiring the first marked trajectory, the text layout display component may determine the character rectangles intersecting with the first marked trajectory based on the position information corresponding to the first marked trajectory and the position information corresponding to the character rectangles, and further obtain the first target character rectangle.

[0423] Figure 25 is a schematic diagram of a scenario in which the electronic device provided in the embodiments of the present application determines a first target character rectangle.

[0424] As shown in Figure 25 , further based on the foregoing example, when the first mark is the highlighter mark 7, the text layout display component may determine that the first target character rectangles are the character rectangle corresponding to the character "精", the character rectangle corresponding to the character "心", the character rectangle corresponding to the character "绘", and the character rectangle corresponding to the character "制".

[0425] In step S144, the electronic device 100 sets the first flag bit in the first character data structure corresponding to each first target character rectangle as a valid flag bit, so as to identify, based on the valid flag bit, that the first target character corresponds to the first marked trajectory, where the first character data structure includes text information corresponding to the first target character, the first target character rectangle, and the first flag bit.

[0426] In an implementation, step S144 includes step S1441.

[0427] In step S1441, the first recognition engine sets the first flag bit in each first character data structure as a valid flag bit, so as to identify, based on the valid flag bit, that the first target character corresponds to the first marked trajectory.

[0428] In an implementation, step S1441 includes step S1441a.

[0429] In step S1441a, the text layout display component sets the first flag bit in each first character data structure as a valid flag bit, so as to identify, based on the valid flag bit, that the first target character corresponds to the first marked trajectory.

[0430] Figure 26is a schematic diagram of a data storage structure provided by an embodiment of the present application.

[0431] As Figure 26 shown in the figure, the character data structure includes text information (Text) corresponding to the first character, a character rectangle (BoundingBox), and a flag bit (markFlag).

[0432] It should be noted here that the character data structure may also include other data, which is not limited in the present application.

[0433] In an implementation manner, the manner for the text layout display component to set the first flag bit as a valid flag bit in each first character data structure may be setting the first flag bit to True.

[0434] Correspondingly, the text layout display component may set the first flag bit as an invalid flag bit in the character data structures corresponding to other character rectangles that do not correspond to the first marking track, and the specific manner may be setting the first flag bit to False.

[0435] Further based on the foregoing example, in a case where the first target character rectangles are the character rectangles corresponding to the characters "精", "心", "绘", and "制", the text layout display component sets the first flag bit to True in the first character data structures corresponding to these character rectangles.

[0436] It should be noted here that the embodiments of the present application do not limit the sequence of execution of the sub-steps in step S14. For example, step S142 may be executed before step S143, may be executed synchronously with step S143, or may be executed after step S143.

[0437] Step S15: In response to a deletion operation on a second target character in the first characters in the target paragraph of the first paragraph, the electronic device 100 determines at least one first target line rectangle.

[0438] The first target line rectangle includes at least one line rectangle where the first part of characters is located, and at least one line rectangle corresponding to the deletion operation. The first part of characters includes at least a part of the first characters in the target paragraph except the second target character.

[0439] In an implementation manner, step S15 includes steps S151 to S152.

[0440] Step S151: In response to a deletion operation on a second target character in the first characters in the target paragraph of the first paragraph, the electronic device 100 acquires a deletion track corresponding to the deletion operation, and removes the second target character corresponding to the deletion track, so as to complete the deletion operation.

[0441] In one implementation, step S151 includes step S1511.

[0442] In step S1511, in response to the deletion operation of the second target character in the first character in the target paragraph of the first paragraph, the view processing module obtains the deletion trajectory corresponding to the deletion operation and removes the second target character corresponding to the deletion trajectory to complete the deletion operation.

[0443] The following is an example of how a user can delete a second target character.

[0444] Figure 27 This is a schematic diagram of the first scenario provided by the embodiments of this application, in which a user performs a deletion operation on the first interface.

[0445] like Figure 27 As shown, further based on the aforementioned example, when the electronic device 100 displays at least one handwritten character 5 corresponding to the handwriting operation, and the first target character has been marked, the user can click on the erase control 31. In response to the click on the erase control 31, the electronic device 100 enters the erase control mode.

[0446] In erase control mode, the user can perform a first swipe operation on the second target character (i.e., the handwritten character 5 in areas A1 and A2) using the stylus 4. In response to the first swipe operation on the second target character using the stylus 4, the input manager obtains the first deletion trajectory corresponding to the first swipe operation. The input manager sends the first deletion trajectory to the view processing module, which removes the handwritten character 5 corresponding to the first deletion trajectory to complete the deletion operation.

[0447] It should be noted that the first deletion trajectory obtained by the view processing module can be a path formed by multiple touch points obtained by parsing the MotionEvent object.

[0448] Figure 28 This is a schematic diagram of a second scenario where a user performs a deletion operation on the first interface, as provided in the embodiments of this application.

[0449] like Figure 28As shown, further based on the aforementioned example, when the electronic device 100 displays at least one handwritten character 5 corresponding to the handwriting operation, and the first target character has been marked, the user can perform a second sliding operation on the second target character using the stylus 4. The second sliding operation can be an operation with a wavy sliding pattern. In response to the second sliding operation of the stylus 4 on the target character, the input manager obtains the second deletion trajectory corresponding to the second sliding operation. The second deletion trajectory is, for example, a wavy trajectory. The input manager sends the second deletion trajectory to the view processing module, and the view processing module removes the handwritten character 5 corresponding to the second deletion trajectory to complete the deletion operation.

[0450] In step S152, after the electronic device 100 completes the deletion operation, it determines at least one first target row rectangle.

[0451] In one implementation, step S152 includes steps S1521-S1523.

[0452] Step S1521: The view processing module sends the first query information to the first recognition engine.

[0453] In one implementation, step S1521 includes step S1521a.

[0454] In step S1521a, the view processing module sends the first query information to the text layout display component.

[0455] In step S1522, in response to the first query information, the first recognition engine sends the first target row rectangle to the view processing module.

[0456] In one implementation, step S1522 includes step S1522a.

[0457] In step S1522a, in response to the first query information, the text layout display component sends the first target row rectangle to the view processing module.

[0458] It should be noted that the electronic device 100 needs to determine the first target row rectangle in order to delete the first marker in the first target row rectangle before moving the first part of the characters, so as to avoid a situation where the first target character and the first marker do not correspond during the automatic typesetting process. The first part of the characters and the first target row rectangle corresponding to the first part of the characters are explained below.

[0459] Figure 29 This is a schematic diagram of the first part of the characters provided in the embodiments of this application.

[0460] like Figure 29 As shown, the first part of the characters includes the handwritten character 5 within the shaded area 53.

[0461] Figure 30 This is a schematic diagram of a scenario in which the electronic device provided in this application determines the first target row rectangle.

[0462] like Figure 30 As shown, based on the previous example, since the row rectangle where the first part of the characters is located is the first newly created row rectangle 45, the first target row rectangle can be the first newly created row rectangle 45.

[0463] It should be noted here that, in addition to the row rectangle corresponding to the first part of the characters being the first target row rectangle, the row rectangle corresponding to the deletion operation is also the first target row rectangle; that is, the first target row rectangle also includes the first row rectangle 43. The row rectangle corresponding to the deletion operation can be used in the subsequent process of restoring the first mark. In step S16, the electronic device 100 deletes the first mark in the first target row rectangle.

[0464] In one implementation, step S16 includes step S161.

[0465] In step S161, the view processing module deletes the first marker in the first target row rectangle.

[0466] Figure 31 This is a schematic diagram of a scenario where an electronic device deletes a first marker, as provided in an embodiment of this application.

[0467] like Figure 30 and Figure 31 As shown, the view processing module can delete the highlighter mark 7 in the first newly created row rectangle 45.

[0468] In step S17, after deleting the first mark, the electronic device 100 moves the position of the first part of the characters to fill the blank area formed by deleting the second target character.

[0469] Figure 32 This is the third interactive schematic diagram of the tag update method provided in the embodiments of this application.

[0470] like Figure 32 As shown, in one implementation, step S17 includes steps S171-S178.

[0471] In step S171, the electronic device 100 triggers a reflow process based on the deletion operation.

[0472] It should be noted that in this embodiment of the application, the electronic device 100 can respond to the user's deletion operation in the area where a target paragraph is located, delete the first mark, and execute steps S171-S178 to achieve automatic layout. Alternatively, after the user performs deletion operations in the areas corresponding to multiple target paragraphs, it can respond to the user's operation again, delete the first mark, and execute steps S171-S178.

[0473] Further as Figure 30 As shown, in one implementation, the user deletes the handwritten character 5 in the A1 and A2 areas of the first paragraph. In response to the deletion operation, the electronic device 100 deletes the first mark and executes steps S171-S178 to automatically format the first part of the characters.

[0474] In another implementation, after deleting the handwritten character 5 in areas A1 and A2 of the first paragraph, the user can also delete the handwritten character 5 in area B1 of the second paragraph. In this way, after the user completes all deletion operations in the first text, the user can perform various user operations such as gesture operations and clicking on the automatic layout control, so that the electronic device 100 responds to the user's operation to delete the first mark and executes steps S171-S178. This implementation achieves unified automatic layout of the first text in the first interface and enhances the user experience.

[0475] The embodiments of this application do not limit the specific manner in which the electronic device 100 performs steps S171 to S178.

[0476] This application embodiment is only illustrated by way of example, in which electronic device 100 responds to deletion operation, deletes the first mark, and executes steps S171-S178.

[0477] In one implementation, step S171 includes steps S1711-S1713.

[0478] In step S1711, after completing the deletion operation, the view processing module sends a set of reflow instructions to the text layout display component.

[0479] In step S1712, upon receiving the first layout result, the text layout display component obtains the deletion trajectory corresponding to the blank area based on the re-layout instruction set.

[0480] In step S1713, the text layout display component sends the deletion trajectory to the text layout module.

[0481] In step S172, when a re-layout process is triggered, the electronic device 100 determines the target paragraph corresponding to the blank area formed by deleting the target character.

[0482] In one implementation, step S172 includes step S1721.

[0483] In step S1721, the text layout module determines the segment rectangle corresponding to the deletion trajectory as the target segment rectangle corresponding to the target paragraph.

[0484] Figure 33 This is a schematic diagram of a scenario where the text layout module provided in this application determines the target segment rectangle.

[0485] like Figure 33 As shown, since the text layout module can obtain the segment rectangle by calling the segment layout module, the text layout module can match the position information of the deletion trajectory with the position information of the segment rectangle after obtaining the deletion trajectory, so as to determine the target segment rectangle corresponding to the target segment.

[0486] Further, based on the aforementioned example, the deletion trajectory is used as the second deletion trajectory for illustrative purposes.

[0487] After obtaining the position information of the second deletion trajectory, the text layout module can determine that the position information of the second deletion trajectory matches the position information of the first rectangle 50. In this way, the text layout module can determine that the first rectangle 50 is the target rectangle.

[0488] In step S173, within the target paragraph, along the first direction, the electronic device 100 traverses the row rectangles to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position. The starting row rectangle is the first row rectangle obtained with at least a portion of blank area, the ending row rectangle is the next row rectangle obtained with at least a portion of blank area, the first offset region corresponds to the target blank area corresponding to the starting row rectangle, and the first index position corresponds to the starting position of the first character rectangle in the ending row rectangle.

[0489] In one implementation, step S173 includes steps S1731-S1732.

[0490] In step S1731, the text layout module sends a first move instruction to the segment layout module, so that the segment layout module performs an offset operation on the row rectangles in each target segment rectangle based on the first move instruction.

[0491] The first movement instruction may include the location information for deleting the trajectory.

[0492] In step S1732, in response to the first move instruction, the segment layout module traverses the row rectangles along the first direction in each target segment rectangle to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position, and sends a second move instruction to the row layout module so that the row layout module performs an offset operation on the word rectangles in each row rectangle based on the second move instruction.

[0493] In one implementation, the segment layout module can first define the segment range for each target segment rectangle, and then execute step S1732 within the defined segment range.

[0494] Figure 34 This is a schematic diagram of a scenario in which the segment layout module provided in this application determines the starting row rectangle, the ending row rectangle, the first offset region, and the first index position.

[0495] like Figure 34 As shown, based on the aforementioned example, when the target segment rectangle is the first segment rectangle 50, the segment layout module can first determine the first row rectangle 43 as the starting row rectangle when traversing the row rectangles along the positive y-axis.

[0496] Specifically, the segment layout module can determine the first row rectangle 43 as the starting row rectangle based on the character rectangle corresponding to the second deletion trajectory. Furthermore, the segment layout module can determine the first row rectangle 43 as the starting row rectangle by defining the first row rectangle 43 as startRow.

[0497] Furthermore, the segment layout module can determine the first new row rectangle 45 as the end row rectangle based on the character rectangle corresponding to the second deletion trajectory. Specifically, the segment layout module can determine the first new row rectangle 45 as the end row rectangle by defining the first new row rectangle 43 as endRow.

[0498] It should be noted that, since this embodiment describes the method in which the electronic device 100 executes steps S171-S178 in response to a deletion operation, the segment layout module can define the first newly created row rectangle 45 as endRow. When the electronic device 100 executes steps S171-S178 in response to a user operation, the segment layout module can also define the first newly created row rectangle 45 as nextRow. This application embodiment does not limit the specific method by which the segment layout module defines the row rectangle.

[0499] Further, the segment layout module may acquire a first start position corresponding to the target blank area and a first offset corresponding to the target blank area, determine a start point based on a center point corresponding to the first start position, determine an end point based on a row midline corresponding to the start row rectangle and the first offset, and further determine a first offset area based on the start point and the end point. The target blank area is an area corresponding to a deletion trajectory.

[0500] Specifically, the segment layout module may acquire the first start position of the area where the second deletion trajectory is located (i.e., the start position of the A1 area), and define the first start position as range.left. Further, the segment layout module may also acquire the first offset of the area where the second deletion trajectory is located (i.e., the width of the A1 area), and define the first offset as startPadding. In this way, the segment layout module may determine a start point (for example, the center point W1 corresponding to the left boundary of the A1 area) based on the center point corresponding to the first start position, determine an end point (for example, the center point W2 corresponding to the right boundary of the A1 area) based on the row midline corresponding to the start row rectangle (i.e., the first row midline L1) and the first offset, and further obtain the first offset area (for example, the A1 area).

[0501] Further, the segment layout module may determine the start position of the first character rectangle in the end row rectangle (i.e., the first newly created row rectangle 45) (i.e., the start position of the character rectangle corresponding to the character "仿") as the first index position, and define the first index position as endIndex.

[0502] In step S174, in the start row rectangle, the electronic device 100 traverses the character rectangles, and performs an offset operation on the character rectangles located after the first offset area, so as to move the character rectangles to the first offset area, and update the first offset area.

[0503] In an implementation, step S174 includes step S1741.

[0504] In step S1741, in response to a second movement instruction, the row layout module traverses the character rectangles in the start row rectangle, and performs an offset operation on the character rectangles located after the first offset area, so as to move the character rectangles to the first offset area, and update the first offset area.

[0505] Further based on the foregoing example, since there is no character rectangle after the first offset area (i.e., the A1 area) in this embodiment, the row layout module does not perform the offset operation after traversing the character rectangles in the start row rectangle.

[0506] Step S175: In the end line rectangle, the electronic device 100 traverses the word rectangles starting from the first index position, and performs an offset operation on each traversed word rectangle to move the word rectangle to the updated first offset region.

[0507] In an implementation, step S175 includes step S1751.

[0508] Step S1751: The line layout module traverses the word rectangles in the end line rectangle starting from the first index position, and performs an offset operation on each traversed word rectangle to move the word rectangle to the updated first offset region.

[0509] Figure 35 is a schematic diagram of a scenario where the line layout module performs an offset operation provided in the embodiments of the present application.

[0510] As shown in Figure 35 , further based on the foregoing example, the line layout module traverses the word rectangles starting from the first index position in the first newly created line rectangle 45, and may first traverse the word rectangle corresponding to the character "Fang". In this way, the line layout module may move the word rectangle corresponding to the character "Fang" to the A1 region.

[0511] The line layout module may specifically move the word rectangle corresponding to the character "Fang" to the A1 region according to the position of range.left and the position of the first line midline L1. Moreover, after the line layout module moves one word rectangle each time, the position of range.left may be updated.

[0512] In a similar manner, the line layout module sequentially traverses the word rectangles respectively corresponding to the characters "Fo", "Shi", "Yi", "Zhong", "Jing", "Xin", "Hui", "Zhi", "De", "Shui", "Cai", "Hua" and the like in the first newly created line rectangle 45.

[0513] Step S176: When the traversal of the word rectangles in the end line rectangle is completed, or the word rectangles in the end line rectangle have completely filled the first offset region, the electronic device 100 ends the offset operation.

[0514] In an implementation, step S176 includes step S1761.

[0515] Step S1761: When the traversal of the word rectangles in the end line rectangle is completed, or the word rectangles in the end line rectangle have completely filled the first offset region, the line layout module ends the offset operation, and sends a third movement instruction to the paragraph layout module, where the third movement instruction is configured to notify the paragraph layout module that the in-line offset operation in the current line has been completed.

[0516] Further based on the foregoing example, if the total width of the character rectangles corresponding to characters such as "fang", "fo", "shi", "yi", "zhong", "jing", "xin", "hui", "zhi", "de", "shui", "cai", "hua" is less than or equal to the first offset, the line layout module may end the offset operation.

[0517] If the total width of the character rectangles corresponding to characters such as "fang", "fo", "shi", "yi", "zhong", "jing", "xin", "hui", "zhi", "de", "shui" is equal to the first offset, the line layout module may also end the offset operation.

[0518] The accompanying drawings in the embodiments of the present application only exemplarily show the scenario where the traversal of the character rectangles in the end line rectangle is completed.

[0519] In step S177, the electronic device 100 again performs the step of traversing the line rectangles along the first direction in the target paragraph to determine the start line rectangle, the end line rectangle, the first offset region, and the first index position.

[0520] In one implementation, step S177 includes step S1771.

[0521] In step S1771, in response to a third movement instruction, the paragraph layout module again performs the step of traversing the line rectangles along the first direction in the target paragraph to determine the start line rectangle, the end line rectangle, the first offset region, and the first index position.

[0522] Further based on the foregoing example, in the embodiment of the present application, since the first line rectangle 43 has been filled and there are no other line rectangles in the paragraph layout module, the paragraph layout module may perform the next step S1791.

[0523] If there are more line rectangles in the first paragraph rectangle 50 and there is a blank area in the line rectangles, the paragraph layout module may perform step S1732 again.

[0524] In step S178, when the traversal of all line rectangles in the target paragraph is completed, the electronic device 100 completes the movement of the first part of characters to fill the blank area.

[0525] In one implementation, step S178 includes steps S1781 to S1783.

[0526] In step S1781, when the traversal of all line rectangles in the target paragraph rectangle is completed, the paragraph layout module sends a fourth movement instruction to the text layout module.

[0527] In step S1782, the text layout module obtains the current character rectangle, the current line rectangle, and the current segment rectangle based on the fourth move instruction, and sends the fifth move instruction to the text layout display component.

[0528] In step S1783, the text layout display component obtains the current character rectangle, the current line rectangle, and the current segment rectangle based on the fifth move instruction, and triggers the update of the first interface to fill the blank area.

[0529] Specifically, the text layout display component can send the data corresponding to the current character rectangle, the current line rectangle, and the current segment rectangle to the rendering process, so that the rendering thread can draw the first interface based on this data and send the first interface to the display screen. In this way, the electronic device 100 automatically fills the blank area corresponding to the deletion operation in the first interface, realizing automatic layout.

[0530] Based on the above steps, the electronic device 100 can divide handwritten text into characters, lines, and paragraphs. After the user deletes some handwritten characters in the handwritten text, the remaining handwritten characters can be reformatted in the paragraph. This solves the problem that the user needs to manually move the remaining handwritten characters multiple times using the lasso control before the electronic device can reformatted the remaining characters, resulting in low editing efficiency of the electronic device and improving the user experience.

[0531] In step S18, if the first part of the characters includes at least a portion of the first target characters, the electronic device 100 updates the position of the first marker so as to display the first update marker on the first target character after moving the position of the first part of the characters.

[0532] In one implementation, step S18 includes step S181.

[0533] In step S181, after moving the position of the first part of the characters, the electronic device 100 displays a first update mark on the first target character based on the valid flag bit.

[0534] In one implementation, step S181 includes steps S1811-S1812.

[0535] In step S1811, the electronic device 100 traverses the first target row rectangle and determines the first target word rectangle corresponding to the valid flag bit in the current first target row rectangle.

[0536] In one implementation, step S1811 includes step S1811a.

[0537] In step S1811a, the view processing module traverses the first target row rectangles, and determines the first target word rectangle corresponding to the valid flag bit in the current first target row rectangle.

[0538] Specifically, the view processing module may traverse the word rectangles in the current first target row rectangle based on a first order, to obtain the maximum word string group with markFlag set to True. The maximum word string group may include a plurality of consecutive word data structures.

[0539] Further based on the foregoing example, since the first target row rectangles include the first row rectangle 43 and the first newly-built row rectangle 45, the view processing module may first traverse the first row rectangle 43, and obtain the maximum word string group with markFlag set to True as a word string group composed of the first word data structures corresponding to the word rectangle of the character "精", the word rectangle of the character "心", the word rectangle of the character "绘", and the word rectangle of the character "制".

[0540] Next, the view processing module may traverse the first newly-built row rectangle 45. Since the view processing module does not traverse to the maximum word string group and there is no other first target row rectangle currently, the view processing module ends the traversal.

[0541] In step S1812, on the first target word rectangle corresponding to the valid flag bit, the electronic device 100 sets the height and width of the first update flag based on the row height and the row midline of the current first target row rectangle, to display the first update flag.

[0542] In an implementation, step S1812 includes step S1812a.

[0543] In step S1812a, on the first target word rectangle corresponding to the valid flag bit, the view processing module sets the height and width of the first update flag based on the row height and the row midline of the current first target row rectangle, to display the first update flag.

[0544] Figure 36 is a schematic diagram of a scenario where the electronic device displays the first update flag according to an embodiment of the present application.

[0545] As shown in Figure 36 , the view processing module may set the height and width of the first update flag 54 based on the row height and the row midline corresponding to the first row rectangle 43, and send the first update flag 54 for display, so that the display screen displays the first update flag 54.

[0546] In an implementation, the view processing module may also set the height and width of the first update flag 54 according to a preset height and a preset width, to display the first update flag. The embodiment of the present application does not limit the specific display manner of the first update flag 54.

[0547] In one implementation, step S181 includes steps S1813-S1814.

[0548] In step S1813, the text layout display component sends the current character rectangle, the current line rectangle, and the current segment rectangle to the view processing module.

[0549] In step S1814, the view processing module displays a first update marker on the first target character based on the word rectangle with valid flag bits.

[0550] Thus, compared with the aforementioned steps S1811-S1812, step S1814 does not require traversing multiple first target row rectangles to locate the word rectangle with a valid flag bit, thereby realizing the display of the first update mark and reducing the delay time for displaying the first update mark.

[0551] It should be noted that step S181 can be performed based on steps S1811-S1812 or steps S1813-S1814, and this application embodiment does not limit this.

[0552] It should also be noted that in step S1783, after the text layout display component obtains the current character rectangle, the current line rectangle and the current segment rectangle based on the fifth movement instruction, it may not trigger the update of the first interface, but wait for step S1812 to be executed before triggering the update of the first interface. This application embodiment does not limit the specific timing of the electronic device 100 triggering the update of the first interface.

[0553] In one implementation, step S17 is followed by step S19.

[0554] In step S19, if the first part of the characters does not include the first target character, then after moving the position of the first part of the characters, the electronic device 100 does not display the first update mark on the first text.

[0555] In some cases, the second target character deleted by the electronic device 100 can be the first target character that has been marked. In this way, after the electronic device 100 moves the position of the first part of the characters, the first update mark is not displayed on the first text.

[0556] It should be noted that in some cases, after the electronic device 100 moves the first part of the characters, a large gap may appear between two adjacent first paragraphs. This embodiment also supports reformatting the first text based on traversing segment rectangles. The method of traversing segment rectangles can be found in the methods of traversing line rectangles and word rectangles described above, and will not be elaborated upon here. Furthermore, the process of traversing segment rectangles can be performed after step S17 or after step S18; this embodiment does not limit this.

[0557] The tag update method provided in this application can divide handwritten text based on characters, lines, and paragraphs. After a user deletes some handwritten characters from the text, the remaining handwritten characters are automatically reformatted within the paragraph, and the tags corresponding to the handwritten characters are updated synchronously during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and tags after automatic reformatting on electronic devices.

[0558] Figure 37 This is a second flowchart illustrating the marker update method provided in this application embodiment.

[0559] like Figure 37 As shown, in some embodiments, the method includes the following steps S21-S24.

[0560] In step S21, the electronic device 100 displays a first interface of the first application and displays first text generated based on handwriting operation on the first interface, wherein the first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character.

[0561] In step S22, in response to a marking operation performed on at least one first target character among the first characters, the electronic device 100 displays a first mark on the first target character.

[0562] In step S23, when deleting at least one second target character from the first characters in the target paragraph of the first paragraph, the electronic device 100 moves the position of the first part of the characters to fill the blank area formed by the deleted target characters, wherein the first part of the characters includes at least a portion of the first characters in the target paragraph other than the second target characters.

[0563] In step S24, if the first part of the characters includes at least a portion of the first target characters, the electronic device 100 updates the position of the first marker so as to display the first update marker on the first target character after moving the position of the first part of the characters.

[0564] The foregoing mainly describes the solutions provided by the embodiments of this application from the perspective of electronic device 100. It is understood that, in order to achieve the above functions, electronic device 100 includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the tag update method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by software-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0565] This application embodiment can divide the above-described electronic device 100 into functional modules or functional units according to the above method example. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0566] Figure 38 This is a schematic diagram of the marker update device provided in the embodiments of this application.

[0567] like Figure 38As shown, the tag updating device 200 provided in this application embodiment can be applied to the electronic device 100 in the above embodiments. The tag updating device may include a display screen 201, a memory 202, a processor 203, and a communication module 204. These devices can be connected via one or more communication buses 205. The display screen 201 may include a display panel 2011 and a touch sensor 2012. The display panel 2011 displays images, and the touch sensor 2012 transmits detected touch operations to the application processor to determine the touch event type. The display panel 2011 provides visual output related to the touch operation. The processor 203 may include one or more processing units, such as an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. Different processing units may be independent devices or integrated into one or more processors. The memory 202 is coupled to the processor 203 and is used to store various software programs and / or computer instructions. The memory 202 may include volatile memory and / or non-volatile memory. When the processor executes computer instructions, the electronic device can perform the various functions or steps performed in the above method embodiments.

[0568] Figure 39 This is a schematic diagram of the structure of a marker update device provided in another embodiment of this application.

[0569] like Figure 39As shown, in some other embodiments, this application also provides a tag update device 300, which can be applied to the electronic device in the above embodiments. The tag update device 300 includes a first display module 301, a second display module 302, a moving module 303, and an update module 304. The first display module 301 is used to display a first interface of the first application and display first text generated based on handwriting operation on the first interface. The first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character. The second display module 302 is used to display a first mark on the first target character in response to a marking operation performed on at least one first target character in the first character. The moving module 303 is used to move the position of a first part of the characters in the target paragraph of the first paragraph to fill the blank area formed by deleting the second target character. The first part of the characters includes at least a part of the first characters in the target paragraph other than the second target characters. If the first part of the characters includes at least a part of the first target characters, the updating module 304 is used to update the position of the first mark so that a first update mark is displayed on the first target character after moving the position of the first part of the characters.

[0570] The tag updating device provided in this application can divide handwritten text based on characters, lines, and paragraphs. After a user deletes some handwritten characters from the text, the device automatically reformatts the remaining handwritten characters within the paragraph, and synchronously updates the tags corresponding to the handwritten characters during the automatic reformatting process. This avoids the problem of mismatch between handwritten characters and tags after automatic reformatting by electronic devices.

[0571] This application also provides an electronic device, which includes a processor and a memory coupled together. The memory stores program instructions, and when the program instructions are executed by the processor, the processor performs the various functions or steps as described in the above method embodiments.

[0572] Figure 40 This is a schematic diagram of the chip system provided in the embodiments of this application.

[0573] like Figure 40As shown, this application embodiment also provides a chip system 400, such as a SoC, which includes at least one processor 401 and at least one interface circuit 402. The processor 401 and the interface circuit 402 can be interconnected via lines. For example, the interface circuit 402 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 402 can be used to send signals to other devices (e.g., the processor 401 or the touchscreen of an electronic device). Exemplarily, the interface circuit 402 can read instructions stored in the memory and send the instructions to the processor 401. When the instructions are executed by the processor 401, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0574] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0575] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the electronic device in the above method embodiments.

[0576] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0577] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0578] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0579] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0580] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0581] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A label update method, characterized in that, Applied to electronic devices, including: The first interface of the first application is displayed, and the first text generated based on the handwriting operation is displayed on the first interface, wherein the first text consists of at least one first paragraph, each first paragraph consists of at least one line, and each line consists of at least one first character; In response to a marking operation performed on at least one first target character among the first characters, a first mark is displayed on the first target character; In the target paragraph of the first paragraph, when at least one second target character in the first character is deleted, the position of the first part of the characters is moved to fill the blank area formed by deleting the second target character, wherein the first part of the characters includes at least a part of the first characters in the target paragraph other than the second target character; If the first part of the characters includes at least a portion of the first target character, then the position of the first marker is updated so that after moving the position of the first part of the characters, a first update marker is displayed on the first target character.

2. The tag update method according to claim 1, characterized in that, In the target paragraph of the first paragraph, after deleting at least one second target character from the first characters, the method further includes moving the position of the first part of the characters to fill the blank area formed by deleting the second target character, and then: If the first part of the characters does not include the first target character, then after moving the position of the first part of the characters, the first update marker will not be displayed on the first text.

3. The tag update method according to claim 1 or 2, characterized in that, The step of displaying a first interface of a first application and displaying first text generated based on handwriting operations on the first interface includes: In response to a handwriting operation performed on the first interface, at least one of the first characters corresponding to the handwriting operation is displayed; At least one character rectangle is obtained based on at least one first character, wherein each character rectangle corresponds to at least one first character, and the character rectangle is the smallest bounding rectangle corresponding to one first character representing a single word, or the character rectangle is the smallest bounding rectangle corresponding to multiple first characters used to represent a single word; At least one line rectangle is determined based on at least one of the character rectangles, wherein each of the line rectangles corresponds to a line in the first text; At least one segment rectangle is determined based on at least one of the row rectangles, wherein each of the segment rectangles corresponds to one of the first paragraphs; When the first text is displayed, the first character is divided based on the character rectangle, the lines in the first text are divided based on the line rectangle, and the first paragraph is divided based on the segment rectangle.

4. The tag update method according to claim 3, characterized in that, Determining at least one row rectangle based on at least one of the character rectangles includes: Create a first row rectangle based on the first obtained first character rectangle, and determine the first row center line corresponding to the first row rectangle, wherein the first row center line passes through the center point of the first character rectangle and is parallel to the horizontal direction; Traverse the second character rectangle, and when traversing to the current second character rectangle, traverse all the currently created row rectangles to determine whether the current second character rectangle intersects with any row midline corresponding to all the current row rectangles, wherein the second character rectangle includes the character rectangles after the first character rectangle obtained in the first order; If the current second-character rectangle intersects with the target row center line in the row center line, the current second-character rectangle is added to the target row rectangle corresponding to the target row center line; If the current second-character rectangle does not intersect with any of the row center lines, create a new row rectangle based on the current second-character rectangle, and traverse the next second-character rectangle; After traversing all the second-order rectangles, all the row rectangles are obtained.

5. The tag update method according to claim 4, characterized in that, Determining at least one segment rectangle based on at least one row rectangle includes: Create the first rectangle segment based on the first row of rectangles; Along the first direction, traverse the second row of rectangles, and when traversing to the current second row of rectangles, obtain the row spacing between the current second row of rectangles and the previous row of rectangles, wherein the first direction is the generation direction of the second row of rectangles, and the second row of rectangles includes the row of rectangles created after the first row of rectangles; If the line spacing is greater than a preset line spacing threshold, or if the line spacing is greater than the average line spacing, a new segment rectangle is created, and the current second row rectangle is marked as belonging to the newly created segment rectangle, and then the next second row rectangle is traversed. If the line spacing is less than or equal to the line spacing threshold, and if the line spacing is less than or equal to the average line spacing, mark the current second row rectangle as belonging to the last created segment rectangle, and traverse the next second row rectangle; After traversing all the rectangles in the second row, all the segment rectangles are obtained.

6. The tag update method according to claim 5, characterized in that, The step of displaying a first mark on the first target character in response to a marking operation performed on at least one first target character among the first characters includes: In response to the marking operation performed on the first target character, the first marking trajectory is obtained; The first mark is displayed on the first target character based on the first mark trajectory.

7. The tag update method according to claim 6, characterized in that, After obtaining the first marking trajectory in response to the marking operation performed on the first target character, the method further includes: Determine at least one first target character rectangle corresponding to the first marker trajectory, wherein one first target character rectangle corresponds to one first target character; In the first character data structure corresponding to each first target character rectangle, the first flag bit is set as a valid flag bit to identify the first target character corresponding to the first marker trajectory based on the valid flag bit. The first character data structure includes the text information corresponding to the first target character, the first target character rectangle, and the first flag bit.

8. The tag update method according to claim 7, characterized in that, After deleting at least one second target character from the first character in the target paragraph of the first paragraph, the method further includes: At least one first target row rectangle is determined, the first target row rectangle including at least one row rectangle where the first part of the characters are located, and at least one row rectangle corresponding to the deletion operation; Delete the first mark within the first target row rectangle.

9. The tag update method according to claim 8, characterized in that, The step of moving the position of the first part of the characters to fill the blank area formed by deleting the second target character includes: Determine the target paragraph corresponding to the blank area; In the target paragraph, the row rectangles are traversed along the first direction to determine the starting row rectangle, the ending row rectangle, the first offset area, and the first index position. The starting row rectangle is the first row rectangle obtained that has at least a portion of the blank area, and the ending row rectangle is the next row rectangle obtained that has at least a portion of the blank area. The first offset range corresponds to the target blank area corresponding to the starting row rectangle, and the first index position corresponds to the starting position of the first character rectangle in the ending row rectangle. Within the starting row rectangle, the word rectangles are traversed, and the word rectangles located after the first offset region are offset to move the word rectangles to the first offset region, and the first offset region is updated. In the ending row rectangle, the word rectangles are traversed starting from the first index position, and an offset operation is performed on each traversed word rectangle to move the word rectangle to the updated first offset region; The offset operation ends when the word rectangles in the ending row rectangle have been traversed, or when the word rectangles in the ending row rectangle have filled the first offset region. The step of traversing the row rectangles along the first direction in the target paragraph again to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position is executed again; After traversing all the line rectangles in the target paragraph, the first part of the characters is moved to fill the blank area.

10. The tag update method according to claim 9, characterized in that, The step of updating the position of the first marker if the first part of the characters includes at least a portion of the first target characters, so as to display a first update marker on the first target character after moving the position of the first part of the characters, includes: After moving the position of the first part of the characters, the first update mark is displayed on the first target character based on the valid flag bit.

11. The tag update method according to claim 10, characterized in that, After moving the position of the first portion of characters, displaying the first update marker on the first target character based on the valid flag bit includes: Traverse the first target row rectangle and determine the first target word rectangle corresponding to the valid flag bit in the current first target row rectangle; On the first target word rectangle corresponding to the valid flag bit, based on the current row height and row center line of the first target row rectangle, the height and width of the first update mark are set to display the first update mark.

12. The tag update method according to claim 11, characterized in that, The step of obtaining at least one character rectangle based on at least one of the first characters includes: The first recognition engine acquires the handwriting data corresponding to the first character and recognizes the handwriting data to obtain at least one full recognition result. Then, based on the full recognition result, the character rectangle is obtained. One character rectangle corresponds to one of the first characters used to represent a single character, or one character rectangle corresponds to multiple of the first characters used to represent a single word.

13. The tag update method according to claim 12, characterized in that, After obtaining at least one character rectangle based on at least one of the first characters, the method further includes: The first recognition engine sends the full recognition results to the text layout module; The text layout module sorts the full recognition results based on a preset first order, so that the character rectangles have the first order.

14. The tag update method according to claim 13, characterized in that, Determining at least one row rectangle based on at least one of the character rectangles includes: The text layout module calls the line layout module to sequentially concatenate the character rectangles with the first order to determine the line rectangle.

15. The tag update method according to claim 14, characterized in that, Determining at least one segment rectangle based on at least one row rectangle includes: The text layout module calls the segment layout module to concatenate the row rectangles to determine the segment rectangles.

16. The tag update method according to claim 15, characterized in that, When displaying the first text, the steps of dividing the first character based on the character rectangle, dividing the lines in the first text based on the line rectangle, and dividing the first paragraph based on the segment rectangle include: The segment layout module sends a first typesetting result to the first recognition engine, so that the first recognition engine can complete the division of the first character based on the character rectangle, complete the division of the lines in the first text based on the line rectangle, and complete the division of the first paragraph based on the segment rectangle, wherein the first typesetting result includes the character rectangle, the line rectangle, and the segment rectangle.

17. The tag update method according to claim 16, characterized in that, The step of obtaining the first marking trajectory in response to the marking operation performed on the first target character includes: In response to the marking operation performed on the first target character, the view processing module obtains the first marking trajectory.

18. The tag update method according to claim 17, characterized in that, After obtaining the first marking trajectory in response to the marking operation performed on the first target character, the method further includes: The view processing module sends the first marked trajectory to the first recognition engine; The first recognition engine determines at least one first target character rectangle corresponding to the first marker trajectory; The first recognition engine sets the first flag bit as a valid flag bit in each first character data structure, so as to identify the first target character corresponding to the first marker trajectory based on the valid flag bit.

19. The tag update method according to claim 18, characterized in that, Before determining the target paragraph corresponding to the blank area, the method further includes: The view processing module triggers a reflow process based on the deletion operation to send a reflow instruction set to the first recognition engine; Upon receiving the first layout result, the first recognition engine obtains the deletion trajectory corresponding to the blank area based on the re-layout instruction set; The first recognition engine sends the deletion trajectory to the text layout module.

20. The tag update method according to claim 19, characterized in that, Determining the target paragraph corresponding to the blank area includes: The text layout module determines the segment rectangle corresponding to the deletion trajectory as the target segment rectangle corresponding to the target paragraph.

21. The tag update method according to claim 20, characterized in that, After the text layout module determines the segment rectangle corresponding to the deletion trajectory as the target segment rectangle corresponding to the target paragraph, it further includes: The text layout module sends a first move instruction to the segment layout module, so that the segment layout module performs an offset operation on the row rectangle in each target segment rectangle based on the first move instruction.

22. The tag update method according to claim 21, characterized in that, The step of traversing the row rectangles along the first direction within the target paragraph to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position includes: In response to the first move instruction, the segment layout module traverses the row rectangles along the first direction in each target segment rectangle to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position, and sends a second move instruction to the row layout module so that the row layout module performs an offset operation on the character rectangles in each row rectangle based on the second move instruction.

23. The tag update method according to claim 22, characterized in that, The step of traversing the character rectangles within the starting row rectangle and offsetting the character rectangles located after the first offset region to move them to the first offset region, and updating the first offset region, includes: In response to the second move instruction, the row layout module traverses the character rectangles within the starting row rectangle and performs an offset operation on the character rectangles located after the first offset region to move the character rectangles to the first offset region and update the first offset region.

24. The tag update method according to claim 23, characterized in that, The step of traversing the word rectangles starting from the first index position within the ending row rectangle, and performing an offset operation on each traversed word rectangle to move it to the updated first offset region, includes: The row layout module traverses the character rectangles in the ending row rectangle starting from the first index position, and performs an offset operation on each traversed character rectangle to move the character rectangle to the updated first offset area.

25. The tag update method according to claim 24, characterized in that, The offset operation ends when the traversal of the character rectangles in the ending row rectangle is completed, or when the character rectangles in the ending row rectangle have filled the first offset region, including: When the word rectangle in the ending row rectangle has been traversed, or when the word rectangle in the ending row rectangle has filled the first offset area, the row layout module ends the offset operation and sends a third move instruction to the segment layout module. The third move instruction is used to notify the segment layout module that the current inline offset operation has been completed.

26. The tag update method according to claim 25, characterized in that, The step of re-executing the step of traversing the row rectangles along the first direction in the target paragraph to determine the starting row rectangle, the ending row rectangle, the first offset region, and the first index position includes: In response to the third move instruction, the segment layout module again executes the step of traversing the row rectangles along the first direction in the target segment to determine the start row rectangle, end row rectangle, first offset region, and first index position.

27. The tag update method according to claim 26, characterized in that, The step of moving the first part of the characters to fill the blank area after traversing all the row rectangles in the target segment rectangle includes: After traversing all the row rectangles in the target segment rectangle, the segment layout module sends a fourth move instruction to the text layout module. The text layout module obtains the current character rectangle, the current line rectangle, and the current segment rectangle based on the fourth movement instruction, and sends the fifth movement instruction to the first recognition engine; The first recognition engine obtains the current character rectangle, the current line rectangle, and the current segment rectangle based on the fifth movement instruction to complete the movement of the first part of the characters and trigger the update of the first interface to fill the blank area.

28. The tag update method according to claim 27, characterized in that, After deleting at least one second target character from the first character in the target paragraph of the first paragraph, the method further includes: The view processing module sends the first query information to the first recognition engine; In response to the first query information, the first recognition engine sends the first target row rectangle to the view processing module; The view processing module deletes the first marker in the first target row rectangle.

29. The tag update method according to claim 28, characterized in that, The step of updating the position of the first marker if the first part of the characters includes at least a portion of the first target characters, so as to display a first update marker on the first target character after moving the position of the first part of the characters, includes: The view processing module displays the first update mark on the first target character based on the valid flag bit.

30. An electronic device, characterized in that, include: A processor and a memory, the memory storing program instructions that, when executed by the processor, cause the electronic device to perform the tag update method as described in any one of claims 1-29.

Images