Method, device and apparatus for mixed layout of handwriting track and text content
By abstracting text and handwritten traces into Text objects, and then cropping and rendering them, the problem of resolution adaptation and reading difficulty of handwritten content across different devices is solved, achieving clear display and editing functions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TIANWEN DIGITAL MEDIA TECH BEIJING
- Filing Date
- 2022-12-19
- Publication Date
- 2026-05-26
AI Technical Summary
Existing hybrid typesetting technologies cannot accurately recognize complex handwritten content, especially mathematical formulas, and the resolution adaptation of handwritten trajectories across different devices and the adjustment of display cell sizes increase the difficulty of reading.
The text and handwritten traces are abstracted into Text objects. A hybrid layout page object is generated through cropping and layout algorithms, and the rendering is performed according to the type of Text object. It supports rendering processing at the Text, Line and Page levels, so as to achieve accurate display and adaptation of handwritten content.
It enables clear display of handwritten content on devices with different resolutions, supports dynamic adjustment of cell size to ensure a good reading experience, and also supports editing and cursor positioning.
Smart Images
Figure CN115952766B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of this application relate to the field of mixed typesetting, and more particularly to methods, apparatus, devices, and computer-readable storage devices for mixed typesetting of handwritten trajectories and text content. Background Technology
[0002] Currently, existing methods that support mixed image and text content layout will either recognize handwritten content as text or save the handwritten traces as an image. However, when encountering complex handwritten content, the recognition becomes extremely inaccurate. For example, it cannot recognize the answers to complex mathematical formulas, thus failing to support answering similar questions. Saving complex handwritten content as an image presents the following problems:
[0003] 1. Issues with image resolution adaptation across different devices;
[0004] 2. On different display devices, adjusting the size of the displayed cells can cause the image size to be scaled, which can reduce the clarity of the trajectory, increase the difficulty and speed of reading, and result in a poor user reading experience. Summary of the Invention
[0005] According to an embodiment of this application, a scheme for mixed typesetting of handwritten trajectories and text content is provided.
[0006] In a first aspect of this application, a method for typesetting a mixture of handwritten trajectories and text content is provided. The method includes:
[0007] Get the text and handwritten trajectory;
[0008] The text and handwritten traces are abstracted into a Text object;
[0009] The Text object is then formatted to obtain a page object with a mixed layout;
[0010] Based on the type of the Text object, the page object with the mixed layout is rendered and displayed to complete the mixed layout of the text and handwritten traces.
[0011] Furthermore, the step of abstracting the handwritten trajectory into a Text object includes:
[0012] The trajectory information of each handwritten input within a preset period is stored in an array; the trajectory information includes the points, size, and color information of the trajectory.
[0013] The data in the array is cropped to obtain a Text object corresponding to the handwritten trajectory.
[0014] Furthermore, the cropping of the data in the array includes:
[0015] Subtract the x-coordinate of the leftmost point from the x-coordinate of all points in the trajectory in the array;
[0016] Subtract the y-coordinate of the vertex of the trajectory from the y-coordinate of all points in the array.
[0017] Furthermore, the width and height of the handwritten trajectory Text object are determined as follows:
[0018] Subtract the x-coordinate of the leftmost point from the x-coordinate of the rightmost point in the array, and then divide the result by the width of the area cell to get the width of the handwritten trajectory Text object.
[0019] Subtract the y-coordinate of the vertex from the y-coordinate of the lowest point in the array, and then divide the result by the height of the area cell to obtain the height of the handwritten trajectory Text object.
[0020] Furthermore, the step of formatting the Text object to obtain a mixed-format page object includes:
[0021] Store all Text objects in a collection;
[0022] Iterate through the Text objects in the collection, perform addition on the width of the Text objects, and if the result exceeds the width of the display area or a newline character is read, store all Text objects before the current Text object in the first Line collection and store the current Text object in the second Line collection.
[0023] Repeatedly iterate until the Text objects in the collection are empty, and then obtain a mixed-layout page object based on all the Line collections.
[0024] Furthermore, rendering the mixed-layout page object according to the type of the Text object includes:
[0025] Initialize the horizontal and vertical coordinates of the Text objects in the hybrid layout page object;
[0026] The page object with mixed layout is rendered based on the type of the Text object and the horizontal and vertical coordinates of the Text object; the type of the Text object includes text objects and handwritten trajectory objects.
[0027] Furthermore, it also includes:
[0028] When adding or deleting a Text object, the type of rendering processing required after this editing operation is indicated during the layout process; the rendering type includes Text-level rendering, Line-level rendering, and / or Page-level rendering.
[0029] Text-level rendering includes drawing only newly added Text objects;
[0030] Line-level rendering includes rendering the Text objects within the current Line range and subsequent Text objects;
[0031] Page-level rendering includes rendering the Text objects within the current page scope and subsequent Text objects.
[0032] In a second aspect of this application, an apparatus for typesetting a mixture of handwritten trajectories and text content is provided. The apparatus includes:
[0033] The acquisition module is used to acquire text and handwritten trajectories;
[0034] The abstract module is used to abstract the text and handwritten trajectories into Text objects;
[0035] The layout module typeset the Text object to obtain a page object with mixed layout;
[0036] The rendering module is used to render and display the mixed-layout page object according to the type of the Text object, and complete the layout of the text and handwritten trajectory.
[0037] In a third aspect of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.
[0038] In a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to the first aspect of this application.
[0039] The method for mixed layout of handwritten trajectories and text content provided in this application embodiment obtains text and handwritten trajectories; abstracts the text and handwritten trajectories into Text objects; typesets the Text objects to obtain a mixed layout page object; and renders and displays the mixed layout page object according to the type of the Text objects, thus completing the mixed layout of the text and handwritten trajectories. This method can save handwritten content as text or trajectory abstract objects. For complex mathematical formulas, they can be saved as abstract objects, and the relevant handwritten content is presented seamlessly in the display area. It also solves the problem of adapting handwritten trajectories to different resolutions.
[0040] It should be understood that the description in the Summary Section is not intended to limit the key or essential features of the embodiments of this application, nor is it intended to restrict the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0041] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0042] Figure 1 A flowchart illustrating a method for typesetting a mixture of handwritten trajectories and text content according to an embodiment of this application is shown;
[0043] Figure 2 A schematic diagram illustrating the handwritten trajectory and text content according to an embodiment of this application is shown;
[0044] Figure 3 A block diagram of an apparatus for typographically combining handwritten trajectories with text content according to an embodiment of this application is shown;
[0045] Figure 4 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown. Detailed Implementation
[0046] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0047] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0048] Figure 1 A flowchart illustrating a method for typesetting a mixture of handwritten trajectories and text content according to an embodiment of the present disclosure is shown. The method includes:
[0049] S110, acquire text and handwritten trajectory.
[0050] Among them, such as Figure 2As shown, text is usually standard characters that the system can directly recognize, such as KaiTi and SongTi, which are input via the system keyboard; handwritten traces are usually non-standard characters (numbers, letters, etc.) that the system cannot directly recognize, which are input via input devices such as handwriting keyboards and styluses.
[0051] S120, the text and handwritten trajectory are abstracted into a Text object.
[0052] The Text object may include the following properties:
[0053] Text.type: Used to distinguish text types, mainly including keyboard input text, line breaks, and handwriting traces;
[0054] Text.content: Used to store the text content entered from the keyboard. Each character is an abstract Text object. This value exists when the type is keyboard text type.
[0055] Text.style: Text style, storing style properties such as font size and color;
[0056] Text.widthLength: The number of cells occupied by the width of the Text object when it is displayed. It supports decimals; for example, a value of 0.345 means that the current Text object occupies the width of 0.356 cells.
[0057] Text.heightLength: The number of cells that the Text object occupies when it is displayed. It supports decimals; for example, a value of 0.6 means that the current Text object occupies 0.6 cells in height.
[0058] Text.Array[DrawPath]: This property exists when writing handwritten text paths. It is used to store a collection of handwritten paths. One DrawPath object is generated in each Down-Move...-Up / Cancel event cycle. DrawPath is a collection of points of the drawn path. Multiple DrawPath objects in one input are stored in this array.
[0059] In some embodiments, text can be automatically generated into a standardized Text object based on the cell size of the display area;
[0060] In some embodiments, when abstracting handwritten trajectories into a Text object, the following operations are required:
[0061] The handwritten input within a preset period (either by clicking the input button or at 5-second intervals) is abstracted into a Text object. Each stroke, i.e., a line or point, is abstracted into a DrawPath object, storing all points, the size, and color of each stroke in a temporary array. After completing one period of input, the stroke data of the Text object is processed using a clipping algorithm and stored in the Text.Array[DrawPath] array property. The Text object is then added to the cursor position for subsequent layout processing.
[0062] Furthermore, the pruning algorithm includes:
[0063] Iterate through the temporary array, extract the x-coordinate mX1 of the leftmost point in the trajectory, extract the x-coordinate mX2 of the rightmost point in the trajectory, the y-coordinate mY1 of the vertex in the trajectory, and the y-coordinate mY2 of the lowest point in the trajectory; then subtract mX1 from the x-coordinate and mY1 from the y-coordinate of all points in the trajectory set and store them in the Text.Array[DrawPath] array property to complete the trimming of the handwritten trajectory.
[0064] Furthermore, the width and height of the handwritten trajectory Text object are determined as follows:
[0065] Subtract the x-coordinate of the leftmost point from the x-coordinate of the rightmost point in the array, and then divide the result by the width of the area cell to get the width of the handwritten trajectory Text object.
[0066] Subtract the y-coordinate of the vertex from the y-coordinate of the lowest point in the array, and then divide the result by the height of the area cell to obtain the height of the handwritten trajectory Text object.
[0067] That is, subtract the value of mX1 from mX2 and divide it by the width of the input area cell, then assign the result to the Text.widthLength property; subtract the value of mY1 from mY2 and divide it by the height of the input area cell, then assign the result to the Text.heightLength property.
[0068] S130, The Text object is formatted to obtain a page object with mixed formatting.
[0069] In some embodiments, all Text objects are stored in a collection; the Text objects in the collection are traversed, and the width of the Text objects is added iteratively. If the result of the operation exceeds the width of the display area or a newline character is read, all Text objects before the current Text object are stored in the first Line collection, and the current Text object is stored in the second Line collection; this process is repeated until the Text objects in the collection are empty. Based on all the Line collections, a mixed-layout page object is obtained.
[0070] That is, initialize a page object and a line object, put all text objects into a temporary collection, traverse the text objects, perform addition iteration on the Text.widthLength property, if the calculation result exceeds the width of the display area, or encounters a newline character, then put the collection of Text objects before the current character into the first Line object, and at the same time construct a new Line object, namely the second Line object, and put the current Text object into the second Line object; continue to traverse the remaining elements, iterate repeatedly until the temporary collection is empty, and then put the collection of all Line objects into the TextLinePage.Array[Line] property.
[0071] Furthermore, after the layout is complete, when a new Text object needs to be added, the Text object after the cursor is added back to the temporary collection, the layout is reformatted according to the layout algorithm, and the new collection of Line objects is put into the TextLinePage.Array[Line] property;
[0072] When a Text object needs to be deleted, delete the Text object before the cursor, add the Text object after the cursor back to the temporary collection, reformat it according to the layout algorithm, and put the new collection of Line objects into the TextLinePage.Array[Line] property.
[0073] S140, according to the type of the Text object, render and display the mixed layout page object to complete the mixed layout of the text and handwritten trajectory.
[0074] In some embodiments, the page object, which has been formatted by a typesetting algorithm, is traversed through the collection of Line objects in the page object, and each Line object is processed.
[0075] Specifically, initialize the vertical coordinate of the Line object: accumulate the heightLength property of each Line object, and the value of the vertical coordinate of the current Line object is the result of accumulating the heightLength properties of all the preceding Line objects; the starting horizontal coordinate of the Line object is 0 by default or the initial value passed in during initialization;
[0076] Process the Text object collection of the Line object: Iterate through the Text object collection, initialize the horizontal and vertical coordinates of the Text objects, the vertical coordinate is the vertical coordinate of the current Line object + the heightLength of the current Line object * the height of the cell - the heightLength of the current Text object * the height of the cell, and the horizontal coordinate is the sum of the widthLength of all Text objects (within the current Line object) before the current Text object and then multiplied by the width of the cell.
[0077] After initializing the horizontal and vertical coordinates, determine the type of the Text object. If it is a text object, then according to the style properties, call the system's text drawing interface to draw the text at the corresponding coordinate position on the canvas.
[0078] If it is a handwritten trajectory object, then check whether the DrawPath object in the Text.Array[DrawPath] property has already generated a Path object that can be directly drawn on the canvas. If not, then iterate through the Text.Array[DrawPath] property of the Text object, multiply the coordinate values of all points in the DrawPath by the value of TextLinePage.zoomQuotient, and then add the horizontal and vertical coordinates of the Text object. After initializing the coordinates of the DrawPath object, call the corresponding system's line drawing interface to convert the DrawPath object into a Path object that can be directly drawn on the canvas according to its style properties. If it has already been generated, then based on the difference between the horizontal and vertical coordinates of the Text object in the previous layout and the current layout, translate all the canvas-drawn Path objects of the Text object.
[0079] Furthermore, when adding or deleting Text objects, the type of rendering processing required after this editing operation is indicated during the layout process; the rendering type includes Text-level rendering, Line-level rendering, and / or Page-level rendering.
[0080] Text-level rendering includes drawing only newly added Text objects;
[0081] Line-level rendering includes: rendering Text objects within the current Line range and subsequent Text objects;
[0082] Page-level rendering includes: rendering Text objects within the current page range and subsequent Text objects.
[0083] Furthermore, it also includes:
[0084] Perform cursor positioning processing in the following manner:
[0085] Click on the editing area to obtain the coordinate values (X, Y) of the click position. Divide the horizontal coordinate value by the width of the display cell, and divide the vertical coordinate value by the height of the display cell to obtain the coordinate values (X1, Y1) after operation of the click position. Traverse the collection of Line objects of the page object. If the vertical coordinate value of the Line object >= Y1, and the vertical coordinate value of the Line object + Line.heightLength < Y1, then position to the range within this Line object of the click position;
[0086] Then process this Line object. Traverse the collection of Text objects of the Line object, and accumulate the Text.widthLength attributes one by one. If when accumulating to N Text objects, the accumulated value of the widthLength of the first N - 1 objects is less than X1, and the accumulated value of the widthLength of N objects >= X1, then this Text object is the click position; then determine the position of the click in the Text object. If it is the first half of the Text object, then the (N - 1)th Text object is the abstract cursor object; if it is the second half of the Text object, then the Nth Text object is the abstract cursor object; usually the cursor is displayed at the tail of the cursor object.
[0087] According to the embodiments of the present disclosure, the following technical effects are achieved:
[0088] The handwritten content can be saved as text or trajectory abstract objects. For complex mathematical formula operations, they can be saved as abstract objects, and the relevant handwritten content is presented without difference in the display area;
[0089] Solve the adaptation problem of handwritten trajectories at different resolutions, and can adapt to all display devices such as computers, pads, and / or mobile phones on the market;
[0090] Under the scenario of ensuring clarity, support dynamic adjustment of the cell size, and support the magnification and reduction of handwritten trajectories following the cell size;
[0091] The typeset content is editable, supports cursor positioning, and can dynamically add, delete text content and / or handwritten trajectory content.
[0092] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0093] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.
[0094] Figure 3 A block diagram of a handwritten trajectory and text content mixed typesetting device 300 according to an embodiment of this application is shown as follows. Figure 3 As shown, the device 300 includes:
[0095] Module 310 is used to acquire text and handwritten trajectories;
[0096] Abstract module 320 is used to abstract the text and handwritten trajectory into a Text object;
[0097] The layout module 330 typesets the Text object to obtain a page object with mixed layout;
[0098] The rendering module 340 is used to render and display the mixed layout page object according to the type of the Text object, and complete the layout of the text and handwriting trajectory.
[0099] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0100] Figure 4 A schematic diagram of a terminal device or server suitable for implementing embodiments of this application is shown.
[0101] like Figure 4 As shown, the terminal device or server 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage section 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the system 400. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0102] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.
[0103] Specifically, according to embodiments of this application, the above method flow steps can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this application.
[0104] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0106] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not, in certain circumstances, constitute a limitation on the unit or module itself.
[0107] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application.
[0108] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.
Claims
1. A method for typesetting handwritten trajectories mixed with text content, characterized in that, include: Get the text and handwritten trajectory; The text and handwritten traces are abstracted into a Text object; The Text object is then formatted to obtain a page object with a mixed layout; Based on the type of the Text object, the page object with the mixed layout is rendered and displayed to complete the mixed layout of the text and handwritten trajectory; The step of abstracting the handwritten trajectory into a Text object includes: Store the trajectory information of each handwritten input within a preset period into an array; The trajectory information includes the points, size, and color information of the trajectory; The data in the array is cropped to obtain a Text object corresponding to the handwritten trajectory; The cropping of the data in the array includes: Subtract the x-coordinate of the leftmost point from the x-coordinate of all points in the trajectory in the array; Subtract the y-coordinate of the vertex in the trajectory from the y-coordinate of all points in the array; The width and height of the handwritten trajectory Text object are determined as follows: Subtract the x-coordinate of the leftmost point from the x-coordinate of the rightmost point in the array, and then divide the result by the width of the area cell to get the width of the handwritten trajectory Text object. Subtract the y-coordinate of the vertex from the y-coordinate of the lowest point in the array, and then divide the result by the height of the area cell to obtain the height of the handwritten trajectory Text object. The process of formatting the Text object to obtain a mixed-format page object includes: Store all Text objects in a collection; Iterate through the Text objects in the collection, perform addition on the width of the Text objects, and if the result exceeds the width of the display area or a newline character is read, store all Text objects before the current Text object in the first Line collection and store the current Text object in the second Line collection. Repeatedly iterate until the Text objects in the collection are empty, and then obtain a mixed-layout page object based on all the Line collections; The step of rendering the mixed-layout page object according to the type of the Text object includes: Initialize the horizontal and vertical coordinates of the Text objects in the hybrid layout page object; The page object with mixed layout is rendered based on the type of the Text object and the horizontal and vertical coordinates of the Text object; The Text object includes text objects and handwritten trajectory objects; Also includes: When adding or deleting a Text object, the type of rendering processing required after the current editing operation is indicated during the layout process; the rendering type includes Text-level rendering, Line-level rendering, and / or Page-level rendering. Text-level rendering includes drawing only newly added Text objects; Line-level rendering includes rendering the Text objects within the current Line range and subsequent Text objects; Page-level rendering includes rendering the Text objects within the current page scope and subsequent Text objects.
2. A device for typesetting handwritten trajectories and text content, characterized in that, include: The acquisition module is used to acquire text and handwritten trajectories; The abstract module is used to abstract the text and handwritten trajectories into Text objects; The layout module typeset the Text object to obtain a page object with mixed layout; The rendering module is used to render and display the mixed layout page object according to the type of the Text object, and complete the layout of the text and handwriting trajectory; The abstract module abstracts the handwritten trajectory into a Text object, including: Store the trajectory information of each handwritten input within a preset period into an array; The trajectory information includes the points, size, and color information of the trajectory; The data in the array is cropped to obtain a Text object corresponding to the handwritten trajectory; The cropping of the data in the array includes: Subtract the x-coordinate of the leftmost point from the x-coordinate of all points in the trajectory in the array; Subtract the y-coordinate of the vertex in the trajectory from the y-coordinate of all points in the array; The width and height of the handwritten trajectory Text object are determined as follows: Subtract the x-coordinate of the leftmost point from the x-coordinate of the rightmost point in the array, and then divide the result by the width of the area cell to get the width of the handwritten trajectory Text object. Subtract the y-coordinate of the vertex from the y-coordinate of the lowest point in the array, and then divide the result by the height of the area cell to obtain the height of the handwritten trajectory Text object. The typesetting module is specifically used for: Store all Text objects in a collection; Iterate through the Text objects in the collection, perform addition on the width of the Text objects, and if the result exceeds the width of the display area or a newline character is read, store all Text objects before the current Text object in the first Line collection and store the current Text object in the second Line collection. Repeatedly iterate until the Text objects in the collection are empty, and then obtain a mixed-layout page object based on all the Line collections; The rendering module renders the mixed-layout page object according to the type of the Text object, including: Initialize the horizontal and vertical coordinates of the Text objects in the hybrid layout page object; The page object with mixed layout is rendered based on the type of the Text object and the horizontal and vertical coordinates of the Text object; The Text object includes text objects and handwritten trajectory objects; Also includes: When adding or deleting a Text object, the type of rendering processing required after the current editing operation is indicated during the layout process; the rendering type includes Text-level rendering, Line-level rendering, and / or Page-level rendering. Text-level rendering includes drawing only newly added Text objects; Line-level rendering includes rendering the Text objects within the current Line range and subsequent Text objects; Page-level rendering includes rendering the Text objects within the current page scope and subsequent Text objects.
3. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in claim 1.
4. A computer-readable storage device having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in claim 1.