A system and method for forming a writing track for editing a file on a non-touch display screen

Abstract

The application discloses a system and method for forming a writing track on a non-touch display screen. The system comprises a display with a micro-nano metal dot array, an active infrared electronic pen and a terminal processing device. The electronic pen shoots and calculates the global absolute physical coordinates of the pen tip. The terminal processing device maps the physical coordinates to the screen pixel coordinates of the document display area, dynamically calculates the line width based on the writing speed to construct a handwriting layer with a real pen stroke effect, extracts the minimum circumscribed rectangle of the handwriting when the pen is lifted to perform local area rasterization, generates a handwriting picture with a transparent background, and finally inserts and anchors the handwriting picture in the document in the form of an independent floating object by calling an application document object model interface. The application realizes cross-platform high-definition picture-level handwriting annotation without modifying the underlying coding of the document, greatly improving the compatibility and legal effectiveness of the document signing on the non-touch display.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and electronic document processing technology. Specifically, it relates to a writing system based on a micro-nano metal dot matrix non-touch display and active infrared visual recognition, as well as a system and method for editing various format files (such as PDF, Word, Excel, web pages, etc.) on a non-touch display using this system, and for rasterizing handwritten trajectories in image format and embedding them into the document. Background Technology

[0002] With the deepening of paperless office practices, handwritten signatures, annotations, and approvals on electronic screens have become an essential scenario for government, finance, education, and corporate offices. Existing electronic screen handwriting technology mainly faces bottlenecks in both hardware and software aspects.

[0003] At the underlying hardware interaction level: traditional capacitive or resistive touchscreens require an additional sensing layer on the screen surface. This not only reduces the monitor's light transmittance, affecting the clear display of small text, but also makes laying a sensing layer on large-size displays extremely costly and difficult in terms of manufacturing processes. Infrared bezel touchscreens rely on an infrared grid emitted from the edge, resulting in a wide bezel that affects aesthetics, and they are highly susceptible to interference and failure in strong light environments. Light diffusion at a distance causes a sharp drop in accuracy. In addition, traditional optical touchscreens are easily affected by changes in ambient light, screen reflections, and other factors, making it difficult to achieve high-precision and stable writing.

[0004] At the electronic document editing and annotation (software underlying level) level: Currently, handwritten annotation systems on the market mainly adopt two approaches. The first is "vector annotation layer overlay," which adds handwritten handwriting as an editable annotation object (such as PDF's / Annots object or Word's Shape graphics) to the document. This method has serious security risks because annotation objects have independent logical levels, and users can easily select, move, scale, or even directly delete the handwritten annotation through document editing software, which cannot meet the extremely high anti-tampering requirements of scenarios such as financial signing and government approval. The second approach is "rasterization and flattening," which renders the document along with the handwritten handwriting as a static image to replace the original page. This method erases the underlying vector text characteristics of the document, making the file unable to be OCR retrieved and automatically audited by the backend system, and the original handwritten traces often have problems such as jagged edges and blurriness due to screen resolution limitations.

[0005] Furthermore, images generated by traditional touchscreen handwriting often have uniform line thickness and lack the nuanced strokes (pressing and pausing) of real pen and paper writing, resulting in electronic signatures lacking biometric identification and raising questions about their legal validity. Additionally, embedding handwritten content into documents of different formats often requires writing complex low-level code injection logic for each format, leading to poor cross-platform compatibility.

[0006] To address the aforementioned issues, there is an urgent need for a non-touch writing system that is borderless in hardware, has strong anti-interference capabilities, is suitable for large screens, and can eliminate the easily tampered vector annotation layer mechanism in software. This system can seamlessly embed handwritten content into various documents as an immutable bitmap image, while preserving the document's underlying searchability and giving the handwritten image extremely high clarity and realistic pen strokes. Summary of the Invention

[0007] The purpose of this invention is to provide a system and method for creating writing trajectories for document editing on a non-touch display screen, in order to solve the problems of existing touch technology, such as reduced light transmittance, poor anti-interference ability, low positioning accuracy on large screens, and the problems of existing document annotation technology, such as vector annotations being easily tampered with, rasterization causing documents to lose their underlying retrieval capabilities, lack of realistic pen strokes in handwriting, and poor cross-platform compatibility.

[0008] To achieve the above objectives, the present invention provides the following technical solution: On one hand, the present invention provides a system (100) for forming writing traces by editing documents on a non-touch display screen, the system comprising: The display (10) has a plurality of dot matrix units (12) disposed on a glass substrate (11) facing the light-emitting side. The plurality of dot matrix units (12) are arranged in rows and columns throughout the glass substrate (11). Each dot matrix unit (12) is divided into rows and columns of dot matrix squares (13). Each dot matrix square (13) is provided with at least one coding point (14). The coding point (14) is a metal point attached to the glass substrate (11). The display (10) is used to display a target document (50) to be edited. An electronic pen (20) includes a pen tip (21), a camera module (22), and a data processing module (23). The camera module (22) is positioned to capture images of the pen tip (21) during writing. The data processing module (23) is used to analyze the captured images and calculate the global absolute physical coordinates of the pen tip (21) at different times based on the position coding points (141), forming a set of physical coordinates. The terminal processing device (30) is communicatively connected to the electronic pen (20) and includes a coordinate receiving and transformation unit (31), a dynamic pen stroke rendering and rasterization unit (32), and a cross-platform image embedding unit (33) that are interconnected. The coordinate receiving and conversion unit (31) is used to receive the physical coordinate set and perform coordinate system conversion to map the global absolute physical coordinates to the screen pixel coordinates within the target document display area; The dynamic stroke rendering and rasterization unit (32) is used to construct a handwriting layer with dynamic line width based on the screen pixel coordinates and writing speed, and to rasterize and crop the handwriting layer in a local area when lifting the pen to generate a handwritten image (60). The cross-platform image embedding unit (33) is used to call the document object model interface of the application to which the current target document (50) belongs, insert the handwritten image (60) as an independent image object and anchor it in the current page of the target document (50) without modifying the underlying vector encoding data of the target document (50).

[0009] Furthermore, the dynamic pen stroke rendering and rasterization unit (32) includes a speed calculation module (321) and a line width mapping module (322); the speed calculation module (321) is used to calculate the instantaneous movement speed of the pen tip (21) based on the timestamp interval and spatial distance of the screen pixel coordinates of two adjacent frames; the line width mapping module (322) has a built-in line width-speed inverse correlation mapping model. When the instantaneous movement speed increases, the corresponding generated handwriting trajectory line width decreases; when the instantaneous movement speed decreases, the corresponding generated handwriting trajectory line width increases, so as to simulate the real writing pen stroke.

[0010] Furthermore, the dynamic stroke rendering and rasterization unit (32) also includes a transparent overlay construction module (323) and a local rasterization module (324); the transparent overlay construction module (323) is used to create a transparent rendering layer covering the display area of ​​the target document (50) in the video memory of the terminal processing device (30), and the handwriting layer with dynamic line width is drawn on the transparent rendering layer in real time; the local rasterization module (324) is used to extract the outer rectangle boundary of the non-transparent pixels in the handwriting layer when a pen lifting event is detected, and only perform rasterization reading and image encoding on the pixel data within the outer rectangle boundary to generate the handwritten image (60) with a transparent background.

[0011] Furthermore, the terminal processing device (30) also includes a document interface adaptation unit (34), which has a pre-set component object model interface call mapping table for various document applications. When embedding an image, the cross-platform image embedding unit (33) obtains the image insertion method handle corresponding to the current target document (50) by querying the component object model interface call mapping table, and passes the handwritten image (60) stream data in memory as a parameter, so that the handwritten image (60) is embedded in the target document (50) as a floating image object.

[0012] Furthermore, the cross-platform image embedding unit (33) is also used to read the relative anchor coordinates of the current handwritten image (60) in the display area of ​​the target document (50), and call the application interface to set the locking mode of the handwritten image (60) to "move with text" or "locked to the absolute position of the page", so as to realize the layout of the handwritten image (60) and the underlying text.

[0013] Furthermore, the dynamic stroke rendering and rasterization unit (32) also includes a pixel-level anti-counterfeiting watermark fusion module (325), which is used to read the background pixel texture of the target document (50) below the bounding rectangle before rasterizing and cropping the handwritten image (60), and to perform bottom-level pixel fusion of the handwritten stroke pixels and the background pixel texture according to a preset weight, so that the generated handwritten image (60) has an anti-counterfeiting background consistent with the document background.

[0014] On the other hand, the present invention also provides a method for forming writing traces by editing documents on a non-touch display screen, the method employing the above-described system (100) and including the following steps: Write on the target document (50) displayed on the non-touch display (10) using the tip (21) of the electronic pen (20), and capture a dot matrix image of the area where the tip (21) is located during the writing process using the active infrared camera module (22) inside the electronic pen (20). The data processing module (23) inside the electronic pen (20) performs feature analysis on the dot matrix image, calculates the global absolute physical coordinates of the pen tip (21) at different times based on the position coding point (141), forms a set of physical coordinates and sends it to the terminal processing device (30); The coordinate receiving and conversion unit (31) of the terminal processing device (30) receives the set of physical coordinates, obtains the actual display pixel area of ​​the target document (50) on the screen, and maps the global absolute physical coordinates to the screen pixel coordinates within the display area of ​​the target document; Using the dynamic pen stroke rendering and rasterization unit (32) of the terminal processing device (30), a handwriting layer with dynamic line width is constructed based on the screen pixel coordinates and writing speed, and when a pen lifting event is detected, the handwriting layer is rasterized and cropped in a local area to generate a handwritten image (60). Using the cross-platform image embedding unit (33) of the terminal processing device (30), the document object model interface of the application to which the target document (50) belongs is called to insert the handwritten image (60) as an independent image object into the current page of the target document (50) without modifying the underlying vector encoding data of the target document (50).

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Unlimited cascading positioning capability and extreme anti-interference performance, breaking through physical size limitations: This invention relies on micro-nano metal dot arrays solidified in the display panel and infrared active vision recognition technology to directly encode absolute position information in a tiny area on the screen surface, completely decoupling the system's positioning accuracy from the screen size. Active infrared illumination, combined with a narrow-band filter, completely shields ambient visible light noise, achieving high signal-to-noise ratio imaging, enabling high-precision and stable writing even on large-size non-touch window terminals in strong light.

[0016] 2. Authentic Pen Stroke Reproduction and High-Resolution Lossless Image Generation: Unlike traditional touchscreens that display lines of varying thicknesses, this system calculates writing speed in real time and establishes a line width-speed inverse correlation mapping model. It then dynamically renders pen stroke trajectories with varying pressure and pauses on a transparent overlay. When the pen is lifted, a local minimum bounding rectangle rasterization technique is used to extract only the effective pen stroke area to generate a high-resolution PNG image, avoiding the memory consumption of full-screen rasterization. Even when magnified, the generated signature image retains smooth and clear edges, making it highly recognizable as a biometric feature for forensic identification.

[0017] 3. Cross-platform anti-tampering image embedding and underlying retrieval preservation: This invention abandons the mechanism of creating easily eraseable vector annotation objects and the destructive full-page rasterization / flattening mechanism. By calling the application's Document Object Model (DOM) interface, the handwritten image is inserted as an independent floating image object into the document page without modifying the underlying vector encoding data. This ensures that handwritten annotations cannot be easily selected and erased as vector objects (requiring a dedicated image erasure operation, thus enhancing the anti-tampering level) while preserving the OCR searchability of the underlying text, greatly improving cross-platform compatibility.

[0018] 4. Pixel-level anti-counterfeiting watermark fusion technology: Before generating the image, the system automatically reads the background pixel texture of the document's signature area and merges the handwritten black pixels with the background pixels according to an algorithm, so that the generated image has an anti-counterfeiting texture consistent with the document's background. Once the image is maliciously cut out or tampered with, the background texture is destroyed, thus giving the electronic document a very high level of anti-counterfeiting capability. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall architecture of the system (100) of the present invention; Figure 2 This is a schematic diagram of the system (100) module of the present invention; Figure 3 This is a schematic diagram showing the distribution and partial magnification of the dot matrix units (12) on the glass substrate (11) of the display (10) in this invention; Figure 4 This is a schematic diagram of the internal structure and infrared light path reflection principle of the electronic pen (20) of the present invention; Figure 5 This is a schematic diagram of the dynamic brush stroke rendering and rasterization unit (32) of the present invention; Figure 6 A flowchart of the method steps provided in the embodiments of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0021] like Figures 1 to 4 As shown, the present invention provides a system (100) for creating writing trajectories for editing documents on a non-touch display screen. This system fundamentally overturns the traditional design concept of touch screens that rely on sensing layers or infrared bezels. It adopts an innovative architecture of "micro-nano metal dot matrix solidified on display panel + active infrared pen tip occlusion visual recognition + decoupling of edge computing and rasterization rendering".

[0022] Specifically, the system (100) mainly includes a display (10), an electronic pen (20), and a terminal processing device (30).

[0023] The display (10) is the display output carrier and writing physical medium of the system. Traditional touch displays have resistive sensing films or capacitive sensing glass. However, the display (10) of the present invention does not have any sensing layer on the glass substrate (11) facing the light-emitting side, but directly has an encoding matrix composed of multiple dot matrix units (12).

[0024] Specifically, multiple dot matrix units (12) are arranged in rows and columns across the entire glass substrate (11), covering the entire display area. Each dot matrix unit (12) is virtually divided into rows and columns of dot matrix squares (13) (e.g., 5×5 or 10×10 as one dot matrix unit (12)), and each dot matrix square (13) contains at least one coded point (14). These coded points (14) are metal points attached to the glass substrate (11). The metal points can be made of aluminum, silver, copper, or their alloys, and are directly solidified onto the surface of the glass substrate (11) by magnetron sputtering, photolithography, or nanoscale inkjet printing processes, and then covered with standard liquid crystal display panel protective glass or polarizer.

[0025] Since the diameter of the metal dots is in the micrometer range (e.g., 10-50 micrometers), they are almost invisible to the naked eye at normal viewing distances. Therefore, they do not affect the light transmittance and display quality of the display (10) at all, thus achieving "invisible coding". The display (10) is used to display the target document (50) to be edited. The target document (50) can be a PDF, Word, Excel, HTML webpage or any file with a display interface.

[0026] To achieve global absolute positioning, at least one dot matrix square (13) in each dot matrix unit (12) is used to set a position coding point (141). The position coding point (141) is used to mark the absolute position of the current dot matrix unit (12) on the entire glass substrate (11), and this absolute position is recorded with the center of the glass substrate (11) as the origin of the coordinate system. Establishing a coordinate system with the center of the screen as the origin makes the four quadrants of the screen symmetrical, the coordinate data structure clear, and facilitates the processing of negative coordinates, which greatly facilitates matrix transformation operations such as mirroring and scaling of the display screen.

[0027] In the specific dot matrix layout, each dot matrix unit (12) includes a position encoding point (141), which is located in the upper left corner dot matrix square (13) of the dot matrix unit (12); each dot matrix square (13) of the remaining dot matrix squares (13) of the dot matrix unit (12) contains a regular encoding point (14), and all encoding points (14) are metal points. Fixing the position encoding point (141) in the upper left corner provides a stable "anchor point" for the image processing algorithm. In the image captured by the camera module (22), no matter how the electronic pen (20) rotates, as long as the position encoding point (141) in the upper left corner is included in the field of view, the algorithm can quickly perform image alignment and orientation correction.

[0028] The location coding point (141) adopts a highly innovative dual-point polar coordinate structure: it includes a first point (142) with the largest diameter and a second point (143) located in any one of the four directions above, below, left, and right of the first point (142); the diameter of the second point (143) is smaller than the diameter of the first point (142). The first point (142) serves as the anchor point (origin) of the polar coordinates, and the orientation of the second point (143) serves as the information carrier of the quadrant. The straight-line distance between the second point (143) and the first point (142) represents the distance of the current lattice unit (12) relative to the center point of the glass substrate (11) through calculation; the angle between the line connecting the second point (143) and the first point (142) and the x-axis represents the row or column in which the lattice unit (12) is located relative to the center of the glass substrate (11). Since the second point (143) is only in the four directions above, below, left, and right of the first point (142), this is equivalent to discretizing the angle of the polar coordinates into the reference directions of the four quadrants, resulting in extremely high information density.

[0029] The electronic pen (20) is the core data acquisition and processing terminal of the system. The electronic pen (20) includes a pen tip (21), a camera module (22), a data processing module (23), a first communication module (24), and a power management module.

[0030] The pen tip (21) is used for writing on the display (10), and its material needs to have two key characteristics: first, it needs to have a certain friction to simulate the real writing feel; second, it needs to have a strong absorption or blocking effect on infrared light. The pen tip (21) can be made of special silicone or polyoxymethylene (POM) material mixed with carbon black.

[0031] The camera module (22) is located inside the electronic pen (20) and faces the pen tip (21) to take pictures. The field of view (FOV) of the camera module (22) is carefully designed so that the range of the picture it captures is greater than at least one dot matrix unit (12). At the same time, the area covered by the pen tip (21) when writing is at least greater than one coding point (14) to ensure that sufficient occlusion features are generated for the algorithm to recognize.

[0032] The camera module (22) of the system includes an infrared generator (221) and an infrared camera (222). The infrared generator (221) emits infrared light toward the glass substrate (11), and the infrared camera (222) receives the infrared light reflected by the glass substrate (11). The infrared generator (221) is an infrared LED (e.g., with a wavelength of 850nm or 940nm). The infrared generator (221) and the infrared camera (222) are integrated together, and a narrowband filter is arranged in front of the lens after integration.

[0033] In this process, visible light from the environment (such as lamplight and sunlight) is filtered out by a narrow-band filter, allowing only infrared light of a specific wavelength to enter the photosensitive element. When the infrared generator (221) emits infrared light toward the shooting area, the metal dots (encoding dots (14)) attached to the glass substrate (11) contain a large number of free electrons. When the infrared generator (221) actively emits infrared light of a specific wavelength (such as 850nm or 940nm) to irradiate the surface of the metal dots, the alternating electric field of the infrared light will drive the free electrons in the metal to undergo forced vibration, forming a high-frequency oscillating current. According to electromagnetic theory, this oscillating current will re-radiate electromagnetic waves of the same frequency, i.e., strong reflection will occur, forming a bright white dot in the image. The encoding dots (14) that are blocked by the pen tip (21) are physically blocked by the pen tip (21) material, preventing the infrared light from reaching the metal dots below. Without incident light, the metal dots naturally cannot generate reflected light. Therefore, in the image captured by the infrared camera (222), the location of the obscured metal point appears as a distinct "black spot" or "hole". This active infrared imaging technology makes the "obscured coding point (14) location" in the captured image exhibit extreme contrast, completely eliminating the interference of ambient light changes and screen color changes on image recognition.

[0034] The data processing module (23) is the brain of the electronic pen (20), typically composed of a low-power DSP (Digital Signal Processor) or ARM microcontroller. Its core function is to perform real-time analysis of images captured by the camera module (22) and calculate the absolute coordinates of the pen tip (21). Its calculation logic is divided into two levels: The first level is "macro positioning": by identifying the location coding points (141) (first point (142) and second point (143)) in the image, the region code (absolute row and column number) of the current dot matrix unit (12) in the global coordinate system is parsed. Specifically, the algorithm scans the upper left corner dot matrix square (13) in the set of bright spots and identifies the first point (142) with the largest diameter and the second point (143) with the smaller diameter. The vector of the second point (143) relative to the first point (142) is calculated. If the vector direction is directly downward, it is determined that the dot matrix unit (12) is located below the center of the screen; by calculating the pixel distance between the two points and converting it into physical distance, the number of rows of the unit from the center is obtained by looking up the table. Furthermore, using the step value fault tolerance algorithm, the screen is set to have a rows, and there are a / 2 rows below. The minimum step value is (a / 2) / 90. Based on the degree of deviation of the vector angle from the reference angle, the step interval it falls into is determined, and the exact number of rows is corrected and determined. Similarly, if the vector direction is left / right, the column number is determined. Thus, the global absolute coordinates of the current dot matrix unit (12) with the screen center as the origin are obtained.

[0035] The second level is "micro-positioning": After determining the region code, the distribution of ordinary coding points (14) occluded by the pen tip (21) in the image is analyzed. Since each ordinary coding point (14) has a fixed relative coordinate within the dot matrix unit (12), the data processing module (23) calculates the local pixel coordinates of the center of the occluded region through the centroid algorithm or template matching algorithm, and maps them to the micro coordinates within the dot matrix unit (12).

[0036] Finally, the absolute coordinates of the region and the local micro coordinates are weighted and fused to calculate the high-precision global absolute physical coordinates of the pen tip (21) on the display (10). The data processing module (23) takes multiple pictures per second (e.g., no less than 25 frames or up to 120 frames), combines multiple coordinates at different times into a set of physical coordinates to represent the writing trajectory, timestamps them, and broadcasts them to the local area network via the UDP protocol through the first communication module (24).

[0037] This “decoupling of computation and rendering” architecture greatly saves wireless bandwidth. The electronic pen (20) completes all the heavy-load image computation and only sends lightweight coordinate data packets (such as X, Y coordinate values ​​and timestamps) to the network. This design greatly saves wireless bandwidth, so that even in high-concurrency scenarios where multiple electronic pens (20) are writing at the same time, there will be no network congestion, and the real-time synchronization of the writing trajectory is guaranteed.

[0038] The terminal processing device (30) (such as a personal computer, smart terminal, etc.) receives the set of physical coordinates sent by the electronic pen (20) and is responsible for subsequent coordinate transformation, graphics rendering, rasterization capture and document embedding. The terminal processing device (30) includes a coordinate receiving and transformation unit (31), a dynamic pen stroke rendering and rasterization unit (32), a cross-platform image embedding unit (33) and a document interface adaptation unit (34).

[0039] 1. Multi-level mapping and error elimination of coordinate receiving and transformation unit (31) Because the monitor (10) may have different resolutions and scaling settings, the global absolute physical coordinates sent by the electronic pen (20) cannot be directly used for document annotation and must undergo rigorous coordinate system transformation. The coordinate receiving and transformation unit (31) performs the following steps: First, a transformation from physical coordinates to screen pixel coordinates is performed. Let the physical width of the glass substrate (11) be W_phy and the height be H_phy; let the current screen resolution have horizontal pixel counts of Res_w and vertical pixel counts of Res_h. Then, the formulas for calculating the screen pixel coordinates (X_pixel, Y_pixel) corresponding to the physical coordinates (X_pen, Y_pen) are: X_pixel = (X_pen + W_phy ÷ 2) × (Res_w ÷ W_phy); Y_pixel = (H_phy ÷ 2 - Y_pen) × (Res_h ÷ H_phy). Among them, the physical coordinate Y-axis is upward and the screen pixel Y-axis is downward, so a subtraction flip is required.

[0040] Next, the actual display pixel area of ​​the target document (50) on the screen is obtained. The target document (50) may not be displayed in full screen, or it may be obscured by a toolbar. The system obtains the coordinates (Doc_X, Doc_Y) and its width and height of the client area rectangle of the display control of the target document (50) through the window handle of the operating system (such as HWND of Windows). The screen pixel coordinates are subtracted from the document area offset to obtain the screen pixel coordinates relative to the display area of ​​the target document. In order to eliminate the micro jaggedness caused by hand tremors or noise, the coordinate receiving and conversion unit (31) will also perform moving average filtering on the coordinate sequence.

[0041] 2. Core Algorithm of Dynamic Stroke Rendering and Rasterization Unit (32) This is the core module of the invention that achieves "realistic brushstrokes" and "efficient image generation." For example... Figure 5 The unit includes a speed calculation module (321), a linewidth mapping module (322), a transparent overlay construction module (323), a local rasterization module (324), and a pixel-level anti-counterfeiting watermark fusion module (325).

[0042] (1) Transparent overlay construction module (323): In the video memory of the terminal processing device (30), a transparent rendering layer is created over the display area of ​​the target document (50) using graphics APIs such as Direct2D or OpenGL. This layer is not mixed with the vector data of the underlying document and is only used to draw handwriting in real time to ensure "zero latency" visual feedback when writing.

[0043] (2) Speed ​​calculation module (321) and line width mapping module (322): In order to simulate the brush tip effect of a real pen or brush, the system cannot use a fixed line width. The speed calculation module (321) calculates the instantaneous moving speed V = ΔS / ΔT of the pen tip (21) based on the ratio of the spatial distance ΔS between the screen pixel coordinates of two adjacent frames to the timestamp interval ΔT.

[0044] The line width mapping module (322) has a built-in line width-speed inverse correlation mapping model. When the instantaneous movement speed V increases (indicating that the user is writing quickly, usually in connected strokes or fine lines), the corresponding generated handwriting trajectory line width W decreases; when the instantaneous movement speed V decreases (indicating that the user is pausing or starting and ending a stroke), the corresponding generated handwriting trajectory line width W increases. On the transparent rendering layer, with the current coordinate point as the center, dots or line segments with smooth anti-aliased edges are drawn according to the calculated target line width W. The graphics of consecutive frames overlap, thus forming a pen tip layer with rich variations in pressure and pause.

[0045] (3) Local rasterization module (324): When the user lifts the pen (the system detects, for example, no new coordinate input for 200ms) to complete a stroke, the vector graphics on the transparent rendering layer need to be converted into an image. If the entire screen is rasterized and cropped, it will consume a lot of memory and computing power. This invention adopts the "local minimum bounding rectangle rasterization" technology. The local rasterization module (324) traverses all non-transparent pixels on the transparent rendering layer, records the minimum value X_min and maximum value X_max of its horizontal coordinate, and the minimum value Y_min and maximum value Y_max of its vertical coordinate, and constructs the minimum bounding rectangle boundary surrounding all the handwriting. The memory space in the video memory is allocated with the same size as the minimum bounding rectangle boundary, and the pixel data (including the alpha transparency channel) located inside the boundary in the transparent rendering layer is copied to the memory space. Then, PNG format encoding compression is performed on these pixel data to generate a high-resolution handwritten image with a transparent background (60). This method makes the generated image size extremely small no matter how large the screen is, and perfectly preserves the clarity of the handwriting and the semi-transparent transition effect.

[0046] (4) Pixel-level anti-counterfeiting watermark fusion module (325): To prevent the generated handwritten image (60) from being maliciously cut out and copied to other documents, the system performs an anti-counterfeiting fusion operation before rasterization. The module reads the background pixel texture of the target document (50) below the boundary of the minimum bounding rectangle. The handwritten stroke pixels (usually black) are fused with the read background pixel texture according to a preset weight. For example, 10% of the background texture color is mixed into the pure black stroke pixels. In this way, the generated handwritten image (60) not only contains the handwriting, but also has an anti-counterfeiting texture consistent with the background of the current document's signature area. If the image is cut out and moved to a document with a solid color background, the difference in texture will immediately expose the fact that it has been tampered with.

[0047] After generating the handwritten image (60), it needs to be embedded into the target document (50). One innovation of this invention is that it does not modify the underlying vector encoding data of the target document (50), but instead inserts the handwritten image (60) as an independent floating image object.

[0048] The document interface adaptation unit (34) has a pre-set call mapping table for the component object model (COM) interface or automation interface of various document applications (such as Microsoft Word, Excel, Adobe Acrobat, WPS, web browser, etc.).

[0049] When embedding images, the cross-platform image embedding unit (33) first obtains the process instance of the application to which the target document (50) currently in the foreground belongs by using the operating system's window handle. Then, it queries the preset call mapping table to obtain the image insertion method handle corresponding to the current application.

[0050] For example, if the target document (50) is a Word document, the system obtains the Application object of the Word application, and then obtains the ActiveDocument object. The cross-platform image embedding unit (33) reads the relative anchor coordinates of the upper left corner of the current handwritten image (60) in the display area of ​​the target document (50), and converts the relative anchor coordinates into coordinates in the Word document's internal coordinate system (in points). Then, the ActiveDocument.Shapes.AddPicture method handle is called, passing the handwritten image (60) stream data and internal coordinates in memory as parameters. The system automatically sets the wrapping mode of the newly inserted Shape object to "in front of text", so that the handwritten image (60) accurately covers the position where the user just wrote.

[0051] If the target document (50) is a PDF file, the system calls the JavaScript API interface or plugin SDK of the PDF reader (such as Acrobat), uses the addAnnot method to create an image type annotation object, assigns the stream data of the handwritten image (60) to the appearance state (AP stream) of the object, and sets its position and immovable and deletable flags (although the underlying layer is an annotation, the presentation layer is no different from ordinary images, and the anti-tampering level is high).

[0052] If the target document (50) is a webpage (HTML), the system dynamically creates a new DOM tree in the webpage. The node converts the handwritten image (60) into Base64 encoding and uses it as the src attribute. Then, it uses CSS's absolute positioning (position:absolute) to position the image node at the specified coordinates in the document.

[0053] Through the above mechanism, the handwritten image (60) is anchored in the current page of the target document (50) as an independent image object. On the one hand, since the underlying text flow or vector path flow of the document is not directly modified, the original OCR searchable features and layout logic of the document are preserved; on the other hand, the handwritten content is fixed in the document as an image, and ordinary users cannot easily select and erase the signature like they would with vector lines, which greatly improves security and cross-platform compatibility. The cross-platform image embedding unit (33) also supports setting the locking mode of the handwritten image (60) to "move with text" or "locked to the absolute position of the page" to adapt to the layout following needs of users when editing the document later.

[0054] To make the technical solution of the present invention clearer, the following is an in-depth analysis of the complete workflow of the system (100) by taking a specific embodiment of handwriting signature and embedding a Word format approval form on a non-touch window terminal in a government affairs hall as an example.

[0055] like Figure 6 As shown, the method includes the following steps: Step S1: System initialization and physical interaction information acquisition.

[0056] At the counter in the government service hall, the clerk faces a 22-inch non-touch display (10). The glass substrate (11) of the display (10) is covered with a matrix of micron-sized metal dots (12). The display (10) shows a Word document (50) to be signed.

[0057] The clerk holds an electronic pen (20) and presses the pen tip (21) against the "applicant's signature" area of ​​the target document (50) on the display (10). The pen tip (21) is made of special silicone mixed with carbon black and physically covers at least one coding point (14) below it. The camera module (22) inside the electronic pen (20) works continuously at a high frame rate of 120fps, and the infrared generator (221) actively emits 850nm infrared light to the glass substrate (11) around the pen tip (21).

[0058] Infrared light is irradiated onto the micro-nano metal dot array on the surface of the glass substrate (11). The metal dots not blocked by the pen tip (21) strongly reflect the infrared light, penetrate the narrow-band filter and enter the infrared camera (222) to form bright white dots; the metal dots blocked by the pen tip (21) cannot reflect light due to the lack of incident light, forming dark areas in the image. The camera module (22) captures this high-contrast infrared grayscale image in real time and transmits it to the data processing module (23).

[0059] Step S2: Global absolute physical coordinate calculation and transmission.

[0060] After receiving the infrared image, the data processing module (23) first performs Gaussian filtering for noise reduction and adaptive threshold binarization, converting metallic points into white pixel blocks and background and occluded points into black. Then, it performs connected component detection to extract the bounding rectangles and centroid coordinates of all bright spots.

[0061] The algorithm searches for the top-left corner location encoding point (141) in the set of highlights, identifying the first point (142) with the largest diameter and the second point (143) with the smaller diameter. It calculates the vector of the second point (143) relative to the first point (142). Assuming the measured vector direction is directly downwards with an argument of 272 degrees, the system determines that the dot matrix unit (12) is located below the center of the screen. Using a step value fault-tolerant algorithm, the screen has 100 rows, with 50 rows below, and the minimum step value is 50 / 90≈0.555 degrees. 272 ​​degrees deviates from the reference angle of 270 degrees by 2 degrees, falling within the 3rd to 4th step interval. The system combines the measured physical distance between the two points and looks up the table to determine that the unit is located in the 4th row below the center of the screen. Similarly, the column number is determined to obtain the global absolute reference coordinates.

[0062] Subsequently, the algorithm analyzes the location of the dark area obscured by the pen tip (21) and calculates the local offset of the centroid of the dark area in the dot matrix (13). The global absolute reference coordinates are added to the local offset to output the high-precision global absolute physical coordinates (X_pen, Y_pen) of the pen tip (21) in the current frame. The data processing module (23) timestamps the coordinates of consecutive frames, encapsulates them into UDP packets, and broadcasts them to the local area network through the first communication module (24). The terminal processing device (30) receives this set of physical coordinates.

[0063] Step S3: Multi-level coordinate system transformation and screen pixel alignment.

[0064] After receiving the physical coordinates, the coordinate receiving and conversion unit (31) of the terminal processing device (30) performs the conversion. Assume that the screen resolution is 1920×1080, the physical width is 480mm, and the height is 270mm. The currently received physical coordinates are X_pen = 100mm, Y_pen = -50mm (with the center of the screen as the origin).

[0065] Calculate the screen pixel coordinates: X_pixel = (100 + 240) × (1920 ÷ 480) = 1360; Y_pixel = (135 - (-50)) × (1080 ÷ 270) = 185 × 4 = 740. That is, the current absolute pixel position of the pen tip on the screen is (1360, 740).

[0066] However, the Word window of the target document (50) is not full screen, and there is a toolbar at the top. The system obtains the client area rectangle of the Word document display control through the Windows API, assuming that the screen pixel coordinates of the upper left corner are (100, 200). Then the screen pixel coordinates relative to the target document display area are (1360-100, 740-200) = (1260, 540). The system performs a moving average filter on this coordinate sequence to eliminate hand tremor noise.

[0067] Step S4: Dynamic brush stroke rendering and local area rasterization capture.

[0068] The dynamic stroke rendering and rasterization unit (32) of the terminal processing device (30) takes over the work. The transparent overlay construction module (323) creates a transparent rendering layer overlaid on the Word window in the video memory. The speed calculation module (321) calculates the instantaneous movement speed V based on the coordinate difference and time difference between the current frame and the previous frame. Assuming that the current pen movement is fast, V is 800 pixels / second. The line width mapping module (322) draws an anti-aliased dot with a line width of 1.33mm at the (1260, 540) position of the transparent rendering layer. With the input of consecutive frames, the lines show the thickness changes due to the speed change, perfectly restoring the stroke of a real pen on paper.

[0069] When the clerk finishes signing and lifts the electronic pen (20), the system detects no new coordinates within 200ms and triggers the local rasterization module (324). The module traverses all non-transparent pixels on the transparent rendering layer, assuming the smallest bounding rectangle found is X_min=1200, X_max=1400, Y_min=500, Y_max=600. This rectangle is only 200×100 pixels in size. The system allocates 200×100 memory space in the video memory and copies the pixel data of this area.

[0070] Before compression encoding, the pixel-level anti-counterfeiting watermark fusion module (325) reads the background pixel texture (assumed to be a light gray grid pattern) below the rectangular area in the Word document. The handwritten black pixel value and the background pixel value are fused at the bottom layer pixel ratio with a weight of 9:1. Finally, PNG encoding is performed on the fused pixel data to generate a 200×100 pixel transparent background handwritten image (60) with anti-counterfeiting pattern, which is stored in memory.

[0071] Step S5: Cross-platform image embedding and anchoring.

[0072] The cross-platform image embedding unit (33) begins the embedding operation. The system obtains the process instance of the current foreground Word application through the window handle. It queries the mapping table in the document interface adaptation unit (34) to obtain the handle of the ActiveDocument.Shapes.AddPicture method.

[0073] The system reads the relative anchor coordinates (1260, 540) of the top left corner of the handwritten image (60) in the document display area and converts it into Word's internal coordinate system coordinates (assuming conversion to points, considering DPI scaling factor). The AddPicture method is called to pass the handwritten image (60) stream data and coordinate parameters from memory. Word automatically inserts the image at the specified position. The system then sets the WrapFormat.Type of the image object to wdWrapFront (floating above the text) through the COM interface and sets LockAnchor to true to ensure that the image is locked in the current paragraph and will not be easily misaligned due to subsequent text editing.

[0074] Thus, the clerk's handwritten signature is permanently and immutably embedded in the Word document as a high-resolution image with authentic pen strokes and anti-counterfeiting features. Since the underlying text data remains unchanged, the backend office system can still extract the text from the approval form for OCR retrieval, while the signature image serves as irrefutable evidence.

[0075] 1. Frame loss prevention and tiered fault tolerance mechanisms ensure writing continuity. In actual use cases in government service halls, clerks often sign quickly and hold pens in various ways, which can easily lead to distortion of the camera module (22)'s shooting angle or missing coding points (141) at some locations. This system introduces a dual fault tolerance mechanism in the data processing module (23).

[0076] First, there is a cross-image index fault-tolerant estimation mechanism. When the pen tip (21) occludes and causes the first point (142) or the second point (143) of the current position coding point (141) to be missing, the algorithm starts estimation. Using the position coding points (141) of the currently visible adjacent dot matrix units (12), based on the preset physical spacing and row-column recursive relationship, the missing data is estimated and the complete region code is restored.

[0077] Secondly, there is the step value fault tolerance algorithm. Due to the tilt of the shooting angle, the angle of the line connecting the second point (143) and the first point (142) may not be an absolute integer. The system divides the 90-degree area into several steps, each step representing a row or a column, and sets a fault tolerance range. For example, the step value is set to 1.8 degrees, and the tolerance is ±0.9 degrees. When the measured angle falls into the fault tolerance range of a certain step, the system directly determines it as the row / column corresponding to that step. This discretization process perfectly solves the problem of image distortion recognition caused by the tilt of the stylus, ensures the high robustness of coordinate calculation, and completely eliminates the phenomena of "frame skipping" and "pen breakage".

[0078] 2. Analysis of Anti-tampering and Anti-forgery Mechanisms for Rasterized Images Traditional vector annotation layers are easily tampered with because each annotation, as an independent logical object, has its own object number and selection handle; users can simply select and delete it. This invention rasterizes the handwritten trajectory into an image and embeds it. While the image can also be selected and deleted, its security level is fundamentally improved, and the system introduces multiple anti-counterfeiting mechanisms.

[0079] Firstly, pixel-level anti-counterfeiting watermark fusion. As described in the previous embodiments, the generated handwritten image (60) is not a completely transparent isolated handwriting, but is fused with the background pixels of the document signature area at the underlying level. If someone attempts to use image processing software to extract the signature and paste it into other documents, the original background texture attached to the edge of the signature will create a significant visual conflict with the background of the new document, thereby exposing the tampering behavior.

[0080] Secondly, image object anchoring and locking. When inserting an image via the COM interface, the system automatically sets the image's locking attribute. In applications like Word, a document protection mode can be set, allowing only form filling and disallowing modification of image objects. In PDFs, inserted image annotation objects can have an immovable flag set. This prevents ordinary users from directly selecting and erasing the signature in the document application interface without special permissions.

[0081] Third, it does not compromise the auditability of the underlying logic. Unlike destructive anti-tampering methods that rasterize and replace the entire page, this invention simply overlays the handwritten content as an image onto the vector text. When parsing the document, the backend auditing system can easily skip the image layer and directly extract and perform OCR retrieval of the underlying text. This "visual fusion, logical separation" design ensures both the indelibility of the signature and the preservation of the document's digital processing capabilities, greatly meeting the compliance requirements of government and financial scenarios.

[0082] 3. Ultra-low latency architecture with decoupling between asynchronous rendering and underlying writes. The system adopts an asynchronous decoupled design in its software architecture. During the writing process, the UI rendering thread (transparent overlay construction module (323)) directly receives the filtered coordinate stream and uses Direct2D to draw in the video memory in real time, achieving zero-latency responsiveness and ensuring that the clerk's writing experience is as smooth as writing on real paper.

[0083] The complex and time-consuming operations, such as local rasterization cropping, anti-counterfeiting watermark fusion, PNG encoding compression, and cross-platform COM interface calls, are all taken over by the background worker thread after the pen-lifting event is triggered. After the background thread completes the image generation and embedding, it notifies the UI thread to clear the temporary handwriting on the transparent overlay through a message mechanism. This architecture avoids UI lag during the writing process and ensures that the system can run smoothly even on low-configuration terminal processing devices (30).

[0084] Part 5: System Compatibility Examples in Different Document Formats The cross-platform image embedding unit (33) and document interface adaptation unit (34) of this invention enable the system to be widely compatible with various mainstream document formats. Several typical application scenarios are listed below: Example 1: Embedding handwritten images in a PDF file In bank face-to-face signing scenarios, tellers often need to annotate electronic contracts in PDF format. The system reads the PDF structure through underlying libraries such as MuPDF or PoDoFo, but does not modify its / Contents content stream, nor does it create traditional / Innots vector annotations. Instead, it inserts an image XObject at specified coordinates on the page through the PDF reader plugin API. This image XObject references the handwritten image (60) stream data in memory and sets its display matrix to precisely overlay it on the contract signature area. Because it is an image object, it cannot be tampered with by conventional PDF text editing tools, and the searchability of the underlying contract text is preserved.

[0085] Example 2: Embedding handwritten images in an Excel spreadsheet In a financial approval scenario, the approver needs to handwrite confirmation next to a specific cell in an Excel spreadsheet. The system obtains the Excel application object and locates the currently active worksheet. The position of the target cell is calculated based on the screen pixel coordinates. The Shapes.AddPicture method is called to insert the handwritten image (60). The system sets the image's Placement property to xlMoveAndSize, so that if the row height or column width changes subsequently, the signature image can automatically adjust its position and size with the cell, maintaining relative consistency in the layout.

[0086] Example 3: Embedding handwritten images in a webpage (HTML) form In the online government service hall, users need to provide an electronic signature after filling in information on a web form. The system takes over the webpage DOM through a browser extension or local proxy service. It obtains the DOM element position of the signature area on the current webpage. The handwritten image (60) is converted to a Base64 encoded string. A new signature is dynamically inserted into the webpage DOM. The `<signature>` tag is set to the Base64 string, and its `src` attribute is set to this string. Using CSS, `position: absolute; left: Xpx; top: Ypx;` it is precisely positioned above the signature area. When the user clicks "Submit Form," the front-end JavaScript script submits the Base64 data of the image along with the form data to the back-end server for storage, achieving high-fidelity handwritten signature embedding in a pure web environment.

[0087] In summary, the system and method for creating writing trajectories in document editing on a non-touch display screen provided by this invention successfully overcomes many shortcomings of existing touch and annotation technologies in government, finance, and other application scenarios through the innovative integration of technologies such as micro-nano metal dot matrix infrared active visual recognition, multi-level coordinate system precise transformation, speed-based dynamic pen stroke rendering, local minimum bounding rectangle rasterization extraction, and cross-platform document object model interface calls. While maintaining the ultimate visual experience of the display and precise positioning for large-size cascading, the system achieves high-fidelity, pen-stroke-enabled, tamper-proof image-level handwritten annotations that are seamlessly embedded across platforms without compromising the underlying vector searchability of the document. This has significant industrial application value and legal compliance implications.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A system (100) for generating writing traces through document editing on a non-touch display screen, characterized in that, The system includes: The display (10) has a plurality of dot matrix units (12) disposed on a glass substrate (11) facing the light-emitting side. The plurality of dot matrix units (12) are arranged in rows and columns throughout the glass substrate (11). Each dot matrix unit (12) is divided into rows and columns of dot matrix squares (13). Each dot matrix square (13) is provided with at least one coding point (14). The coding point (14) is a metal point attached to the glass substrate (11). The display (10) is used to display a target document (50) to be edited. An electronic pen (20) includes a pen tip (21), a camera module (22), and a data processing module (23). The camera module (22) is positioned to capture images of the pen tip (21) during writing. The data processing module (23) is used to analyze the captured images and calculate the global absolute physical coordinates of the pen tip (21) at different times based on the position coding points (141), forming a set of physical coordinates. The terminal processing device (30) is communicatively connected to the electronic pen (20), and includes a coordinate receiving and transformation unit (31), a dynamic pen stroke rendering and rasterization unit (32), and a cross-platform image embedding unit (33) that are interconnected. The coordinate receiving and conversion unit (31) is used to receive the physical coordinate set and perform coordinate system conversion to map the global absolute physical coordinates to the screen pixel coordinates within the target document display area; The dynamic stroke rendering and rasterization unit (32) is used to construct a handwriting layer with dynamic line width based on the screen pixel coordinates and writing speed, and to rasterize and crop the handwriting layer in a local area when lifting the pen to generate a handwritten image (60). The cross-platform image embedding unit (33) is used to call the document object model interface of the application to which the current target document (50) belongs, insert the handwritten image (60) as an independent image object and anchor it in the current page of the target document (50) without modifying the underlying vector encoding data of the target document (50).

2. The system (100) according to claim 1, characterized in that, The dynamic brush stroke rendering and rasterization unit (32) includes a speed calculation module (321) and a line width mapping module (322); The speed calculation module (321) is used to calculate the instantaneous movement speed of the pen tip (21) based on the timestamp interval and spatial distance between the pixel coordinates of two adjacent frames of the screen. The line width mapping module (322) has a built-in line width-speed inverse correlation mapping model. When the instantaneous movement speed increases, the line width of the corresponding handwritten trajectory decreases; when the instantaneous movement speed decreases, the line width of the corresponding handwritten trajectory increases, so as to simulate the real writing stroke.

3. The system (100) according to claim 2, characterized in that, The dynamic brush stroke rendering and rasterization unit (32) also includes a transparent overlay construction module (323) and a local rasterization module (324); The transparent overlay construction module (323) is used to create a transparent rendering layer in the video memory of the terminal processing device (30) that covers the display area of ​​the target document (50), and the handwriting layer with dynamic line width is drawn on the transparent rendering layer in real time. The local rasterization module (324) is used to extract the outer rectangle boundary of the non-transparent pixels in the transparent rendering layer when a pen lifting event is detected, and only perform rasterization reading and image encoding on the pixel data within the outer rectangle boundary to generate the handwritten image (60) with a transparent background.

4. The system (100) according to claim 1, characterized in that, The terminal processing device (30) further includes a document interface adaptation unit (34), which has a pre-set component object model interface call mapping table for various document applications. When embedding an image, the cross-platform image embedding unit (33) queries the component object model interface to call the mapping table, obtains the image insertion method handle corresponding to the current target document (50), and passes the handwritten image (60) stream data in memory as a parameter, so that the handwritten image (60) is embedded in the target document (50) as a floating image object.

5. The system (100) according to claim 4, characterized in that, The cross-platform image embedding unit (33) is also used to read the relative anchor coordinates of the current handwritten image (60) in the display area of ​​the target document (50), and call the application interface to set the locking mode of the handwritten image (60) to "move with text" or "lock to the absolute position of the page", so as to realize the layout of the handwritten image (60) and the underlying text.

6. The system (100) according to claim 1, characterized in that, The dynamic stroke rendering and rasterization unit (32) also includes a pixel-level anti-counterfeiting watermark fusion module (325), which is used to read the background pixel texture of the target document (50) below the bounding rectangle before rasterizing and cropping the handwritten image (60), and to perform bottom-level pixel fusion of the handwritten stroke pixels and the background pixel texture according to a preset weight, so that the generated handwritten image (60) has an anti-counterfeiting background consistent with the document background.

7. A method for forming writing trajectories by editing documents on a non-touch display screen based on the system according to any one of claims 1 to 6, characterized in that, The method includes the following steps: Write on the target document (50) displayed on the non-touch display (10) using the tip (21) of the electronic pen (20), and capture a dot matrix image of the area where the tip (21) is located during the writing process using the active infrared camera module (22) inside the electronic pen (20). The data processing module (23) inside the electronic pen (20) performs feature analysis on the dot matrix image, calculates the global absolute physical coordinates of the pen tip (21) at different times based on the position coding point (141), forms a set of physical coordinates and sends it to the terminal processing device (30); The coordinate receiving and conversion unit (31) of the terminal processing device (30) receives the set of physical coordinates, obtains the actual display pixel area of ​​the target document (50) on the screen, and maps the global absolute physical coordinates to the screen pixel coordinates within the display area of ​​the target document; Using the dynamic pen stroke rendering and rasterization unit (32) of the terminal processing device (30), a handwriting layer with dynamic line width is constructed based on the screen pixel coordinates and writing speed, and when a pen lifting event is detected, the handwriting layer is rasterized and cropped in a local area to generate a handwritten image (60). Using the cross-platform image embedding unit (33) of the terminal processing device (30), the document object model interface of the application to which the target document (50) belongs is called to insert the handwritten image (60) as an independent image object into the current page of the target document (50) without modifying the underlying vector encoding data of the target document (50).

8. The method according to claim 7, characterized in that, The process involves creating a handwriting layer with dynamic line widths based on writing speed, specifically including: Calculate the ratio of the spatial distance between screen pixel coordinates of two adjacent frames to the timestamp interval to obtain the instantaneous movement speed V; Calculate the target line width W at the current coordinate point based on the preset line width-velocity inverse correlation mapping model; On the transparent rendering layer, with the current coordinate point as the center, draw circles or line segments with smooth edges according to the target line width W. The circles or line segments of consecutive frames overlap to form a stroke layer with a brush stroke effect.

9. The method according to claim 8, characterized in that, Local area rasterization cropping to generate handwritten image (60), specifically including: Iterate through all non-transparent pixels on the transparent rendering layer, record the minimum and maximum values ​​of their horizontal and vertical coordinates, and construct the minimum bounding rectangle that encloses all the handwriting. Allocate memory space in the video memory that is the same size as the boundary of the minimum bounding rectangle; The pixel data located inside the boundary of the minimum bounding rectangle in the transparent rendering layer is copied to the memory space, and the pixel data is compressed in PNG format to generate a bitmap file with a transparent background as a handwritten image (60).

10. The method according to claim 7, characterized in that, Inserting and anchoring the handwritten image (60) as an independent image object into the target document (50) includes: Obtain the process instance of the application to which the target document (50) currently in the foreground belongs by the window handle of the operating system; Query the preset component object model interface call mapping table to obtain the handle of the image insertion method corresponding to the current application; Obtain the relative anchor coordinates of the upper left corner of the handwritten image (60) in the display area of ​​the target document (50), and convert the relative anchor coordinates into coordinates in the document's internal coordinate system; Call the image insertion method handle, pass the stream data of the handwritten image (60) and the coordinates of the document's internal coordinate system as parameters, so that the handwritten image (60) is embedded in the document page as a floating image object; Setting the wrapping mode of the floating image object to "floating above the text" anchors the image while keeping the underlying text and vector-encoded data of the document unchanged.

Images