A non-touch display-based pdf underlying data handwritten modification method and system

Abstract

The application discloses a PDF underlying data handwriting modification method and system based on a non-touch display, and is suitable for tamper-proof signature in the fields of banks and the like. The system comprises a display with metal dot arrays and an electronic pen with an infrared camera module. The method comprises the following steps: writing on the non-touch display by using the electronic pen, and actively shooting the dot array image of the pen tip by using the infrared camera module; solving global physical coordinates based on position coding points; mapping the physical coordinates into PDF page coordinates; performing down-sampling and curve fitting on the coordinates to generate a PDF path content flow instruction; and analyzing the underlying structure of the PDF, abandoning the traditional annotation layer mode which is easy to be erased, and directly injecting the instruction into the / Contents content flow of the page and saving. The application realizes high-precision non-touch positioning, permanently fuses the signature as a vector path into the underlying layer of the PDF, and cannot be individually selected or erased, thereby solving the problems of easy tampering or rasterization distortion of the traditional electronic signature, and giving the financial archives high security.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction and electronic document processing technology. Specifically, it relates to a non-touch display writing system based on micro-nano metal dot matrix and infrared active visual recognition, and a method for modifying the underlying data (page content stream) of PDF format electronic documents in the banking and other financial fields using this system to perform non-erasable vector handwritten signature modification. Background Technology

[0002] With the deepening digitalization of banking operations, paperless office practices and compliant electronic record archiving have become essential requirements for the financial industry. In business scenarios such as account opening, loan approval, and wealth management contract signing, customers must sign electronic contracts displayed on electronic screens. This scenario places extremely high demands on the legal validity of signatures, tamper-proof capabilities, and long-term retrieval of records.

[0003] Existing bank electronic signature interaction systems and document modification technologies have the following significant drawbacks: At the underlying hardware interaction level: Physical size and durability limitations: Bank counters typically use large interactive displays or window-based interactive terminals. Traditional infrared bezel touchscreens rely on an infrared grid emitted from the edge; the larger the screen, the more emitters are required, and the bezel is easily damaged in the high-frequency use environment of banks; the divergence of light over long distances can lead to a sharp decrease in accuracy.

[0004] Impaired light transmittance and display quality: Capacitive or resistive screens require an additional sensing layer on the screen surface. These sensing layers reduce the light transmittance of the display, affecting the clear display of small clauses in bank contracts.

[0005] Poor resistance to ambient light interference: Bank branches are usually brightly lit and have complex lighting conditions. Traditional camera optical positioning systems based on visible light are easily affected by changes in ambient light, screen color content, customer shadows, and other factors, which can lead to failure in image feature extraction and result in broken lines or garbled text.

[0006] At the level of electronic document signature modification (software underlying layer): Currently, the technical approaches used by bank electronic signature systems to process signatures after a customer completes the signature on the screen are mainly as follows: Erasable annotation overlay (traditional mainstream method): Handwritten notes are added as annotation objects to the PDF (such as / Ink annotations, / FreeText annotations, or stamps) to the PDF's / Annots dictionary, overlaid on top of the page. This method poses extremely serious security risks: annotations are essentially independent, manipulable layer objects. Many PDF editing software or lightweight readers circulating internally in banks allow users to directly click and select the annotation object to erase, move, scale, or even copy and paste it onto other contracts. This "erasable signature editing" feature completely undermines the uniqueness and seriousness of electronic signatures, easily leading to financial fraud disputes and lacking true legal effect.

[0007] Rasterization and flattening (destructive processing): To prevent annotations from being erased, some systems resort to extreme measures, rendering the PDF page along with the handwritten text into a static bitmap image (JPG or PNG) and replacing the original page. The fatal flaw of this method is that it erases the vector characteristics of the text within the PDF. The massive amounts of contracts in bank archives will no longer be able to be used by the backend system for OCR text retrieval, keyword searches, and automated compliance audits; furthermore, signatures become severely blurred and distorted when enlarged, failing to meet the requirements for forensic identification.

[0008] To address the aforementioned issues, there is an urgent need for a non-touch writing system that is borderless in hardware, has strong anti-interference capabilities, is suitable for large-screen terminals in banks, and can completely eliminate the "erasable annotation layer" mechanism in software, directly writing the customer's handwritten signature as a vector path into the underlying content stream of the PDF file, placing it at the same logical level as the original contract text, and achieving permanent integration that cannot be selected or erased individually. Summary of the Invention

[0009] This invention aims to solve the problems in the prior art where writing accuracy is limited by size, anti-interference ability is poor, and handwritten signatures in bank electronic archives PDFs are easily erased or tampered with as annotation objects, or rasterized, causing the text to lose its retrieval ability. It provides a method and system for handwritten modification of PDF underlying data based on a non-touch display.

[0010] In one aspect, this invention provides a method for handwritten modification of underlying PDF data based on a non-touch display. This method relies on an independent non-touch display writing system and includes the following steps: S1: Use an electronic pen to write on a non-touch display. Use the active infrared camera module inside the electronic pen to capture a dot matrix image of the pen tip area during the writing process to collect physical interaction information. S2: The data processing module inside the electronic pen performs feature analysis on the dot matrix image, calculates the global absolute physical coordinates of the pen tip at different times based on the position coding points, and realizes the extraction of the physical coordinate set of the pen tip's motion trajectory. S3: Utilize the terminal processing device to receive the set of physical coordinates and perform coordinate system transformation to map the global absolute physical coordinates to the target coordinates under the PDF page coordinate system, thereby achieving precise alignment from physical space to the digital space of electronic archives. S4: Use a terminal processing device to perform vector downsampling and curve fitting on the target coordinate set, generate a sequence of Bézier curve control points, and generate a path content stream drawing instruction that conforms to the PDF specification based on the control point sequence, thereby realizing the conversion from discrete coordinates to continuous vector mathematical curves. S5: Utilize the terminal processing device to parse the underlying structure of the target PDF file and locate the content flow object of the current page. Inject the path content flow drawing instructions into the content flow object, update the PDF file structure and save it, so that the handwritten signature is permanently and indelibly integrated into the underlying content flow of the PDF as a vector path.

[0011] In another aspect of this invention, a PDF underlying data handwriting modification system based on a non-touch display is provided, comprising: The display has multiple dot matrix units arranged in rows and columns on its glass substrate, and each dot matrix contains metal coded dots. An electronic pen includes a pen tip, a camera module, and a data processing module; the camera module is positioned to capture images of the pen tip and contains an infrared generator and an infrared camera. The terminal processing device, which is communicatively connected to the electronic pen, includes a coordinate transformation unit, a vector fitting unit, and a PDF underlying injection unit.

[0012] This invention addresses the scenario of electronic document signature in banks and has the following significant advantages: Unlimited cascading positioning capability that breaks through physical size limitations: The independent hardware system upon which this invention relies directly encodes absolute position information into every tiny area of ​​the screen surface, with each dot matrix unit being an independent absolute coordinate anchor point. The system's positioning accuracy is completely decoupled from the screen size, achieving true "unlimited cascading precise positioning," perfectly suited to the needs of large-screen interactive terminals in banks.

[0013] Extreme resistance to ambient light interference and high signal-to-noise ratio imaging: Utilizing active infrared illumination combined with narrowband filter technology, the system completely shields the bright and complex visible light noise of the bank lobby. The strong reflective properties of the metal dots on specific infrared bands, combined with the strong blocking properties of the light-absorbing pen tip, result in captured images exhibiting near-black-and-white binarization characteristics, significantly reducing the computational power consumption of the data processing module.

[0014] The coding scheme features ultra-high information density and extremely low visual interference: It adopts polar coordinate position coding points with "large dots + small dots". The diameter of the metal dots is on the micrometer level, which is invisible to the naked eye. It completely preserves the high light transmittance of the display and ensures that the small clauses in the bank contract are clearly visible, thus achieving "invisible coding".

[0015] Completely eliminates the possibility of "erasable signature editing" with financial-grade anti-tampering capabilities (core advantage): Unlike the / Annots annotation layer overlay scheme widely used in existing banking systems, this invention abandons the mechanism of creating independent annotation objects. The customer's handwritten signature is injected directly into the page's / Contents content stream via path drawing instructions after parsing the PDF's underlying structure. The signature handwriting and the original contract text are at the same logical level, without independent object numbers, and are permanently bonded together. No conventional PDF editing tool can select, move, edit, or erase this signature separately, completely eliminating the risk of financial fraud and repudiation, giving electronic documents the same legal effect as handwritten paper documents.

[0016] Vector-based lossless mapping and full-text searchability in the archival system: By downsampling coordinate points and fitting Bézier curves, discrete points are transformed into smooth mathematical curves and written into PDFs. No matter how many times the contract is enlarged for forensic authentication, signatures remain clear and sharp. Simultaneously, because no rasterization or flattening is applied, the original vector text characteristics of the contract are fully preserved, allowing the bank's back-end system to still perform efficient OCR retrieval and automated risk control audits of archived contracts.

[0017] Highly robust fault-tolerant reading and anti-frame-dropping mechanism: It introduces a cross-image index fault-tolerant calculation mechanism based on mathematical correlation and a step value fault-tolerant algorithm, which completely solves the problems of "lost points" and "frame skipping" when customers quickly sign their names on the bank's large screen.

[0018] Ultra-low latency architecture with decoupling between asynchronous rendering and underlying writing: During the writing process, the UI layer renders visual feedback in real time through the graphics API to achieve zero-latency responsiveness and ensure the user's writing experience; at the moment the pen is lifted, a background thread is triggered to perform complex PDF underlying parsing, decompression, injection, and compression and saving operations, and a double-buffered atomic file replacement mechanism is adopted to ensure both responsiveness and the stable modification and absolute security of the underlying data of the bank's core business system. Attached Figure Description

[0019] Figure 1 This is an overall architecture diagram of the non-touch display writing system of the present invention; Figure 2 This is a schematic diagram of the non-touch display writing system of the present invention; Figure 3 This is a schematic diagram of the dot matrix unit distribution on the glass substrate of a display. Figure 4 A schematic diagram of the internal structure and infrared light path reflection principle of an electronic pen; Figure 5 This is a flowchart of the non-touch display writing method of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] 1. In-depth analysis of the system hardware architecture like Figures 1 to 3 As shown, an embodiment of the present invention provides a PDF underlying data handwriting modification system based on a non-touch display, including a display (10), an electronic pen (20), and a terminal processing device (30). The display (10) is used to display the PDF file stored in the terminal processing device (30).

[0023] The display (10) has multiple dot matrix units (12) arranged in rows and columns on the glass substrate (11) facing the light-emitting side. Each dot matrix unit (12) is logically divided into dot matrix squares (13) arranged in rows and columns. A position coding point (141) is set in the upper left corner dot matrix square (13) of each dot matrix unit (12), and each square in the remaining dot matrix squares (13) contains a general coding point (14). All coding points (14) are micron-sized metal points. The display (10) has a second communication module (15) inside.

[0024] The electronic pen (20) includes a pen tip (21), a camera module (22), a data processing module (23), and a first communication module (24). The camera module (22) is positioned to capture images of the pen tip (21). The data processing module (23) is a low-power MCU (such as the STM32H7 series) or DSP used to perform image analysis and coordinate calculation.

[0025] The electronic pen (20) is the core data acquisition and processing terminal of the system. The electronic pen (20) includes a pen tip (21), a camera module (22), a data processing module (23), a first communication module (24), and a power management module.

[0026] The pen tip (21) is used for writing on the display (10), and its material needs to have two key characteristics: first, it needs to have a certain friction to simulate the real writing feel; second, it needs to have a strong absorption or blocking effect on infrared light. The pen tip (21) can be made of special silicone or polyoxymethylene (POM) material mixed with carbon black.

[0027] The camera module (22) of the system includes an infrared generator (221) and an infrared camera (222). The infrared generator (221) emits infrared light toward the glass substrate (11), and the infrared camera (222) receives the infrared light reflected by the glass substrate (11). The infrared generator (221) is an infrared LED (e.g., with a wavelength of 850nm or 940nm). The infrared generator (221) and the infrared camera (222) are integrated together, and a narrowband filter is arranged in front of the lens after integration.

[0028] In this process, visible light from the environment (such as lamplight and sunlight) is filtered out by a narrow-band filter, allowing only infrared light of a specific wavelength to enter the photosensitive element. When the infrared generator (221) emits infrared light toward the shooting area, the metal dots (encoding dots (14)) attached to the glass substrate (11) strongly reflect the infrared light due to the resonance effect of the free electrons in the metal, forming bright white dots in the image. The encoding dots (14) that are blocked by the pen tip (21) cannot produce reflected light because the pen tip (21) material absorbs or blocks the incident light, and appear as dark spots or black dots in the image. This active infrared imaging technology enables the "blocked encoding dots (14)" to exhibit extreme contrast in the captured image, completely eliminating the interference of changes in ambient light and screen color on image recognition.

[0029] The data processing module (23) is the brain of the electronic pen (20), and is usually composed of a low-power DSP (digital signal processor) or ARM microcontroller. Its core function is to analyze the images captured by the camera module (22) in real time and calculate the absolute coordinates of the pen tip (21).

[0030] Its computational logic is divided into two levels: The first level is "macro positioning": by identifying the location coding points (141) (first point (142) and second point (143)) in the image, the region code (absolute row and column number) of the current lattice unit (12) in the global coordinate system is parsed.

[0031] The second level is "micro-positioning": After determining the region code, the distribution of ordinary coding points (14) occluded by the pen tip (21) in the image is analyzed. Since each ordinary coding point (14) has a fixed relative coordinate within the dot matrix unit (12), the data processing module (23) calculates the local pixel coordinates of the center of the occluded region through the centroid algorithm or template matching algorithm, and maps them to the micro coordinates within the dot matrix unit (12).

[0032] Finally, the absolute coordinates of the region and the local micro coordinates are weighted and fused to calculate the high-precision absolute coordinates of the pen tip (21) on the display (10). The data processing module (23) takes multiple pictures per second (e.g., no less than 25 frames or up to 120 frames) and combines the multiple coordinates at different times into a set of coordinates to represent the writing trajectory.

[0033] The electronic pen (20) contains a first communication module (24), which supports wireless communication protocols such as Wi-Fi, Bluetooth (BLE), or Ultra Wideband (UWB). The first communication module (24) is connected to a data network to transmit the processed coordinate set with extremely low latency.

[0034] The communication feature of this system is "decoupling of computation and rendering". The electronic pen (20) completes all the heavy-load image computation and only sends lightweight coordinate data packets (such as X, Y coordinate values ​​and timestamps) to the network. This design greatly saves wireless bandwidth, so that even in high-concurrency scenarios where multiple electronic pens (20) are writing at the same time, there will be no network congestion, and the real-time synchronization of the writing trajectory is guaranteed.

[0035] The terminal processing device (30) includes a coordinate transformation unit (31), a vector fitting unit (32), and a PDF underlying injection unit (33), which are implemented through a bank counter application at the software level.

[0036] 2. Physical mechanisms of active infrared imaging and high signal-to-noise ratio image acquisition Step S1: Use the electronic pen (20) to write on the non-touch display (10), and use the active infrared camera module (22) inside the electronic pen (20) to capture the dot matrix image of the area where the pen tip (21) is located during the writing process, so as to realize the collection of physical interaction information.

[0037] This step is the entry point for the system to obtain interactive information from the physical world. In the bank counter scenario, the customer uses the tip (21) of the electronic pen (20) to write on the signature area of ​​the account opening contract displayed on the monitor (10). The material of the pen tip (21) needs to have a strong absorption or blocking effect on infrared light, and is usually made of conductive rubber with added carbon black or special soft plastic.

[0038] The camera module (22) inside the electronic pen (20) takes pictures towards the pen tip (21). In terms of imaging principle, the infrared generator (221) of the camera module (22) actively emits infrared light of a specific wavelength (such as 850nm or 940nm) to illuminate the shooting area. There are a large number of free electrons inside the metal dots on the glass substrate (11). Under the alternating electric field of a specific infrared wavelength, they undergo forced vibration and re-radiate electromagnetic waves of the same frequency, forming a strong reflection; while the coding point (14) blocked by the pen tip (21) made of light-absorbing material physically blocks the incident light and cannot generate reflected light.

[0039] Therefore, the unobstructed coding point (14) reflects infrared light to the infrared camera (222) of the imaging module (22), forming a bright white point, while the obstructed coding point (14) appears as a dark point in the image. The front end of the infrared camera (222) is equipped with a narrow-band filter, whose bandpass center wavelength is consistent with that of the infrared generator, and its half-bandwidth is extremely narrow (e.g., ±5nm), allowing only infrared light of this specific wavelength to pass through, completely shielding the strong visible light noise interference in the bank's business hall, thus forming an imaging model with an extremely high signal-to-noise ratio. The imaging module (22) continuously captures images at a high frame rate (e.g., 120fps) and transmits them to the data processing module (23).

[0040] 3. Global Absolute Physical Coordinate Calculation Mechanism Based on Polar Coordinates and Fault-Tolerant Algorithms Step S2: Use the data processing module (23) inside the electronic pen (20) to perform feature analysis on the dot matrix image, and calculate the global absolute physical coordinates of the pen tip (21) at different times based on the position coding point (141), so as to extract the set of physical coordinates of the pen tip (21) motion trajectory.

[0041] This step is performed in the data processing module (23) of the electronic pen (20) to convert the infrared image into precise physical coordinates.

[0042] S21: Image binarization and connected component extraction. The data processing module (23) uses the high-contrast image obtained by infrared active illumination and performs adaptive threshold binarization using an algorithm to separate the metal points from the background. Then, connected component extraction (Two-Pass algorithm) is performed to filter out noise points that are too large or too small and extract the effective coded point contours.

[0043] S22: Location-coded point identification. The algorithm searches for the larger point (first point (142)) and the smaller point (second point (143)) at the top left corner of the current lattice element (12) in the connected domain. The first point (142) has the largest diameter and serves as the pole (origin) of the polar coordinates. The second point (143) is located in any of the four quadrants with the first point (142) as the origin. The modulus and argument of the line connecting the two points are calculated.

[0044] S23: Ladder value row and column determination (macro positioning). Since the shooting angle of the camera module (22) may be slightly tilted, the actual included angle is often not an absolute integer. The system introduces a very robust ladder fault tolerance algorithm: set the display (10) to have a rows, and a / 2 rows on the center side, and use 90 / (a / 2) degrees as a minimum ladder value to divide the angle fault tolerance interval. For example, set a=100, a / 2=50, and the ladder value is approximately 1.8 degrees. When the measured included angle falls into a certain ladder interval, the system determines it to be the row number corresponding to the center point on that side; the column direction is similarly set to column b for determination. The distance between the second point (143) and the first point (142) is calculated to represent the distance of the current dot matrix unit (12) relative to the center point of the glass substrate (11). If the pen tip (21) obstructs and causes some position coding points (141) to be missing, the cross-image index fault tolerance mechanism is activated, and the data of the adjacent dot matrix unit (12) is used to calculate and complete the missing points, and the exact global reference coordinates are locked.

[0045] S24: Local micro-coordinate calculation (micro-positioning). After the global reference coordinates are established, the distribution of the dark area occluded by the pen tip (21) in the dot matrix grid (13) is analyzed. The grid where the dark area is located has a fixed row and column index within the dot matrix unit (12). By calculating the offset of the centroid of the dark area relative to the center of the grid, the sub-pixel level local offset is obtained. The sub-pixel local offset is the position coordinate in a dot matrix unit (12).

[0046] S25: Coordinate Fusion and Output. The global absolute reference coordinates and local offsets are added and fused to output the final high-precision absolute physical coordinates (X_pen, Y_pen). These coordinates are based on the center of the glass substrate (11) and are in micrometers (μm) or millimeters. The electronic pen (20) timestamps the coordinates calculated for each frame and broadcasts them to the bank's local area network via the first communication module (24) using the UDP protocol. The terminal processing device (30) receives the coordinate stream.

[0047] Part Two: PDF Underlying Data Modification Method Based on Physical Coordinate Flow This application, as a separate application, completes the permanent fusion of PDF underlying data on the terminal processing device (30) side.

[0048] 4. High-precision mapping and mathematical conversion model for multi-level coordinate systems Step S3: Receive the physical coordinate set using the terminal processing device (30) and perform coordinate system transformation to map the global absolute physical coordinates to the target coordinates under the PDF page coordinate system, thereby achieving precise alignment from physical space to digital space of electronic archives.

[0049] After receiving the absolute physical coordinates sent by the electronic pen (20), the terminal processing device (30) cannot directly use them for PDF modification; it must undergo two coordinate system transformations. To enable software engineers to implement this accurately, step S3 also includes: S31: Physical coordinates to screen pixel coordinates conversion.

[0050] Since the display (10) may have different resolutions and scaling settings, it is first necessary to convert the physical coordinates based on the center of the glass substrate (11) into screen display pixel coordinates.

[0051] Let the physical width of the glass substrate (11) be W_phy and the height be H_phy; let the current screen resolution have horizontal pixels of Res_w and vertical pixels of Res_h.

[0052] The formula for calculating the screen pixel coordinates (X_pixel, Y_pixel) corresponding to the physical coordinates (X_pen, Y_pen) is as follows: X_pixel = (X_pen + W_phy ÷ 2) × (Res_w ÷ W_phy) Y_pixel = (H_phy ÷ 2 - Y_pen) × (Res_h ÷ H_phy) The physical Y-axis points upwards, while the screen pixel Y-axis points downwards, so a subtraction flip is required.

[0053] S32: Screen pixel coordinates to PDF page coordinates conversion.

[0054] As per the PDF specification (ISO 32000), the origin of the PDF coordinate system is at the bottom left corner of the page, and the unit is points, with 1 inch = 72 points. The terminal processing device (30) reads the currently opened bank electronic file PDF, obtains the MediaBox (page size, in PDF points) of the current page, and sets it as PageWidth_pdf and PageHeight_pdf.

[0055] Get the actual display area of ​​the PDF on the monitor screen. Let the screen pixel coordinates of the top-left corner of the PDF display area be (Disp_X, Disp_Y), and the display scaling factor be Zoom.

[0056] First, calculate the offset pixels of the screen pixels relative to the top left corner of the PDF display area: dx = X_pixel - Disp_X dy = Y_pixel - Disp_Y Next, calculate the screen DPI (dots per inch). Assume the operating system DPI is set to 96 DPI.

[0057] The formula for calculating the scale conversion factor is: ScaleFactor = 72 ÷ Screen_DPI At this point, the formula for calculating the target coordinates (X_pdf, Y_pdf) in the PDF coordinate system is: X_pdf = dx × ScaleFactor ÷ Zoom Y_pdf = PageHeight_pdf - (dy × ScaleFactor ÷ Zoom) Flipping: The Y-axis of a PDF is upward, while the Y-axis of the screen is downward. The value must be calculated by subtracting the screen Y-offset from the page height. Through this mathematical conversion, every swipe a customer makes on the physical screen is precisely mapped to an absolute coordinate point within the bank's electronic document PDF page.

[0058] To further eliminate microscopic jagged edges caused by hand tremors or sampling noise from the electronic pen, the terminal processing device will perform a moving average filter on the (X_pixel, Y_pixel) sequence before executing S32. The filtered and smoothed coordinate sequence will then be fed into subsequent steps.

[0059] 5. Vector downsampling algorithm and cubic Bézier curve mathematical fitting Step S4: Use the vector fitting unit (32) in the terminal processing device (30) to perform vector downsampling and curve fitting on the target coordinate set, generate a Bezier curve control point sequence, and generate a path content flow drawing instruction that conforms to the PDF specification based on the control point sequence, thereby realizing the conversion from discrete coordinates to continuous vector mathematical curves.

[0060] Because the electronic pen (20) samples at a high frame rate of 120Hz, 120 coordinate points are generated per second of writing. If all these points are written into the PDF in coordinate form, the file size will increase dramatically, and the lines will appear jagged. Therefore, downsampling and curve fitting must be performed to generate the instructions for route C (PDF free vector drawing). This step S4 specifically includes: S41: Douglas-Puk algorithm downsampling.

[0061] The vector fitting unit (31) of the terminal processing device (30) performs the DP algorithm on the received coordinate point sequence P = {p1, p2,..., pn}: (1) Draw a straight line AZ between the two points A and Z at the beginning and end of the curve; (2) Calculate the distance from all other points on the curve to the straight line AZ; (3) Find the point Pmax with the largest distance. If its distance is greater than a preset threshold (e.g., 0.5 PDF points); (4) Then, using Pmax as the dividing point, the original curve is divided into two segments, APmax and PmaxZ, and the above steps are executed recursively. (5) If the maximum distance is less than the threshold, discard all intermediate points and keep only A and Z.

[0062] Through this step, hundreds of points are compressed into dozens of feature inflection points.

[0063] Depth expansion of the distance calculation formula: Let the coordinates of the two endpoints of line AZ be (x1, y1) and (x2, y2), and the coordinates of any point P on the curve be (x0, y0). The formula for calculating the distance d from point P to line AZ is: d = |(y2 - y1) × x0 - (x2 - x1) × y0 + x2 × y1 - y2 × x1| ÷ sqrt((y2 - y1)^2 + (x2 - x1)^2) To avoid the computational burden of square root operations, the squared distance is used for comparison in actual operations, which greatly improves the processing speed of downsampling.

[0064] S42: Cubic Bézier curve fitting.

[0065] To achieve a smooth writing effect, the characteristic inflection points are connected into a cubic Bézier curve by utilizing the condition that the slopes of the tangents to adjacent points are equal. For three consecutive points P0, P1, and P2 in the point sequence, their control points C1 and C2 are calculated.

[0066] Let the expression for a Bézier curve be: B(t) = (1-t)^3 × P0 + 3 × (1-t)^2 × t × C1 + 3 × (1-t) × t^2 × C2 + t^3 × P3, where t belongs to [0, 1].

[0067] To make the curve smooth at the characteristic inflection point P1 (i.e., C1 continuous), the tangent vector of curve segment P0-P1 at point P1 must be collinear with the tangent vector of curve segment P1-P2 at point P1. By making the tangent line of the curve at P1 parallel to the line connecting P0 and P2, the coordinates of control points C1 and C2 can be solved.

[0068] Let C1_a and C2_a be the control points of line segment P0-P1, and C1_b and C2_b be the control points of line segment P1-P2. By solving the system of linear equations, we can obtain: C2_a = P1 + (P1 - P0) ÷ 6 C1_b = P1 - (P1 - P0) ÷ 6 In this way, the two points are no longer connected by a straight line, but by a smooth mathematical curve.

[0069] S43: PDF Path content flow instruction construction.

[0070] According to the PDF specification, the fitted curve sequence is converted into a low-level plotting instruction string. The terminal processing device (30) generates a string with the following structure. The format must strictly conform to the PDF syntax requirements. Operators and operands are separated by spaces. Each instruction can be wrapped to increase readability, but in actual concatenation, it is a single string stream: pdfq% Saves the current image state 0 0 0 RG% Sets the stroke color to black (RGB mode, range 0-1). 1.5 w% sets the line width to 1.5 points. 1 J% Set the line endpoints to rounded corners (0: butt joint, 1: rounded corner, 2: projected corner) 1 j% Set the line connection to a rounded corner (0: sharp corner, 1: rounded corner, 2: beveled corner) X0 Y0 m% Move to the starting point, where m represents move to X1 Y1 X2 Y2 X3 Y3 c% Plot a cubic Bézier curve, where c represents the curve to X4 Y4 X5 Y5 X6 Y6 c% Continue drawing the next curve segment S% Stroke Path renders the path as a visible line. Q% Restore graphics state In the above instruction string, variables such as X0 and Y0 are replaced with specific values ​​from the Bézier curve control point sequence calculated by S42. The values ​​must retain at least two decimal places to ensure precision. This string fully conforms to the underlying PDF data format and contains all information related to path construction and rendering.

[0071] 6. PDF underlying file structure parsing and content stream injection anti-tampering mechanism Step S5: Use the PDF underlying injection unit (33) in the terminal processing device (30) to parse the underlying structure of the target PDF file and locate the content flow object of the current page, inject the path content flow drawing instructions into the content flow object, update the PDF file structure and save it, so that the handwritten signature is permanently and indelibly integrated into the underlying content flow of the PDF as a vector path.

[0072] This is the core step in achieving "modifying the underlying data of bank electronic archives and completely eliminating erasable editability." The traditional method involves adding a separate object to / Annots, while this invention directly modifies the / Contents stream. This step is performed by the PDF underlying injection unit, directly manipulating the PDF's byte stream.

[0073] S51: Parse the target page content stream object.

[0074] A PDF file structure consists of a Header, a Body (a collection of objects), a cross-reference table, and a Trailer. The terminal processing device calls the underlying PDF parsing library (such as PoDoFo or MuPDF in C++, or PyMuPDF / fitz in Python).

[0075] By parsing the cross-reference table, the page object of the current bank electronic file page is located. The page object is a dictionary containing a key named / Contents, whose values ​​are indirect references to content flow objects.

[0076] The parser reads this indirect reference and finds the corresponding content stream object. The stream object is usually also a dictionary, containing keys such as / Length (uncompressed or compressed length) and / Filter (filter name, such as / FlateDecode).

[0077] S52: Decompress the original content stream.

[0078] Because PDF content streams are typically compressed to reduce file size, the raw content stream object often comes with a / Filter / FlateDecode filter. The terminal processing device reads the raw byte stream of this stream object and performs decompression (inflate) using the zlib library.

[0079] In the C++ implementation, the uncompress() function is used: int ret = uncompress(dest,&destLen, source, sourceLen); After successful decompression, you will obtain a plaintext PDF instruction string (e.g., BT / F1 12 Tf 100 700 Td(Account Holder: Zhang San) Tj ET). This string defines how all text, images, and graphics on the page are drawn.

[0080] S53: Content stream splicing and state isolation.

[0081] Append the PDF Path content stream string generated in step S43 to the end of the decompressed original content stream.

[0082] Key technical point: To prevent the graphic state of handwritten signatures (such as line width and color) from contaminating subsequent original page content or table lines, it is essential to ensure that the appended string begins with 'q' and ends with 'Q'. This way, when rendering, the PDF reader pushes the current graphic state onto the stack at 'q', executes the handwritten signature instruction, and then pops it off the stack at 'Q' to restore the original state, achieving perfect style isolation.

[0083] New content stream = Original decompressed content stream + Handwritten signature Path instruction.

[0084] S54: Recompression and Write-back.

[0085] The concatenated new content stream string is then compressed again using the zlib library (using the deflate operation).

[0086] In the C++ implementation, the `compress()` function is used: int ret = compress(dest,&destLen, source, sourceLen); Generate a new compressed stream of bytes. Create a new stream object dictionary, update its / Length property to the compressed byte length, and retain / Filter / FlateDecode.

[0087] Assign new object numbers to the PDF's Body and write the new compressed stream to the file.

[0088] S55: Update the cross-reference table and save it.

[0089] Update the / Contents reference of the original page object to point to the newly generated stream object.

[0090] Rebuild the PDF's cross-reference table, recording the new byte offsets of all objects in the file.

[0091] The modified PDF file stream is overwritten and saved to the bank's core storage system, or sent back to the document management system on the display (10) via the network.

[0092] Explanation of the anti-tampering mechanism principle (for banking scenarios): Because this invention injects the signature as a path instruction (m, c, S) directly into the / Contents content stream, these instructions reside in the same stream object as the Tj instruction that renders the text of the "account opening contract" in PDF format. Therefore, there are no independently referenceable "annotation objects" in the underlying PDF data structure. When bank tellers or auditors open the document using conventional PDF editing software, the signature is treated as part of the page text and cannot be selected by mouse clicks like traditional annotations, nor can it be "deleted" or "modified." This mechanism fundamentally eliminates the possibility of "erasable signature writing and editing," ensuring the absolute security and compliance of financial documents.

[0093] Example of a complete process for electronic signature on bank account opening contracts Taking the signing of the "Personal Bank Settlement Account Management Agreement" on a smart teller machine (equipped with a 22-inch non-touch display (10)) at a bank branch as an example, the entire process is explained in detail: When a customer arrives at the counter, the teller uses a terminal processing device (30) to retrieve a PDF-formatted agreement and displays it on the monitor (10). The customer holds an electronic pen (20) and writes their name in the lower right corner of the monitor (10) where it reads "I confirm my intention to open an account and sign: ____".

[0094] Phase 1 (Hardware Acquisition and Calculation): The infrared generator (221) inside the electronic pen (20) emits 940nm infrared light, illuminating the glass substrate (11) below the pen tip (21). The micro-nano metal dot array on the substrate (11) reflects the infrared light, and the part blocked by the light-absorbing pen tip (21) forms a dark area. The infrared camera (222) captures images at 120fps. The data processing module (23) extracts the centroid of the dark area and the upper left corner position encoding point (141), and calculates the absolute physical coordinate sequence of the pen tip on the glass substrate (11) using a stepped fault-tolerant algorithm (e.g., [(-50.1, 20.5), (-50.0, 20.4), ...]). The coordinate sequence is sent to the ATM host via the UDP protocol.

[0095] The second stage (real-time UI feedback): The UI rendering thread of the ATM host receives the coordinate stream, immediately converts it into screen pixel coordinates, and uses Direct2D to draw black lines on the screen surface. The customer sees the handwriting follow the pen tip's movement with zero latency.

[0096] Phase 3 (Pen Lift Triggers Low-Level Injection): After the customer finishes signing, they lift the electronic pen (20). The system detects that no new coordinates are added within 200ms and triggers a background thread.

[0097] The background thread executes step S3: Read the PDF's MediaBox as [0 0 595 842], the system DPI as 96, and the scaling ratio as 100%. Convert the physical coordinates to PDF coordinates (e.g., (400.25, 50.15)).

[0098] The background thread executes step S4: using the DP algorithm to compress 120 sampling points into 30 feature points, calculates the Bézier control points, and generates the PDF instruction string. q 0 0 0 RG 2 w 1 J 1 j 400.25 50.15 m 401.10 49.80 402.50 49.10403.20 48.00 c ... SQ The background thread executes step S5: Parses the account opening agreement PDF and locates the / Contents stream on the last page. The original content is decompressed as BT / F1 12 Tf 100 700 Td (Account Holder) Tj ET. The system does not add any objects to the / Annots dictionary; instead, it appends the handwritten instruction string to ET, forming a new content stream. The new content stream is compressed using zlib, the cross-reference table is updated, it is written to a temporary file, atomically replaces the original file, and then uploaded to the bank's core system.

[0099] Phase Four (Verification and Audit): Bank back-office auditors reopened the PDF file and viewed it using professional software such as Acrobat. The signature was clearly visible. Even if auditors attempted to modify the contract, the signature could not be erased by clicking alone; it was integrated with the contract text. Simultaneously, other text within the contract remained in a vector-searchable state, facilitating big data risk control analysis by the bank's back-office staff. This fully meets all financial regulatory requirements for tamper-proof, traceable, and high-fidelity electronic records.

[0100] Through the aforementioned hardware and software independent yet tightly supported architecture, this invention achieves the following significant technical advantages: Unlimited cascading positioning capability that breaks through physical size limitations: Traditional infrared bezel touchscreens rely on infrared grids emitted from the edges, and the accuracy decreases as the screen size increases. Moreover, handwriting is implemented on non-touch displays. The independent hardware system of this invention directly encodes absolute position information in every tiny area of ​​the screen surface, with each dot matrix unit (12) being an independent absolute coordinate anchor point. The system's positioning accuracy is completely decoupled from the screen size, achieving true "unlimited cascading precise positioning," perfectly meeting the needs of large-size window interactive terminals in banks.

[0101] Extreme resistance to ambient light interference and high signal-to-noise ratio imaging: This method uses active infrared illumination of the camera module (22) in conjunction with narrowband filter technology to completely shield the bright and complex visible light noise in the bank lobby. The strong reflective properties of the metal point to a specific infrared band combined with the strong occlusion properties of the light-absorbing pen tip (21) make the captured image present a near "black and white" binary feature, which greatly reduces the computing power consumption of the data processing module (23), allowing the electronic pen (20) to use a low-power chip to achieve high frame rate real-time processing.

[0102] The coding scheme with ultra-high information density and extremely low visual interference adopts polar coordinate position coding points (141) of "large point + small point" to achieve a globally unique identifier in an extremely small micro-area (such as 0.5mm×0.5mm). The diameter of the metal point is at the micrometer level and is invisible to the naked eye. It completely preserves the high light transmittance and high contrast of the display (10) and ensures that the small clauses in the bank contract are clearly visible. It achieves "invisible coding" and gives the display high-precision interactive capabilities without changing the original industrial design and visual appearance of the display.

[0103] Completely eliminates the possibility of erasable signature editing with financial-grade anti-tampering capabilities (core advantage): Unlike the widely adopted / Annots annotation layer overlay scheme in existing banking systems, this invention abandons the mechanism of creating independent annotation objects. The customer's handwritten signature is injected directly into the page's / Contents content stream using path drawing instructions (m, c, S) by parsing the PDF's underlying structure. The signature handwriting and the original contract text are at the same logical level, without independent object numbers, and are permanently soldered together. No conventional PDF editing tool can select, move, edit, or erase this signature separately, completely eliminating the risk of financial fraud and repudiation, giving electronic documents the same legal effect as handwritten paper documents, making it particularly suitable for scenarios with strict anti-tampering requirements such as bank account opening and loan interviews.

[0104] Vector-based lossless mapping and full-text searchability in the archival system: By applying Douglas-Puck downsampling and cubic Bézier curve fitting to coordinate points, discrete points are transformed into smooth mathematical curves, which are then written into the PDF. No matter how many times the contract is enlarged for forensic authentication, the signature remains clear and sharp. Simultaneously, because no rasterization or flattening is performed, the original vector text characteristics of the contract are fully preserved, allowing the bank's back-end system to still perform efficient OCR retrieval and automated risk control auditing of archived contracts.

[0105] Robust fault-tolerant reading and frame-loss prevention mechanism: A step-value fault-tolerant algorithm is introduced into macro-localization, allowing for a certain degree of distortion in the angle vector; and a cross-image index fault-tolerant calculation mechanism is activated when information is missing. This dual fault-tolerant mechanism completely solves the problems of "lost points" and "frame skipping" when customers quickly sign their names on the bank's large screen, ensuring the continuity and smoothness of the signature trajectory.

[0106] An ultra-low latency architecture decoupled from asynchronous rendering and underlying writing: During the writing process, the UI layer renders visual feedback in real time through a graphics API to achieve zero-latency responsiveness and ensure a smooth writing experience for the user. The instant the pen is lifted, a background thread is triggered to perform complex underlying PDF parsing, decompression, injection, and compression / saving operations, employing a double-buffered atomic file replacement mechanism. This architecture ensures both smooth writing and stable modification and absolute security of the underlying data in the bank's core business systems.

[0107] In summary, the present invention provides a method and system for handwritten modification of PDF underlying data based on a non-touch display. Through the innovative integration of technologies such as micro-nano metal dot matrix, infrared active visual recognition, multi-coordinate system precise transformation, Bezier vector fitting, and direct injection of PDF underlying content stream, it successfully overcomes many shortcomings of existing touch and annotation technologies in the application of electronic archives in banks. While maintaining the ultimate visual experience of the display, it achieves high-precision, tamper-proof, and permanently integrated non-touch vector signature interaction, which has significant application value in the financial industry.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for handwritten modification of underlying PDF data based on a non-touch display, characterized in that, The method includes the following steps: S1: Use an electronic pen (20) to write on a non-touch display (10), and use the active infrared camera module (22) inside the electronic pen (20) to capture a dot matrix image of the area where the pen tip (21) is located during the writing process, so as to collect physical interaction information; S2: The data processing module (23) inside the electronic pen (20) performs feature analysis on the dot matrix image, and calculates the global absolute physical coordinates of the pen tip (21) at different times based on the position coding point (141), thereby realizing the extraction of the physical coordinate set of the pen tip (21) motion trajectory. S3: Utilize the terminal processing device (30) to receive the set of physical coordinates and perform coordinate system transformation, mapping the global absolute physical coordinates to the target coordinates under the PDF page coordinate system, thereby achieving precise alignment from physical space to the digital space of electronic archives; S4: Using the terminal processing device (30), the target coordinate set is vector downsampled and curve fitted to generate a Bezier curve control point sequence, and a path content stream drawing instruction conforming to the PDF specification is generated based on the control point sequence to realize the conversion of discrete coordinates to continuous vector mathematical curves; S5: Using the terminal processing device (30), the underlying structure of the target PDF file is parsed and the content flow object of the current page is located. The path content flow drawing instructions are injected into the content flow object, the PDF file structure is updated and saved, so that the handwritten signature is permanently and indelibly integrated into the underlying content flow of the PDF as a vector path.

2. The method according to claim 1, characterized in that, The non-touch display writing system includes a display (10) and an electronic pen (20). The glass substrate (11) of the display (10) is provided with multiple dot matrix units (12) arranged in rows and columns. Each dot matrix unit (12) contains metal coding points (14). The electronic pen (20) includes a pen tip (21), a camera module (22), and a data processing module (23). In step S1, the infrared generator (221) of the camera module (22) actively emits infrared light to illuminate the glass substrate (11). The infrared camera (222) captures the infrared light image reflected by the metal coding points (14). The pen tip (21) made of light-absorbing material blocks part of the metal coding points (14) to form a dark area, thereby realizing the acquisition of a high signal-to-noise ratio dot matrix image.

3. The method according to claim 2, characterized in that, In step S2, the global absolute physical coordinates of the pen tip (21) are calculated based on the position encoding point (141), specifically including: Identify the first point (142) and the second point (143) located at the top left corner of the dot matrix unit (12) in the image, and calculate the modulus and argument of the line connecting the two points; The distance between the second point (143) and the first point (142) is calculated to represent the distance of the current lattice unit (12) relative to the center point of the glass substrate (11); the quadrant and step value range of the argument indicate the row or column in which the lattice unit (12) is located relative to the center of the glass substrate (11); The distribution of the dark area obscured by the pen tip (21) in the dot matrix grid (13) is analyzed, and the local offset at the sub-pixel level is calculated. The global absolute reference coordinates and the local offset are added and fused to output high-precision absolute physical coordinates (X_pen, Y_pen).

4. The method according to claim 1, characterized in that, The coordinate system transformation in step S3 specifically includes: Obtain the actual display pixel area and scaling ratio of the current PDF page on the monitor (10); Establish the first mapping relationship between the global absolute physical coordinate system and the screen display pixel coordinate system; A second mapping relationship is established between the screen display pixel coordinate system and the PDF page coordinate system. The Y-axis of the screen pixel coordinate is flipped, and a scale conversion is performed according to the conversion relationship between screen DPI and PDF point units. The calculation formula is: Y_pdf = PageHeight_pdf - (Y_pixel × ScaleFactor), where Y_pdf is the ordinate of the PDF coordinate system, PageHeight_pdf is the height of the PDF page, Y_pixel is the ordinate of the screen pixels, and ScaleFactor is the scale conversion factor.

5. The method according to claim 1, characterized in that, The vector downsampling in step S4 specifically includes: Downsample the original target coordinates, draw a straight line between the first and last points of the curve, and calculate the distance from all other points on the curve to this straight line. Find the point with the largest distance. If the distance is greater than a preset threshold, use this point as the dividing point to divide the original curve into two segments, and recursively execute the above steps. If the largest distance is less than the threshold, discard all intermediate points and only keep the first and last points, thereby eliminating redundant coordinate points and retaining feature inflection points.

6. The method according to claim 5, characterized in that, The curve fitting in step S4 specifically includes: Based on the retained characteristic inflection points, and using the condition that the slopes of the tangents at adjacent points are equal, the coordinates of the two control points of the cubic Bézier curve are calculated. Let the expression for the Bézier curve be: B(t) = (1-t) 3 × P0 + 3 × (1-t) 2 × t × C1 + 3 ×(1-t) × t 2 × C2 + t 3 × P3, where P0 is the starting point, P3 is the ending point, C1 and C2 are the control points to be determined, and t is the parameter; By making the tangent line of the curve at the characteristic inflection point parallel to the line connecting the adjacent inflection points, the coordinates of control points C1 and C2 are solved in reverse. The starting point, subsequent control points, and ending point are arranged in sequence to generate the Bézier curve control point sequence.

7. The method according to claim 6, characterized in that, The step S4 of generating path content stream drawing instructions specifically includes: Construct an instruction header that includes the graphic state save instruction q, color setting instruction RG, line width setting instruction w, line endpoint setting instruction J, and line connection setting instruction j; Based on the Bézier curve control point sequence, a movement command m and a cubic Bézier curve drawing command c are generated, wherein the m command is followed by the starting point coordinates, and the c command is followed by the coordinates of two control points and the ending point coordinates. Generate a stroke instruction S and a graphic state recovery instruction Q, and concatenate them to form a complete PDF path content stream string.

8. The method according to claim 1, characterized in that, The injection and saving in step S5 specifically include: Read the target PDF file using a PDF parsing library and locate the content flow object of the current page object; If the content stream object has a / FlateDecode filter, the zlib library is used to decompress and decode it to obtain the original plaintext content stream; The PDF path content stream string is appended to the end of the original plaintext content stream to form a new content stream; The new content stream is compressed using zlib, generating a new stream object. The PDF cross-reference table is then updated and the saved file is overwritten.

9. The method according to claim 1, characterized in that, The method modifies the page / Contents stream object so that the signature handwriting and the original text of the bank's electronic archive are at the same logical level.

10. A PDF underlying data handwriting modification system based on a non-touch display, characterized in that, The handwriting modification system for performing the method as described in claim 1 includes: The display (10) has a glass substrate (11) with a plurality of dot matrix units (12) arranged in rows and columns, each dot matrix unit (12) containing metal coding dots (14); An electronic pen (20) includes a pen tip (21), a camera module (22), and a data processing module (23); the camera module (22) is positioned to capture images of the pen tip (21), and contains an infrared generator (221) and an infrared camera (222); the data processing module (23) is used to perform image analysis and calculate physical coordinates. The terminal processing device (30) is communicatively connected to the electronic pen (20) and includes a coordinate transformation unit, a vector fitting unit and a PDF underlying injection unit, which are used to convert the physical coordinates calculated by the electronic pen (20) into PDF coordinates and generate path instructions to inject into the underlying layer of the bank's electronic archive PDF file.

Images