Paperless interactive information multi-terminal real-time synchronization processing method and system

Abstract

This invention relates to the fields of computer data processing and network communication technology, specifically to a method and system for real-time synchronization of paperless interactive information across multiple terminals. The method includes: a logic construction step: acquiring the original interactive command stream, constructing a semantic causal logic tree based on a vector clock, and dividing strong barriers and weak streaming events; a backpressure monitoring step: establishing a rendering backpressure model to calculate the rendering backpressure value of the target terminal; a dynamic shaping step: responding to a backpressure value exceeding a threshold, reducing the dimensionality of the command stream and reorganizing its timing, attaching weak streaming events to strong barrier events and merging them into an atomic state snapshot for transmission; and a consistency verification step: comparing the screen content-aware hash value, and if a mismatch occurs, performing full state synchronization. This invention solves the problem of visual semantic confusion caused by differences in rendering capabilities between heterogeneous terminals.

Description

Technical Field

[0001] This invention relates to the field of computer data processing and network communication technology, specifically to a paperless interactive information multi-terminal real-time synchronization processing method and system. Background Technology

[0002] Paperless interactive technology is a core support for modern remote collaboration and multimedia teaching. It collects touch trajectories and document page turning commands from the source end in real time and transmits them to various target terminals for synchronous rendering via the network, realizing cross-regional information sharing and collaborative work. It features strong immediacy and intuitive interactive experience.

[0003] However, with the increasing diversification of access terminal types, there are significant differences in hardware performance and rendering capabilities among devices. In existing synchronous processing solutions, the local physical system time is usually used to mark the order of instructions, and there is a lack of an effective feedback mechanism for the real-time load pressure of the receiving end. When high-frequency interactive instruction streams are sent to low-computing-power terminals, rendering congestion is easily triggered, causing the execution sequence of weak streaming events and strong barrier events to be logically reversed, resulting in visual semantic confusion and excessive synchronization latency, which makes it difficult to meet the needs of high-precision real-time collaboration. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for real-time synchronization of paperless interactive information across multiple terminals. Specifically, the technical solution of this invention is as follows:

[0005] A method for real-time synchronization of paperless interactive information across multiple terminals includes:

[0006] The original interactive instruction stream generated by the source is obtained, and a semantic causal logic tree representing the instruction execution dependency relationship is constructed based on vector clock technology. The semantic causal logic tree divides the original interactive instruction stream into strong barrier events and weak streaming events.

[0007] Real-time collection of operational feedback data from the target terminal; establishment of a rendering backpressure model reflecting the terminal load status; and calculation of the rendering backpressure value of the target terminal based on the rendering backpressure model.

[0008] In response to the rendering backpressure value exceeding a preset congestion threshold, a dynamic data shaping process is triggered, including: Step 1, based on the dependency relationship of the semantic causal logic tree, performing dimensionality reduction and temporal reorganization on the original interactive command stream to generate an adapted command package; Step 2, forcibly attaching the weak streaming event to the logically nearest preceding strong barrier event, merging the two into an atomic state snapshot; Step 3, sending the adapted command package or atomic state snapshot to the target terminal to perform the rendering operation; Step 4, obtaining the perceptual hash value of the screen content rendered at the key node on the target terminal, comparing the perceptual hash value with the source baseline hash value, performing full state synchronization if they do not match, and confirming synchronization consistency if they match.

[0009] Preferably, constructing a semantic causal logic tree includes:

[0010] Assign a logical timestamp to each instruction in the original interactive instruction stream;

[0011] Commands that alter the global context state are identified as strong barrier events;

[0012] Identify incremental modification commands in the current context as weak streaming events;

[0013] Establish a one-way dependency constraint between weak streaming events and preceding strong barrier events, prohibiting weak streaming events from being rendered before the corresponding strong barrier events have completed execution.

[0014] Preferably, the rendering backpressure model is established, including:

[0015] Extract rendering queue length, frame rate data, and CPU utilization from the runtime feedback data;

[0016] The rendering latency index is calculated based on the difference between the rendering queue length and the preset processing capacity.

[0017] Using a weighted summation algorithm or a preset mapping table, combined with the rendering latency index and the frame rate data, the current rendering backpressure value of the target terminal is calculated and generated.

[0018] Preferably, the original interactive command stream is subjected to dimensionality reduction processing, including:

[0019] Since the original interactive command stream is a set of high-frequency continuous coordinate points, a Bézier curve fitting algorithm is used.

[0020] Extract key control points from a set of high-frequency continuous coordinate points and generate parametric curve instructions;

[0021] The original set of high-frequency continuous coordinate points is replaced with parametric curve instructions as part of the adaptation instruction package.

[0022] Preferably, the original interactive instruction stream is reassembled in sequence, including:

[0023] In response to the existence of multiple consecutive strong barrier events in the semantic causal logic tree, the intermediate frame discarding strategy is executed;

[0024] The last strong barrier event in the sequence is retained, while strong barrier events in the middle are discarded.

[0025] The weak streaming events following the last strong barrier event are realigned to the timeline of that last strong barrier event to generate a recombined instruction sequence.

[0026] Preferably, the two are combined into an atomic state snapshot, including:

[0027] Intercept weak streaming events that have not yet been sent;

[0028] Waiting for the data packets of the strong barrier events it depends on to complete construction;

[0029] Packets of strong barrier events and packets of intercepted weak streaming events are encapsulated into the same indivisible network transmission unit, ensuring that the target terminal processes the network transmission unit within the same rendering frame.

[0030] Preferably, if there is a mismatch, a full state synchronization is performed, including:

[0031] Suspend the target terminal's reception and processing of the current adapted instruction packet;

[0032] Obtain complete view state data at the current moment from the source;

[0033] Forcefully overwrite the current framebuffer contents of the target terminal and reset the target terminal's rendering queue.

[0034] A paperless interactive information multi-terminal real-time synchronization processing system, including:

[0035] The logic building module is used to acquire the original interactive command stream and construct a semantic causal logic tree;

[0036] The status monitoring module is used to collect operational feedback data and calculate the rendering back pressure value;

[0037] The dynamic shaping module includes: a data processing unit, used to reduce and reorganize the instruction stream based on a semantic causal logic tree to generate an adapted instruction package; an atomic merging unit, used to merge weak streaming events and strong barrier events into an atomic state snapshot; and a transmission control unit, used to send the processed data to the target terminal.

[0038] The consistency verification module is used to compare the perceived hash value. It is configured to: trigger full state synchronization if there is a mismatch, and confirm synchronization consistency if there is a match.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. This invention effectively solves the problem of visual semantic confusion caused by differences in rendering capabilities of heterogeneous terminals by constructing a semantic causal logic tree based on vector clock technology. Unlike traditional solutions that rely on the time stamp order of the local physical system, this solution establishes a logical timing system independent of physical time. By identifying the dependency relationship between strong barrier events and weak streaming events, it strictly defines the partial order constraint of instruction execution, ensuring that incremental modification instructions can only be executed after the corresponding global context state change is completed, thus preventing the logical inversion phenomenon caused by rendering congestion.

[0041] 2. This invention establishes a rendering backpressure model that integrates multi-dimensional operational feedback data, solving the technical problem that the source end cannot perceive the load pressure of the target terminal in real time. By comprehensively collecting rendering queue length, real-time frame rate and processor utilization, and converting heterogeneous instruction streams into standardized load values ​​with the same dimensions as processing power, this method can accurately quantify the congestion level and digestion capacity of the target terminal, thereby providing accurate physical criteria for triggering dynamic data shaping processes and avoiding the impact of high-frequency instruction streams on low-computing-power devices.

[0042] 3. This invention employs a dynamic data shaping and atomic state snapshot merging strategy, which significantly improves the smoothness and consistency of system interaction in weak network environments and high-load scenarios. By performing Bézier curve fitting and dimensionality reduction on high-frequency continuous coordinate points and executing a strategy to discard strong barrier events in the intermediate process, this method significantly reduces data transmission volume and rendering overhead. At the same time, it uses atomic encapsulation technology to force weak streaming events and their dependent strong barrier events to be processed in the same frame, eliminating the ghost semantic phenomenon of having only annotations without a background page.

[0043] 4. This invention constructs a closed-loop consistency verification and full self-healing mechanism based on perceptual hash comparison, giving the system the ability to automatically recover under extreme anomalies; the system can quickly detect visual fingerprint differences at key rendering nodes, and when a mismatch is determined, it can achieve second-level error correction and reset of the terminal state by pausing reception, obtaining the complete view state data copied at write time from the source end, and forcibly overwriting the frame buffer. Moreover, after full synchronization, it forcibly updates the local logical clock, ensuring the logical continuity of subsequent incremental events, and ensuring strict synchronization of multi-terminal screens without manual intervention. Attached Figure Description

[0044] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0045] Figure 1 This is a flowchart of the method of the present invention;

[0046] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0048] Example 1:

[0049] Please see Figure 1 A method for real-time synchronization of paperless interactive information across multiple terminals, including:

[0050] The original interactive instruction stream generated by the source is obtained, and a semantic causal logic tree representing the instruction execution dependency relationship is constructed based on vector clock technology. The semantic causal logic tree divides the original interactive instruction stream into strong barrier events and weak streaming events.

[0051] Real-time collection of operational feedback data from the target terminal; establishment of a rendering backpressure model reflecting the terminal load status; and calculation of the rendering backpressure value of the target terminal based on the rendering backpressure model.

[0052] In response to the rendering backpressure value exceeding a preset congestion threshold, a dynamic data shaping process is triggered, including: Step 1, based on the dependency relationship of the semantic causal logic tree, performing dimensionality reduction and temporal reorganization on the original interactive command stream to generate an adapted command package; Step 2, forcibly attaching the weak streaming event to the logically nearest preceding strong barrier event, merging the two into an atomic state snapshot; Step 3, sending the adapted command package or atomic state snapshot to the target terminal to perform rendering operations; Step 4, obtaining the perceptual hash value of the screen content rendered at the key node on the target terminal, comparing the perceptual hash value with the source baseline hash value, performing full state synchronization if they do not match, and confirming synchronization consistency if they match.

[0053] This embodiment details the specific execution logic of the above method, aiming to solve the problem of visual semantic confusion caused by differences in rendering capabilities of heterogeneous terminals. The system intercepts all user input events at the source end. In order to establish a logical sequence independent of physical time, the system adopts a vector clock algorithm instead of local system time to construct a semantic causal logic tree representing the dependency relationship of instruction execution. This logic tree is a data structure that transcends the order of physical time. Its core function is to ensure that the order of instruction execution on heterogeneous terminals strictly follows the business logic. The system runs a lightweight daemon process to collect the running feedback data of the target terminal in real time at a period of 50 milliseconds, and establishes a rendering backpressure model to quantify the congestion level of the terminal. This model calculates the physical index reflecting the degree of terminal indigestion, namely the rendering backpressure value, by comprehensively considering the rendering queue length, real-time frame rate, and CPU utilization.

[0054] In response to the calculated rendering backpressure value exceeding the preset congestion threshold, the system determines that the terminal has entered the rendering congestion zone and triggers a dynamic data shaping process. During this process, the system performs dimensionality reduction and temporal reorganization on the original interactive command stream, generates an adaptation command package, and forcibly attaches the weak streaming event to the logically nearest preceding strong barrier event, merging the two into an atomic state snapshot. The lightweight and atomic adaptation command package or atomic state snapshot is sent to the high-load target terminal to perform rendering operations. At critical rendering nodes, such as after completing a page turn, the system obtains the perceptual hash value of the screen content rendered by the target terminal and compares it with the source baseline hash value. The specific comparison process is as follows: the system uses a perceptual hash algorithm to reduce the screen screenshot to 32×32 pixels and convert it into a grayscale image. After calculating the discrete cosine transform, the system extracts the low-frequency component of the upper left corner of 8×8, calculates the average value of these 64 pixels, and generates a 64-bit binary fingerprint based on the relationship between each pixel and the average value.

[0055] The Hamming distance is calculated between the fingerprint of the target terminal and the reference fingerprint of the source terminal. If the Hamming distance exceeds the fault tolerance range, the fault tolerance range is strictly set to an integer threshold T in this embodiment. ham =3, which is the Hamming distance threshold. This value is derived from a large number of image difference experiments and aims to filter out non-semantic minor noise caused by JPEG compression or rendering anti-aliasing, while ensuring that it can keenly capture content changes. That is, when the Hamming distance D of two 64-bit fingerprints is greater than 3, it is considered a mismatch. Under this threshold, the false alarm rate (FAR) of the system is less than 0.1% and the false negative rate (FRR) is less than 0.5%. The system determines that there is a visual inconsistency and performs full state synchronization. Otherwise, it confirms that the synchronization is consistent.

[0056] Example 2:

[0057] Constructing a semantic causal logic tree includes:

[0058] Assign a logical timestamp to each instruction in the original interactive instruction stream;

[0059] Commands that alter the global context state are identified as strong barrier events;

[0060] Identify incremental modification commands in the current context as weak streaming events;

[0061] Establish a one-way dependency constraint between weak streaming events and preceding strong barrier events, prohibiting weak streaming events from being rendered before the corresponding strong barrier events have completed execution.

[0062] This embodiment is a further specification of the steps for constructing the semantic causal logic tree in Embodiment 1; the system assigns a logical timestamp to each instruction in the original interactive instruction stream; instead of using Unix timestamps, vector clock technology is adopted, and the system maintains an integer vector of length N. Where N is the number of terminals, for each new event at the source terminal with index i, execute... The updated vector is then attached to the instruction to strictly define the partial order relationship between events, ensuring that the logical order in the distributed system is not affected by physical clock deviations. The system identifies event types based on a built-in state change sensitivity table. This sensitivity table predefines the mapping relationship between instruction opcodes and event types. In response to instructions involving changes to underlying layers such as PPT backgrounds or video streams, the system identifies them as strong barrier events based on the configuration in the table. These events are like dams, possessing exclusivity. In response to instructions involving only changes to overlay layers such as Canvas drawing or floating windows, the system identifies them as weak streaming events. These events depend on the existence of preceding strong barrier events. The system establishes a unidirectional dependency constraint between weak streaming events and preceding strong barrier events, i.e., establishing directed edges in the logic tree.

[0063] A causal locking mechanism is introduced into the rendering engine of the target terminal. When the rendering thread is about to process a weak streaming event, it must first check the status flag of its parent node, i.e., the strong barrier event. In response to the strong barrier event being incomplete or not received, the system will forcibly put the corresponding weak streaming event into the waiting queue and prohibit it from being rendered on screen until the strong barrier event is completed.

[0064] Example 3:

[0065] Establish the rendering backpressure model, including:

[0066] Extract rendering queue length, frame rate data, and CPU utilization from the runtime feedback data;

[0067] The rendering latency index is calculated based on the difference between the rendering queue length and the preset processing capacity.

[0068] The current rendering backpressure value of the target terminal is calculated by using a weighted summation algorithm or a preset mapping table, combined with the rendering latency index and frame rate data.

[0069] This embodiment further specifies the steps for establishing the rendering backpressure model in Embodiment 1. It strictly adheres to the principle of full disclosure in the specification, defining the physical meaning and calculation logic of the core parameters. The system extracts the rendering queue length L by monitoring the instruction buffer of the underlying graphics interface, such as Android Surface Flinger. It should be noted that, to ensure a closed-loop physical meaning, L here is not a simple instruction count, but a standardized load value calculated using the instruction-computing power equivalent conversion function. The system pre-stores an instruction overhead mapping table, using the standard primitives in the benchmark test as the unit 1 SRU. Here, SRU is defined as StandardRenderingUnit, a dimensionless physical quantity used to quantify the relative complexity of rendering instructions, serving as a unified calculation benchmark for heterogeneous instructions.

[0070] To ensure the executability of the computation, the instruction cost mapping table is defined as follows: For the path drawing instruction Path, the cost is defined as... , where N points The cost is the number of coordinate points included in the path instruction; for the Image rendering instruction, the cost is defined as:

[0071] ;

[0072] Where Width is the pixel width of the image involved in the current image rendering command, and Height is the pixel height of the image involved in the current image rendering command; when applying Gaussian blur effect, K effect =5.0, this coefficient is an empirical value derived from GPU shader performance analysis, reflecting the computational overhead of a standard 5×5 Gaussian blur convolution kernel relative to a single texture sample. This value can be adjusted within the range of 4.0 to 6.0 depending on the specific GPU architecture. For example, for mobile GPU architectures with weak floating-point computing power, it is recommended to take the upper limit of 6.0, while for desktop discrete graphics cards, it is recommended to take the lower limit of 4.0, otherwise K effect =1.0; For the Text drawing instruction, the overhead is defined as 1.0. , where N chars The number of characters in the text command; the system iterates through the queue in real time and calculates... This transforms the heterogeneous instruction stream into a value with the same dimensions as the processing power C; at the same time, it counts the number of screen refreshes in the last second to obtain the frame rate data F, and extracts the CPU main thread utilization rate U from the operating system kernel statistics. In order to match the weight coefficients in the following text, U is defined as a normalized value with a value range of [0,1].

[0073] The system calculates the rendering latency index I using the following formula:

[0074] ;

[0075] in, The physical difference calculation based on the unified SRU dimension explicitly represents the concept of difference, which is the current backlog of instructions minus the system's maximum processing capacity within the time window; here, it is multiplied by the time window. The throughput C is converted into SRU, which has the same physical dimension as the load L, thus ensuring the legality of the physical meaning of the subtraction operation.

[0076] C: The currently effective processing capacity. Here, C is not the general instruction count, but is defined as the standardized computing throughput, with its unit strictly anchored to SRU / s, perfectly aligned with the numerator L; this value is determined by the baseline processing capacity C. base Combining real-time status export; the system obtains the device's inherent computing power C through benchmark testing during the application initialization phase. base During the device cold start phase, perform 1000 off-screen renderings of standard primitives and record the total time T. cost And calculate Since the benchmark uses standard primitives defined as 1 SRU, the calculated result C base The physical meaning is the standard workload that can be processed per second, SRU / s;

[0077] To ensure that benchmark tests accurately reflect the GPU's rasterization and texture sampling bottlenecks, standard primitives are strictly defined as RGBA texture rectangles of 256×256 pixels. Bilinear filtering and alpha blending must be enabled during rendering to simulate real-world UI over-draw load. If a CPU frequency drop of more than 20% is detected, meaning the system obtains the current frequency by reading ` / sys / devices / system / cpu / scaling_cur_freq` and compares it with ` / sys / devices / system / cpu / cpuinfo_max_freq` to determine the frequency drop magnitude, then C in the formula is set to C. base Multiply by the thermal decay coefficient of 0.8, otherwise This dynamic value-taking logic distinguishes between the baseline value and the real-time value, ensuring that the denominator of the formula reflects the true physical digestion capacity under the thermal throttling state of the equipment, and avoiding underestimation of the delay exponent due to overestimation.

[0078] Sampling time window, set to 0.05 seconds;

[0079] Numerical stability constant, taken as 10 -6 To prevent division by zero errors when C≈0 due to extreme equipment lag;

[0080] The system combines rendering latency exponent and frame rate data, and uses a weighted summation algorithm to calculate the current rendering backpressure value P of the target terminal; to prevent high refresh rate screens from causing 1-F / R to be negative and thus masking the true CPU load, boundary constraints are introduced into the formula:

[0081] ;

[0082] in, It is a hyperbolic tangent function used to non-linearly map an unbounded rendering latency exponent to a bounded interval;

[0083] : Force the non-negativity of the frame rate term to ensure that even if F > R, the contribution of this term is at least 0, rather than negative;

[0084] R: Baseline reference frame rate, taken as 60Hz;

[0085] U: CPU main thread utilization rate of the target terminal;

[0086] α, β, γ: Weighting coefficients, adjusted and set to 0.6, 0.3, and 0.1 respectively; this parameter combination optimizes the system's response sensitivity. On one hand, α=0.6 ensures that the backpressure value P can quickly approach the threshold when the rendering queue is severely backlogged; on the other hand, increasing β to 0.3 is to cover low-queue, high-latency scenarios, ensuring that even when the queue is not full but the frame rate drops significantly, the CPU load factor can trigger the congestion threshold of 0.7, thereby initiating the dynamic shaping process. The values ​​of the above weighting coefficients are not arbitrarily set, but are obtained based on offline supervised learning: the R&D team pre-collected 5000 sets of sample pairs containing rendering status data (I, F, U) of different devices under various network environments and user subjective stuttering ratings; the data was input into the logistic regression model, with user ratings greater than 5 as positive examples, and iterative training was performed by minimizing the log-likelihood loss function; since the logistic regression outputs log odds, it cannot be directly used in the linear weighting formula, so the system uses the original regression coefficients obtained from training. Perform absolute value normalization, i.e., calculate , , This maps unbounded statistical coefficients to physically meaningful [0,1] convex combination weights, ensuring the interpretability of the calculated P-value. The original logistic regression coefficient values ​​represent the corresponding rendering latency index obtained from training the logistic regression model. The original logistic regression coefficient values ​​represent the corresponding frame rate decreases obtained from training the logistic regression model. The original logistic regression coefficients represent the CPU utilization values ​​obtained from training the logistic regression model; the mathematical principle behind using absolute value normalization here is that the logistic regression coefficients... The magnitude of the value directly reflects the corresponding feature. The sensitivity contribution to the results; because in paperless interaction scenarios, the physical features of the input are designed to be positively correlated with the system load, the trained... Theoretically, all values ​​are positive.

[0087] Taking the absolute value is to eliminate the small negative fluctuations that may be introduced by statistical noise and to ensure the non-negativity of the weights; while normalization is to build a linear surrogate model, which retains the relative importance ratios of each feature learned by logistic regression while reducing the computational complexity from exponential to constant level to meet the high-frequency real-time computing requirements of 60 frames per second on mobile devices; this method ensures that the model can reproduce human perception of rendering congestion with an accuracy of 94.5%.

[0088] Example 4:

[0089] Dimensionality reduction processing of the original interactive command stream includes:

[0090] Since the original interactive command stream is a set of high-frequency continuous coordinate points, a Bézier curve fitting algorithm is used.

[0091] Extract key control points from a set of high-frequency continuous coordinate points and generate parametric curve instructions;

[0092] The original set of high-frequency continuous coordinate points is replaced with parametric curve instructions as part of the adaptation instruction package.

[0093] This embodiment further specifies the dimensionality reduction steps in Embodiment 1, and specifically optimizes handwriting by adding necessary preprocessing steps to ensure the algorithm's practical executability; the system monitors the original interactive command stream and responds to the detection of a set of high-frequency continuous coordinate points. The system initiates the dimensionality reduction process;

[0094] To address the issue that a single Bézier curve cannot fit complex handwriting, the system employs a curvature-based segmentation strategy: Step 0, performing overlapping point cleaning; before calculating angles and curvature, the system pre-traverses the set of coordinate points and calculates the Euclidean distance between adjacent points. ; in response to Less than the floating-point precision threshold (e.g. 10) -5 ), here Defined as a pixel distance threshold, this threshold is set to eliminate high-frequency jitter noise from the digitizer and mathematically prevent the denominator from being zero or the coordinates of two points from being completely identical when the system makes a judgment. Invalid sampling points are removed from the set; the purpose of this step is to mathematically ensure that the denominator terms in subsequent formulas are not invalid. It must never be zero to prevent division by zero exceptions during program execution. It is the point immediately adjacent to the current sampling point in the set of high-frequency continuous coordinate points. The coordinates of the previous sampling point;

[0095] Calculate each point deflection angle :

[0096] ;

[0097] In this formula, the operator · in the numerator is explicitly defined as the dot product of vectors, while · in the denominator is a regular scalar multiplication; the angle represents the change in handwriting direction; for normalization purposes, the system further calculates the discrete curvature density. ;

[0098] Response to deflection angle Greater than the preset threshold In this embodiment Values The radian angle, an empirically set threshold, is used to effectively distinguish smooth strokes from semantically significant turning points in handwriting analysis, such as the features of stroke breaks in Chinese characters. This point is marked as a segmentation point, and the set S... pts Divide into multiple sub-segments S1, S2, ...;

[0099] For each sub-segment, a third-order Bézier curve is fitted using the least squares method. To fill the parameterization black box between the coordinate points and the curve equation, the system performs chord length parameterization here: for m points within the sub-segment, the parameterization is calculated for each point. Corresponding parameters :

[0100] ;

[0101] This step ensures that the parameter distribution is proportional to the physical handwriting distance, avoiding curve knots caused by uniform parameterization; the fitted model is as follows:

[0102] ;

[0103] Where u is a parameter variable, and Q0...Q3 are the control points to be determined;

[0104] To ensure that the piecewise fitted curve maintains C0 continuity at the connection points, the system does not solve for all control points, but instead performs constrained least squares: a forced constraint is applied. and The system constructs only a system of linear equations about intermediate control points Q1 and Q2; and constructs the objective function. ,in, The coordinates of the j-th actual sampling point within the current sub-segment. Let this be the parameterized variable corresponding to the sampling point on the Bézier curve. For parameters The corresponding third-order Bézier curve fitting model calculates the point coordinates and transforms them into matrix form. Solve the equations, where A is the coefficient matrix and Y is the constant term result matrix on the right side of the equation.

[0105] The specific construction logic of the above matrix form is as follows: rewrite the Bessel formula in a separated variable form. The residual vector basis function coefficients , Based on the least squares principle, let the objective function be related to the point to be solved. Since the partial derivatives are zero, construct the system of normal equations:

[0106] ;

[0107] Where m is the actual number of sampling points within the currently segmented sub-segment; the system solves the system of two linear equations using Cramer's rule or LU decomposition; to solve the vector term R in the above formula... j Regarding the dimension matching problem of the scalar coefficient matrix, the manual clearly outlines the specific mathematical processes for each calculation: The system decouples the above vector equations into two independent systems of scalar linear equations; The X-axis equation system is established: Let R... j,x For R j Solve for the x-coordinate components. Obtain the x-coordinates of the control points; similarly, establish the Y-axis equation system: Let R j,y For R j Solve for the ordinate components. The ordinates of the control points are obtained. Through this method, the generated parametric curve instructions significantly reduce the amount of data while ensuring that each curve segment is connected end to end, accurately restoring the continuous handwriting characteristics of the user.

[0108] Example 5:

[0109] Timing reordering of the original interactive command stream includes:

[0110] In response to the existence of multiple consecutive strong barrier events in the semantic causal logic tree, the intermediate frame discarding strategy is executed;

[0111] The last strong barrier event in the sequence is retained, while strong barrier events in the middle are discarded.

[0112] The weak streaming events following the last strong barrier event are realigned to the timeline of that last strong barrier event to generate a recombined instruction sequence.

[0113] This embodiment is a further specification of the timing reassembly steps in Embodiment 1, correcting the semantic confusion that may have occurred in the original description; the system scans the semantic causal logic tree, and in response to the detection of multiple consecutive strong barrier events, the system determines that the intermediate state is a transient process; the system executes the intermediate frame discarding strategy, physically deleting the intermediate strong barrier events B1 and B2 and their data packets, retaining only the final strong barrier event B3;

[0114] In this process, to prevent ghost semantics issues, the system executes a cascading cleanup protocol: 1. Traverse the instruction queue, identify all weak streaming events whose ParentID points to discarded events, and remove them from the queue; this step ensures that only incremental modifications related to the final state B3 semantics are retained;

[0115] For weak streaming events following the last strong barrier event B3, the system executes a logical clock remapping algorithm to realign the timeline. The specific steps are as follows: 1. Dependency Confirmation: Confirm that the parent node pointer ParentID of event W is the unique identifier UUID of B3; 2. Vector Clock Correction: Because the deletion of intermediate events changes the density of the global event sequence, the system needs to recalculate the logical time of W; let the vector clock of B3 be V. B3 The new ordinal number of event W relative to B3 in the cleaned sequence is The system constructs a new vector clock for W. :

[0116] ;

[0117] in, The operation is defined as a component update operation, specifically mathematically defined as: if k is the dimension index of the current source end in the vector clock, then the updated vector... Satisfying components And for the remaining components j≠k, maintain ; here The sequence increment of event W relative to B3 after recombination is defined and determined as follows: the system sorts all weak-stream events belonging to B3 in ascending order according to their original generation timestamps, and assigns sequence numbers starting from the integer 1. The calculation method is as follows: Where M is the total number of weakly flowing events, which are allocated sequentially in chronological order to assign this increment value; this operation ensures that the partial order relation is satisfied. This generates a new instruction sequence that is logically rigorous and semantically unambiguous; to prevent the logical timestamp of subsequent newly generated events from the source from being less than a certain value due to human modification of the timeline. The system performs source clock synchronization submission: during generation Then, the source end forces an update to the local vector clock component. This ensures the monotonically increasing nature of the subsequent event flow;

[0118] Specifically, when the source sends the reassembled instruction sequence, it will explicitly include the updated vector in the protocol header. The receiving end parses the vector and forces logical clock synchronization. Where sup is the maximum value operation for each component, which is explicitly defined as This is to eliminate timing gaps that may be caused by intermediate discard events.

[0119] Example 6:

[0120] The two are merged into an atomic state snapshot, including:

[0121] Intercept weak streaming events that have not yet been sent;

[0122] Waiting for the data packets of the strong barrier events it depends on to complete construction;

[0123] Packets of strong barrier events and packets of intercepted weak streaming events are encapsulated into the same indivisible network transmission unit, ensuring that the target terminal processes the network transmission unit within the same rendering frame.

[0124] This embodiment is a further specification of the atomic state snapshot merging step in Embodiment 1. When the system decides to send a weak streaming event, such as a circle drawing command, it checks whether the strong barrier event it depends on, such as a map display command, has been confirmed to be sent. In response to the strong barrier event not being sent or being constructed, the system intercepts the weak streaming event and puts it into a temporary storage area. The system waits for the data packet of the strong barrier event to be constructed, and then physically merges the data packet of the strong barrier event with the intercepted weak streaming event data packet in the temporary storage area, encapsulating them into the same indivisible network transmission unit, namely an atomic packet. The atomic packet contains unified header information, strong barrier event data, and a list of weak streaming events. The network transmission unit is sent to the target terminal, and the parser of the target terminal is configured to submit all events to the rendering thread at once only after the unit is completely unpacked and passes verification, ensuring processing within the same rendering frame.

[0125] This embodiment uses atomic encapsulation technology to enforce logical consistency at the physical transmission level, eliminating ghost annotations. This prevents the appearance of only annotations without a background page in weak network environments, ensuring the continuity of the user's visual experience.

[0126] Example 7:

[0127] If there is a mismatch, a full state synchronization is performed, including:

[0128] Suspend the target terminal's reception and processing of the current adapted instruction packet;

[0129] Obtain complete view state data at the current moment from the source;

[0130] Forcefully overwrite the current framebuffer contents of the target terminal and reset the target terminal's rendering queue.

[0131] This embodiment is a further specification of the full state synchronization steps in Embodiment 1; in response to the perceptual hash comparison result showing that the Hamming distance between the source and target ends exceeds a preset threshold, the system determines that the synchronization has failed and triggers an error correction mechanism; the system sends a pause command to the target terminal to suspend the target terminal's reception and processing of the current adaptation command packet to prevent further accumulation of error states; the system obtains the complete view state data at the current moment from the source end, that is, the full state package generated by serializing the current document model and annotation layers; in order to solve the data inconsistency problem caused by concurrent modifications, the source end uses copy-on-write technology to take a snapshot of the document object model DOM when performing serialization, to ensure that the main thread does not block the response to new input events during the generation of the full package, while ensuring the consistency of the time point of the exported state;

[0132] After receiving the full state packet, the target terminal does not perform difference merging but instead directly forces an overwrite of the current frame buffer content, redraws the entire screen using the full packet, resets the rendering queue, and clears all backlogged instructions. To prevent dirty data issues caused by network latency, the system introduces a global synchronization epoch mechanism. When sending a synchronization request, the target terminal increments its local epoch version number (EpochID), which is included in the source terminal's full state packet. During the forced overwrite and queue reset, the target terminal verifies the EpochID of all newly arriving data packets. Any lagging instruction packets carrying the old EpochID are discarded to prevent historical dirty data from contaminating the new state after reset. As a crucial step in the logical closed-loop of the full synchronization process, after completing the buffer overwrite, the system must parse the snapshot vector clock metadata in the full state packet header and force an atomic update of the target terminal's local logical clock state V. local =V snapshot , where V snapshot This is the global complete vector clock state extracted and attached by the source when generating a complete view state snapshot during serialization; this step is crucial because a full reset cuts off the original incremental dependency chain. If the local clock is not updated, subsequent legitimate weak streaming events will be incorrectly intercepted by the target terminal's causal consistency checker as future events or missing dependencies, leading to a new synchronization split.

[0133] This embodiment constructs a self-healing mechanism for the system. Even if a large amount of intermediate data is lost or corrupted due to extreme network conditions, the terminal state can be automatically restored to be completely consistent with the presenter's state within seconds through full coverage and Epoch version verification, without manual intervention, which greatly improves the robustness of the system.

[0134] Example 8:

[0135] Please see Figure 2 A paperless interactive information multi-terminal real-time synchronous processing system, including:

[0136] The logic building module is used to acquire the original interactive command stream and construct a semantic causal logic tree;

[0137] The status monitoring module is used to collect operational feedback data and calculate the rendering back pressure value;

[0138] The dynamic shaping module includes: a data processing unit, used to reduce and reorganize the instruction stream based on a semantic causal logic tree to generate an adapted instruction package; an atomic merging unit, used to merge weak streaming events and strong barrier events into an atomic state snapshot; and a transmission control unit, used to send the processed data to the target terminal.

[0139] The consistency verification module is used to compare the perceived hash value. It is configured to: trigger full state synchronization if there is a mismatch, and confirm synchronization consistency if there is a match.

[0140] This embodiment provides a hardware system architecture for executing the above method. The system includes a logic construction module whose core function is to maintain the semantic tree data structure and ensure the logical order of instructions. The status monitoring module is integrated into the terminal application layer and is responsible for collecting CPU and GPU load data in real time and calculating the rendering back pressure value according to the aforementioned formula. The dynamic shaping module, as the core scheduler of the system, includes three sub-units: the data processing unit has a built-in Bezier fitting algorithm library to perform dimensionality reduction operations and sequence recombination; the atomic merging unit is responsible for performing atomic encapsulation of data packets; the transmission control unit dynamically switches between full and incremental transmission channels according to the back pressure value; and the consistency verification module uses the GPU shader to quickly calculate the perceptual hash value of the screen content, completes visual consistency comparison without occupying CPU resources, and triggers synchronization or confirmation operations accordingly.

[0141] This embodiment uses modular and decoupled design to distribute complex semantic synchronization logic to various dedicated modules, enabling the system to have a strong adaptive capability to heterogeneous terminal environments. This effectively solves the problem of fragmentation in traditional conferencing systems under network fluctuations or equipment aging scenarios, and ensures the overall high availability of the system.

[0142] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for real-time synchronization of paperless interactive information across multiple terminals, characterized in that, include: The original interactive instruction stream generated by the source is obtained, and a semantic causal logic tree representing the instruction execution dependency relationship is constructed based on vector clock technology. The semantic causal logic tree divides the original interactive instruction stream into strong barrier events and weak streaming events. Real-time collection of operational feedback data from the target terminal; establishment of a rendering backpressure model reflecting the terminal load status; and calculation of the rendering backpressure value of the target terminal based on the rendering backpressure model. In response to the rendering backpressure value being higher than a preset congestion threshold, a dynamic data shaping process is triggered, including: Step 1, based on the dependency relationship of the semantic causal logic tree, the original interactive instruction stream is subjected to dimensionality reduction and temporal reorganization to generate an adapted instruction package; Step 2: Force the weak streaming event to be attached to the logically nearest preceding strong barrier event, and merge the two into an atomic state snapshot; Step 3: Send the adaptation instruction package or atomic state snapshot to the target terminal to perform the rendering operation; Step 4: Obtain the perceptual hash value of the screen content rendered at the key node on the target terminal, compare the perceptual hash value with the source baseline hash value, if they do not match, perform full state synchronization, if they match, confirm that the synchronization is consistent.

2. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, Constructing a semantic causal logic tree includes: Assign a logical timestamp to each instruction in the original interactive instruction stream; Commands that alter the global context state are identified as strong barrier events; Identify incremental modification commands in the current context as weak streaming events; Establish a one-way dependency constraint between weak streaming events and preceding strong barrier events, prohibiting weak streaming events from being rendered before the corresponding strong barrier events have completed execution.

3. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, Establish the rendering backpressure model, including: Extract rendering queue length, frame rate data, and CPU utilization from the runtime feedback data; The rendering latency index is calculated based on the difference between the rendering queue length and the preset processing capacity. Using a weighted summation algorithm or a preset mapping table, combined with the rendering latency index and the frame rate data, the current rendering backpressure value of the target terminal is calculated and generated.

4. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, Dimensionality reduction processing of the original interactive command stream includes: Since the original interactive command stream is a set of high-frequency continuous coordinate points, a Bézier curve fitting algorithm is used. Extract key control points from a set of high-frequency continuous coordinate points and generate parametric curve instructions; The original set of high-frequency continuous coordinate points is replaced with parametric curve instructions as part of the adaptation instruction package.

5. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, Timing reordering of the original interactive command stream includes: In response to the existence of multiple consecutive strong barrier events in the semantic causal logic tree, the intermediate frame discarding strategy is executed; The last strong barrier event in the sequence is retained, while strong barrier events in the middle are discarded. The weak streaming events following the last strong barrier event are realigned to the timeline of that last strong barrier event to generate a recombined instruction sequence.

6. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, The two are merged into an atomic state snapshot, including: Intercept weak streaming events that have not yet been sent; Waiting for the data packets of the strong barrier events it depends on to complete construction; Packets of strong barrier events and packets of intercepted weak streaming events are encapsulated into the same indivisible network transmission unit, ensuring that the target terminal processes the network transmission unit within the same rendering frame.

7. The paperless interactive information multi-terminal real-time synchronization processing method according to claim 1, characterized in that, If there is a mismatch, a full state synchronization is performed, including: Suspend the target terminal's reception and processing of the current adapted instruction packet; Obtain complete view state data at the current moment from the source; Forcefully overwrite the current framebuffer contents of the target terminal and reset the target terminal's rendering queue.

8. A paperless interactive information multi-terminal real-time synchronization processing system, applied to the paperless interactive information multi-terminal real-time synchronization processing method according to any one of claims 1-7, characterized in that, include: The logic building module is used to acquire the original interactive command stream and construct a semantic causal logic tree; The status monitoring module is used to collect operational feedback data and calculate the rendering back pressure value; The dynamic shaping module includes: a data processing unit, used to reduce and reorganize the instruction stream based on a semantic causal logic tree to generate an adapted instruction package; an atomic merging unit, used to merge weak streaming events and strong barrier events into an atomic state snapshot; and a transmission control unit, used to send the processed data to the target terminal. The consistency verification module is used to compare the perceived hash value. It is configured to: trigger full state synchronization if there is a mismatch, and confirm synchronization consistency if there is a match.

Images