Multi-input interactive screen system based on cloud collaboration

Abstract

The invention discloses a multi-input interactive screen system based on cloud collaboration, and particularly relates to the field of man-machine interaction, which comprises an interactive screen terminal, a communication module, an input analysis module, an action package generation module and a local collaboration presentation module, the interactive screen terminal comprises a processor, a memory, a display screen, and a touch control assembly, a handwriting pen assembly and an external input access assembly which are electrically connected with the processor, and is used for collecting input events from the touch control assembly, the handwriting pen assembly and the external input access assembly and forming an input event stream. Multi-source input is aligned into an input event stream, an action packet sequence carrying an action target identifier, an action constraint set, change amount data and target state verification information is generated through analysis, and a cloud side executes consistency judgment and conflict judgment based on a session rendering state and then issues a confirmation sequence to drive all terminals to update consistently.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and more specifically, to a multi-input interactive screen system based on cloud collaboration. Background Technology

[0002] In the field of edge-cloud collaboration for multi-input interactive screens, the mainstream practice in the industry aims to achieve real-time joint editing and synchronous presentation of multiple terminals. Typically, different inputs such as touch, stylus, remote control pointer, and mobile phone projection are uniformly abstracted into coordinate event streams or graphic command streams, which are then sent to the cloud or cached on the terminal and distributed to each collaborating terminal in chronological order to reconstruct the same screen. Taking the cloud-based collaborative whiteboard in the conference room as an example, participants may simultaneously use a stylus to write, drag and zoom objects with their fingers, and select objects with a remote control to trigger attribute changes. It also needs to meet the hard constraint of maintaining "consistency of what is seen and reproducibility of operation" even under the transmission jitter environment of cross-network isolation gateways. However, under the constraints of edge-cloud collaboration, the mainstream approach will consistently expose the bottleneck that the same input is interpreted as different results on different ends. Specific observable phenomena include: after network jitter, drag and stroke operations of different granularities are out of order, causing objects to appear on some ends as "scaled first and then covered by the old crop" or "scaled to the replaced object instance". In addition, the logs only record coordinates and time and cannot deduce the operation intention and the basis for determining the target object, so that conflicts cannot be consistently judged and merged and ultimately cause long-term screen splits between ends. The technical problem this application aims to solve is: in a scenario of multi-input mixed and network jittery end-to-end collaborative interactive screen, how to avoid inconsistent results of the same operation on different collaborative ends due to differences in input granularity and interpretation. Summary of the Invention

[0003] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a cloud-based collaborative multi-input interactive screen system. This system aligns multiple input sources into an input event stream and parses it to generate an action package sequence carrying an action target identifier, an action constraint set, change quantity data, and target state verification information. The cloud then performs consistency determination and conflict resolution based on the session rendering state and issues a confirmation sequence to drive each terminal to perform consistent updates under end-cloud collaboration, thereby solving the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a cloud-based collaborative multi-input interactive screen system, comprising an interactive screen terminal, a communication module, an input parsing module, an action package generation module, and a local collaborative presentation module; An interactive screen terminal includes a processor, a memory, a display screen, and a touch component, a stylus component, and an external input access component that are electrically connected to the processor, respectively, for collecting input events from the touch component, the stylus component, and the external input access component and forming an input event stream; The communication module, located on the interactive screen terminal and electrically connected to the processor, is used to establish a two-way communication connection with the cloud collaboration server through a network communication link, so as to send the action packet sequence corresponding to the input event stream and receive the confirmation sequence. The input parsing module is set in the interactive screen terminal and executed by the processor. It is used to parse the input event stream according to the preset parsing rules, determine the action target identifier, action constraint set and change amount data based on each input event, and output an action description triplet. An action package generation module is set on the interactive screen terminal and executed by the processor. It is used to generate action packages based on action description triples. The action package includes a session identifier, an action target identifier, an action constraint set, change quantity data, and target state verification information, and outputs a sequence of action packages to be synchronized. The local collaborative rendering module is set up on the interactive screen terminal and executed by the processor. It is used to write the sequence of action packages to be synchronized into the local collaborative buffer and to perform real-time updates on the rendering state based on the sequence of action packages to be synchronized in order to output the local rendering result.

[0005] In a preferred embodiment, it further includes: a cloud-based adjudication module; The cloud-based adjudication and distribution module, located on and executed by the cloud collaboration server, receives a sequence of action packages to be synchronized. For each action package, it locates the target object based on the action target identifier, limits the writable attribute fields according to the action constraint set, and performs validity judgment and conflict handling in conjunction with the target state verification information to generate the confirmation sequence. The confirmation sequence is then distributed to each interactive screen terminal participating in the same session, enabling each interactive screen terminal to perform a consistent update of its rendering state based on the confirmation sequence to output a consistent collaborative presentation result between the terminals.

[0006] In a preferred embodiment, the cloud-based adjudication distribution module is used to receive the sequence of action packets to be synchronized as adjudication input, and extract the session identifier, action target identifier, action constraint set, change quantity data and target status verification information for each action packet to form a set of adjudication fields per packet. Based on the action target identifier, the target object is located in the session rendering state maintained in the cloud and the set of attribute values ​​corresponding to the current writable attribute domain of the target object is read. The set of attribute values ​​is sorted by attribute identifier, and then the unit conversion and precision truncation are performed and concatenated into a state summary string. Then, the state summary string is compared with the target state verification information field by field to obtain the number of difference fields and the set of difference magnitudes, and the consistency judgment result is output accordingly. Based on the writable attribute domain limitation information in the action constraint set, filter out change records in the change quantity data one by one whose attribute identifier does not belong to the writable attribute domain, and merge similar items of the retained change records according to attribute identifier to obtain valid change quantity data. When concurrent action packets corresponding to the same action target identifier appear under the same session identifier, first extract the attribute identifier set contained in the effective change amount data of each concurrent action packet and determine whether there is an intersection between the attribute identifier sets; when there is no intersection, directly merge the effective change amount data of each concurrent action packet according to the attribute identifier and generate a decision action packet; when there is an intersection, for each concurrent action packet, based on the attribute value set corresponding to the current writable attribute domain, apply the effective change amount data of the concurrent action packet to the corresponding attribute identifier to obtain candidate update results, and generate a candidate state summary string for the candidate update results and compare it field by field with the target state verification information carried by the concurrent action packet to obtain the number of candidate difference fields and the set of candidate difference magnitudes. Select the concurrent action packet corresponding to the candidate update result with the smallest number of candidate difference fields and the smallest cumulative magnitude of the candidate difference magnitude set as the decision action packet; The adjudication action packets are grouped by session identifier, sorted in ascending order of transmission sequence number, and connected to form an acknowledgment sequence. This sequence is then sent via the network communication link to each interactive screen terminal participating in the same session, so as to output an acknowledgment sequence for each interactive screen terminal to perform consistent updates.

[0007] In a preferred embodiment, in the interactive screen terminal, the process of collecting input events from the touch component, the stylus component, and the external input access component and forming an input event stream includes: The processor acquires raw input samples from the touch component, stylus component, and external input access component, and writes an input source identifier, channel identifier, and local incrementing sequence number for each raw input sample to form a normalized input record set. The normalized input record set is then written with a relative timestamp within the source according to the input source identifier, and cross-source timing alignment is performed within a preset alignment window to obtain an aligned input record set. The aligned input record set is then concatenated into an input event stream according to a preset sorting rule and written to a local buffer, thereby outputting the input event stream.

[0008] In a preferred embodiment, the communication module is configured to perform the following steps to send the action packet sequence corresponding to the input event stream and receive the acknowledgment sequence: Based on the session identifier and the interactive screen terminal identifier, a connection request is initiated to the cloud collaboration server and a session establishment response is received to generate a two-way communication session bound to the session identifier; wherein the session identifier is used to uniquely identify a cloud collaboration session, and the interactive screen terminal identifier is used to uniquely identify a specific interactive screen terminal participating in the session; the cloud collaboration server is a server-side computing node used to carry the collaboration session and interact with each interactive screen terminal to realize collaborative processing and result distribution. Each action packet in the sequence of action packets to be sent is written with a sending sequence number and a sending window identifier. The written action packets are then sorted according to the sending sequence number and encapsulated into an uploading data frame to output an uploading data frame sequence. The sequence of uploaded data frames is sent to the cloud collaboration server via a network communication link, and a pending confirmation cache corresponding to the sending sequence number is maintained locally to output the set of sent pending confirmations; wherein the network communication link is a network transmission channel used to transmit data between the interactive screen terminal and the cloud collaboration server to establish and maintain a bidirectional communication connection. The system receives an acknowledgment sequence from the cloud collaboration server and parses it to obtain an acknowledgment sequence number set. Based on the acknowledgment sequence number set, it deletes confirmed action packets from the sent pending acknowledgment set and retransmits unconfirmed action packets to output the updated action packet sequence sending status. The acknowledgment sequence is generated by the cloud collaboration server after receiving the action packet sequence from the interactive screen terminal and completing validity determination and conflict handling. It is then sent back to the interactive screen terminal via the network communication link as an acknowledgment result for the action packet sequence. The acknowledgment sequence number set is derived from the sequence number field carried in the acknowledgment sequence. The sequence number field is used to indicate the corresponding sending sequence number of the action packet confirmed by the cloud collaboration server in the previously sent action packet sequence. The communication module parses and summarizes this sequence number field to obtain the acknowledgment sequence number set. The received confirmation sequence is deduplicated and rearranged according to its confirmation number set and then provided to the interactive screen terminal for subsequent consistency update processing, so as to output an ordered confirmation sequence.

[0009] In a preferred embodiment, the input parsing module includes: receiving an input event stream as parsing input in an interactive screen terminal, reading the input source identifier, event type identifier, event parameter set, and timestamp information for each input event in the input event stream, and outputting a set of event feature fields required for event-by-event parsing accordingly; The input parsing module further includes performing target determination processing on the event feature field set based on preset parsing rules. The preset parsing rules include: mapping the input source identifier and the event type identifier to a target positioning mode, wherein the target positioning mode is limited to an object index pointing mode or a coordinate hit mode; in the object index pointing mode, reading the object index parameter from the event parameter set and determining a unique candidate target accordingly; in the coordinate hit mode, reading the coordinate parameter from the event parameter set and performing a hit detection in the object boundary set of the current rendering state to obtain a candidate target set; when the candidate target set contains multiple candidate targets, selecting the candidate target with the smallest difference between its most recently selected timestamp and the input event timestamp as the unique target and outputting the corresponding action target identifier.

[0010] In a preferred embodiment, the input parsing module further includes generating an action constraint set based on the preset parsing rules. The preset parsing rules further include: retrieving a set of allowed action types from a pre-stored mapping table based on the input source identifier; retrieving a set of writable attribute fields from a pre-stored mapping table based on the event type identifier; reading a set of role permissions corresponding to the input event from the current collaboration state and intersecting it with the set of writable attribute fields to obtain a set of valid writable attribute fields; and encapsulating the set of allowed action types and the set of valid writable attribute fields into an action constraint set and outputting it. The input parsing module further includes generating change amount data based on the preset parsing rules. The preset parsing rules include: when an input event is determined to be a continuous input event, reading the coordinate parameters of the input event under the same action target identifier and calculating the difference with the coordinate parameters of the previous continuous input event to obtain the geometric change amount; when an input event is determined to be a discrete input event, reading the attribute identifier and attribute value from the event parameter set of the input event to obtain the attribute change amount; performing unified unit conversion and numerical truncation on the geometric change amount and the attribute change amount according to preset dimensions and precision rules to form the change amount data, and outputting an action description triplet composed of the action target identifier, the action constraint set, and the change amount data; wherein a discrete input event refers to an input event generated in a single triggering manner and not dependent on a continuous sampling sequence, and its event parameter set is used to represent the attribute value of a one-time action rather than a continuous coordinate trajectory; the attribute value refers to reading the corresponding parameter value or parameter enumeration value from the event parameter set of the input event according to the attribute identifier as the new value of the attribute, for example, reading the color parameter value when the attribute identifier is color, and reading the line width parameter value when the attribute identifier is line width.

[0011] In a preferred embodiment, the action package generation module is used to receive an action description triple as the generation input, and write the action target identifier, action constraint set and change quantity data into the action package field based on the session identifier to output the action package; Based on the action target identifier, read the state summary corresponding to the target object in the current rendering state of the interactive screen terminal as the target state verification information, and write the target state verification information into the action package to output the action package containing the target state verification information; Action packets generated under the same session identifier are written with incrementing sequence numbers in the order of generation and concatenated into a sequence of action packets to be synchronized, so as to output the sequence of action packets to be synchronized.

[0012] In a preferred embodiment, the local collaborative presentation module is used to receive the sequence of action packets to be synchronized as presentation input, and write the sequence of action packets to be synchronized into the local collaborative buffer in the order of writing to output the buffer write result; Read the action target identifier and change amount data sequentially from the action package sequence to be synchronized, and filter the change amount data by writable attribute domain according to the action constraint set to output valid change amount data; Based on the action target identifier, the target object in the current rendering state is located, and the effective change data is applied to the corresponding writable attribute field of the target object to update the current rendering state, thereby outputting the updated rendering state; The updated rendering state is converted into a display frame on the screen and the screen is driven to refresh to output the local rendering result; The sequence of action packets to be synchronized written to the local cooperative buffer is deduplicated and rearranged according to its sending sequence number to output an ordered sequence of action packets to be synchronized.

[0013] In a preferred embodiment, when the local collaborative rendering module locates the target object in the current rendering state based on the action target identifier and updates the current rendering state, it includes performing the following steps to output the updated rendering state: Step 1: Using the action target identifier as an index, retrieve the target object in the current rendering state and read the target state verification information and writable attribute domain set of the target object. Calculate the consistency cost based on the target state verification information and the effective change amount data to be applied, and generate a candidate update set to output the candidate update set. The target state verification information is a verification summary used to characterize the current state of the target object, and the writable attribute domain set is a set of attributes that are allowed to be changed by the target object in the current cooperative state. Step 2: In the candidate update set, the candidate update with the minimum consistency cost and that satisfies the constraints of the writable attribute domain set and the local rendering latency limit is selected as the optimal update. The update confidence of the optimal update is calculated and merged with the historical update confidence of the target object to output the update confidence. For each candidate update, the process includes first breaking it down into several attribute change records and verifying whether the attribute identifier of each attribute change record belongs to the set of writable attribute domains. Then, according to the pre-stored rendering time estimation rules, the unit rendering time corresponding to each attribute change record is accumulated to obtain the estimated rendering time and it is determined whether it does not exceed the local rendering latency limit. Subsequently, the consistency cost is calculated only among the candidate updates that pass the above two determinations, and the candidate update with the smallest consistency cost value is selected as the optimal update. Step 3: When the update confidence is not lower than the preset confidence threshold and the consistency cost is not higher than the preset cost threshold, the optimal update is applied to the corresponding writable attribute domain of the target object to update the current rendering state. If the conditions are not met, the effective change data is truncated or quantized according to the preset approximation rules to generate approximate change data. The candidate update generation and optimal update selection are repeated until the stopping condition is met, thereby outputting the updated rendering state. The preset confidence threshold and the preset cost threshold are pre-configured as numerical thresholds by the interactive screen terminal during deployment. The preset confidence threshold is used to limit the confidence of the optimal update to not be lower than the threshold, and the preset cost threshold is used to limit the consistency cost of the optimal update to not be higher than the threshold. Both can be selected as the current value according to the cooperation mode or network status. The preset approximation rules are pre-configured by the interactive screen terminal as a set of rules for performing quantization step size rounding and amplitude truncation on valid change data. The quantization step size and truncation upper limit are used to determine the rounding granularity and the maximum allowable change amount of the change data, respectively. When the threshold judgment is not met, approximate change data is generated according to the rules to re-enter the candidate update generation and optimal update selection process.

[0014] The technical effects and advantages of this invention are as follows: 1. By writing target state verification information into the action package and having the cloud compare and adjudicate field by field, the target and semantic meaning of the same operation in the end-to-cloud collaboration process are unified, which can relatively suppress long-term screen splitting between ends due to jitter and disorder. 2. By generating input event streams through cross-source alignment and determining action target identifiers according to target positioning patterns, and combining the most recently selected records to achieve uniqueness, the probability of the same input hitting different object instances in edge-cloud collaboration scenarios is reduced; 3. By intersecting input source mapping, event type mapping and role permissions, an action constraint set is generated and the writable attribute domain is limited to reduce inconsistent updates on the end side caused by unauthorized or out-of-domain writes during end-cloud collaboration. 4. By generating change data through continuous event difference and discrete event value taking, and unifying the unit and precision truncation, the change scope under different input granularities can be aligned, which relatively improves the consistency of end-to-cloud reproduction. 5. By first determining the intersection of attributes of concurrent action packages in the cloud, then simulating the application and selecting the one with the smallest difference, a deterministic decision action package is obtained and sorted and distributed, which relatively alleviates the merging discrepancies caused by concurrent conflicts. 6. By sending sequence numbers to drive the deletion and retransmission of pending acknowledgments, and by deduplicating and rearranging the acknowledgment sequence for consistent updates, the impact of packet loss and out-of-order delivery on the synchronization order is reduced. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Refer to the instruction manual appendix Figure 1 The present invention provides a cloud-based collaborative multi-input interactive screen system, comprising an interactive screen terminal, a communication module, an input parsing module, an action package generation module, a local collaborative presentation module, and a cloud-based adjudication and distribution module; An interactive screen terminal includes a processor, a memory, a display screen, and a touch component, a stylus component, and an external input access component that are electrically connected to the processor, respectively, for collecting input events from the touch component, the stylus component, and the external input access component and forming an input event stream; The purpose of this embodiment is to aggregate the multiple input sources from the touch component, stylus component, and external input access component of the interactive screen terminal into an input event stream that can be sequentially read by the subsequent input parsing module, using a unified data standard on the terminal side. This ensures that even under the objective constraints of multiple parallel inputs, different sampling frequencies, and arrival timing jitter, the event order can still be verified, fields can be retrieved, and the alignment process can be executed, providing a stable input foundation for determining subsequent action target identifiers, action constraint sets, and change quantity data. This implementation process includes the following steps: First, the processor reads raw input samples from the touch component, stylus component, and external input access component respectively. The raw input samples include at least the sampling time and event parameter set. The event parameter set for the touch component includes touch point coordinates and touch point state, the event parameter set for the stylus component includes pen tip coordinates and pressure value, and the event parameter set for the external input access component includes peripheral type identifier and peripheral event parameters. Then, the processor writes an input source identifier and a channel identifier to each raw input sample, and maintains a local incrementing sequence number under the same input source identifier and the same channel identifier to ensure the uniqueness of the order within the source. This encapsulates the raw input samples from multiple sources into a normalized input record and aggregates them to obtain a normalized input record set. When any raw input sample is missing an event parameter set or the sample value exceeds the allowed value range, the raw input sample is marked as an invalid record and skipped from writing to avoid contaminating subsequent alignment. Next, the processor maintains intra-source relative timestamps for the normalized input record set according to the input source identifier. These intra-source relative timestamps are generated by the local timing reference of the input source and monotonically incremented according to the local incrementing sequence number, so that the relative order within the same input source can be directly determined by the intra-source relative timestamps. Then, the processor performs cross-source timing alignment in units of a preset alignment window. The preset alignment window is pre-configured by the terminal side as a time length parameter and is used to limit the range of intra-source relative timestamps included in one alignment. Within each alignment window, the processor maps the normalized input records of each input source to a unified alignment timeline within the window according to the intra-source relative timestamps, and sorts multiple normalized input records falling into the same alignment time slice according to a preset sorting rule. The processor determines the cross-source order, and the preset sorting rule is pre-configured by the terminal as a priority rule and is determined based on at least the input source identifier and channel identifier to obtain an aligned input record set. Finally, the processor connects the aligned input record set into an input event stream according to the preset sorting rule, and writes the input event stream into the local buffer in a sequential append manner for subsequent reading. When the remaining capacity of the local buffer is insufficient, the processor first clears the historical records that have been read and confirmed by the input parsing module from the head of the buffer according to the local incrementing sequence number before writing them into the current input event stream. If it is still insufficient, it suspends the reception of non-critical input events from the external input access component and records the discard count to ensure that the input event stream can be continuously generated and can be stably consumed by subsequent steps. Through the above method, the interactive screen terminal can converge input events into an input event stream and write it to a local buffer under the condition of multiple parallel input sources and jitter in arrival time. This is achieved by using a unified field, traceable sequence number within the source, and cross-source verifiable sorting rules. This reduces ambiguity and uncertainty in the subsequent parsing stage and makes the source, order, and alignment criteria of each input event verifiable and reproducible. In practical applications, such as in a multi-terminal collaborative whiteboard scenario in a conference room, one user writes a trajectory with a stylus while another user simultaneously drags an object with their finger, and an external input access component is connected to a remote-controlled page turner for triggering. When switching pages, the processor collects the pen tip coordinates and pressure values, touch point coordinates and touch point status, and page turner event parameters, and writes them into the corresponding input source identifier and local incrementing sequence number. Then, according to the preset alignment window, the three types of inputs are mapped to a unified alignment time axis and the order within the same time slice is determined according to the preset sorting rules. The results are connected to form an input event stream and written to the local buffer. The input parsing module then reads and parses the input event stream in order, which avoids target misselection and change calculation jumps caused by out-of-order or missing fields, thereby supporting the stable execution of subsequent action package generation and inter-end consistent updates.

[0018] The communication module, located on the interactive screen terminal and electrically connected to the processor, is used to establish a two-way communication connection with the cloud collaboration server through a network communication link, so as to send the action packet sequence corresponding to the input event stream and receive the confirmation sequence. The purpose of this embodiment is to enable the interactive screen terminal to reliably and orderly send the action packet sequence generated on the terminal side to the cloud collaboration server under the objective constraints of network jitter, packet loss, and out-of-order delivery, and to reliably and orderly deliver the acknowledgment sequence returned from the cloud to the interactive screen terminal for subsequent consistent updates. Its basic principle is to use a bidirectional communication session bound by a session identifier as the transmission boundary, to form a retransmittable upload data frame sequence using the sending sequence number and sending window identifier, and to drive the deletion of acknowledged and retransmitted unacknowledged data using the unacknowledged cache and acknowledgment sequence number set, thereby converging the uncertainty of the communication link into an executable acknowledgment and retransmission process. This implementation process includes the following steps: First, to establish a traceable transmission context between the device and the cloud and ensure consistency between subsequent sequence numbers and cached information, the communication module initiates a connection request to the cloud collaboration server using a session identifier and an interactive screen terminal identifier as connection inputs. The session identifier uniquely identifies a single cloud collaboration session and is generated by the interactive screen terminal when creating the session and written into subsequent action packet fields. The interactive screen terminal identifier uniquely identifies a specific interactive screen terminal participating in the session and is written into the terminal's local configuration area during factory configuration or initial registration for the communication module to read. When sending the connection request, the communication module writes the session identifier and the interactive screen terminal identifier into the connection message and sends it to the cloud collaboration server via the network communication link, subsequently receiving a response from the cloud collaboration server. The session establishment response reads the session acceptance flag and session parameter set and generates a bidirectional communication session bound to the session identifier accordingly. The session parameter set includes at least the maximum allowed sending window size and heartbeat maintenance period for this session and is issued by the cloud collaboration server in the session establishment response. After the bidirectional communication session is established, the communication module takes the sequence of action packets to be sent as the input and writes a sending sequence number and a sending window identifier to each action packet in the action packet sequence. The sending sequence number is incremented in the order of generation under the same session identifier and is obtained by reading from the local sequence number counter and incrementing it by one. The sending window identifier is calculated based on the number of currently unconfirmed action packets and the maximum sending window size and is used to mark the window batch to which the action packet belongs. The communication module sorts the written action packets by their sending sequence number and encapsulates them into upload data frames to output an upload data frame sequence. Each upload data frame contains at least a session identifier, a sending window identifier, and one or more action packets arranged by their sending sequence number within the same window. Subsequently, the communication module sends the upload data frame sequence to the cloud collaboration server via the network communication link and establishes a pending confirmation cache locally using the sending sequence number as an index to output a set of sent pending confirmations. The pending confirmation cache records at least the sending sequence number, the corresponding action packet content, the sending timestamp, and the retransmission count. When the network communication link is unavailable or the transmission fails, the communication module marks the upload data frame as pending transmission and retryes transmission according to the heartbeat maintenance period. When the retransmission count reaches the preset upper limit, the action packet is marked as an upload exception and written to the local log for subsequent diagnosis without affecting the continued transmission of other windows. Next, in order to align the cloud-based adjudication results with the terminal-side pending confirmation cache and drive reliable retransmission, the communication module receives the confirmation sequence from the cloud collaboration server and parses the confirmation sequence to obtain the confirmation sequence number set. The confirmation sequence is generated by the cloud collaboration server after receiving the action packet sequence sent by the interactive screen terminal and completing the validity judgment and conflict handling. The sequence number field carried in the confirmation sequence is used to indicate the corresponding sending sequence number of the action packet confirmed by the cloud collaboration server in the previously sent action packet sequence. The communication module reads the sequence number field one by one and summarizes it to obtain the confirmation sequence number set. The communication module takes the confirmation sequence number set and the sent pending confirmation set as input, deletes the action packets in the sent pending confirmation set whose sending sequence number belongs to the confirmation sequence number set to release the corresponding cache record, and retransmits the action packets in the sent pending confirmation set whose sending sequence number does not belong to the confirmation sequence number set and whose sending timestamp exceeds the preset retransmission waiting time to output the updated action packet sequence sending status. The preset retransmission waiting time is pre-configured by the terminal side or issued by the cloud in the session establishment response and is used to limit the upper limit of the waiting time for confirmation after a single transmission. After deletion and retransmission, the communication module performs deduplication and rearrangement on the received acknowledgment sequences according to their acknowledgment sequence number set to output an ordered acknowledgment sequence. Deduplication is used to remove acknowledgment items corresponding to the same sending sequence number that appear repeatedly in the acknowledgment sequence, and rearrangement is used to sort the acknowledgment items in ascending order of sending sequence number to eliminate the impact of network out-of-order. The ordered acknowledgment sequence is then provided to the interactive screen terminal for subsequent consistent update processing. When the acknowledgment sequence is missing, parsing fails, or the acknowledgment sequence number set is empty, the communication module keeps the already sent acknowledgment set unchanged and triggers the retransmission waiting timer logic. If a valid acknowledgment sequence is not obtained after the preset timeout number is reached, the session status is marked as weak network abnormal and the maximum sending window size is reduced to reduce the number of unacknowledged action packets in transit, thereby avoiding buffer bloat and maintaining the continuous operation of the session. Through the above steps, after establishing a two-way communication session, the communication module can organize the action packet sequence into a sequence of acknowledgable and retransmitable uploaded data frames using the sending sequence number and sending window identifier. It also uses the pending acknowledgment buffer and acknowledgment sequence number set to delete acknowledged action packets and retransmit unacknowledgment action packets. Simultaneously, it deduplicates and rearranges the acknowledgment sequence returned from the cloud into an ordered acknowledgment sequence for delivery to the terminal side for consistent update processes. This ensures that action packet transmission and acknowledgment processing are executable, traceable, and that the references to preceding and following fields remain consistent even under weak network and out-of-order conditions. In practical applications, for example, when multiple interactive screen terminals participate in the same meeting… In a conference whiteboard scenario, a terminal generates and uploads multiple action packets in succession due to short-term packet loss and latency fluctuations in the wireless network. The communication module writes an incrementing sequence number to each action packet and encapsulates and sends it according to a window. At the same time, it establishes a cache of pending confirmations locally. After cloud adjudication, it returns a confirmation sequence containing the sequence numbers. The communication module parses the confirmation sequence number set, deletes confirmed action packets, and retransmits timed-out unconfirmed action packets. Finally, it deduplicatizes and rearranges the confirmation sequence into an ordered confirmation sequence and delivers it to the local consistent update logic, so that the terminal can still achieve consistent presentation with other terminals based on the same confirmation sequence after the network is restored.

[0019] The input parsing module is set in the interactive screen terminal and executed by the processor. It is used to parse the input event stream according to the preset parsing rules, determine the action target identifier, action constraint set and change amount data based on each input event, and output an action description triplet. The purpose of this embodiment is to enable interactive screen terminals to stably convert each input event in the input event stream into an action description triplet that can be used for subsequent action package generation and cloud-based adjudication, even under constraints such as multiple parallel inputs, diverse event types, and constantly changing object states. The underlying process involves first extracting a set of computable event feature fields from each input event stream, then determining the action target identifier according to preset parsing rules and achieving target uniqueness in the current rendering state. Subsequently, an action constraint set is generated according to the same set of preset parsing rules to limit allowed actions and writable attribute domains. Finally, under the constraints of the action target identifier and the action constraint set, the change amount data is calculated and its dimensions and precision are normalized, thereby outputting an action description triplet composed of the action target identifier, the action constraint set, and the change amount data. This implementation process includes the following steps: To ensure that subsequent target determination, constraint generation, and change calculation all have valid input values ​​and consistent field definitions, the input parsing module receives the input event stream as parsing input in the interactive screen terminal. It reads each input event sequentially, including its input source identifier, event type identifier, event parameter set, and timestamp information, and writes these fields into a per-event feature record to form an event feature field set. The input source identifier is determined by the source channel of the touch component, stylus component, or external input access component and is used to distinguish the input source. The event type identifier indicates whether the input event is a continuous trajectory event or a discrete trigger event, and its specific interaction type. The event parameter set carries the event's coordinate parameters, object index parameters, or attribute identifiers and values. The timestamp information indicates the time the event occurred in the input event stream to support the selection of subsequent candidate targets. When any input event lacks an event type identifier or its event parameter set is empty, the input event is marked as invalid and removed from the event feature field set to prevent unexecutable paths from being generated during subsequent parsing. The input parsing module then executes the target on the event feature field set based on preset parsing rules. The process involves pre-configuring a set of mapping and judgment rules by the interactive screen terminal, including at least a target positioning mode mapping rule and a target uniqueness rule. The input parsing module first uses the input source identifier and event type identifier as mapping input to determine the target positioning mode, which is limited to either object index pointing mode or coordinate hit mode. When the target positioning mode is object index pointing mode, the module reads the object index parameter from the event parameter set and retrieves the unique candidate target from the object index table of the current rendering state. When the target positioning mode is coordinate hit mode, the module reads the coordinate parameter from the event parameter set and performs a hit detection in the object boundary set of the current rendering state to obtain a candidate target set. When the candidate target set contains multiple candidate targets, the input parsing module reads the most recent selected timestamp of each candidate target from the current rendering state and calculates the difference between the timestamp and the input event timestamp. The candidate target with the smallest difference is selected as the unique target, and the corresponding action target identifier is output. If the candidate target set is empty, the input event is marked as a targetless event, and an empty action target identifier is output so that subsequent steps can skip the change amount calculation or enter the default target strategy. To ensure that the executable scope under different input sources, different role permissions, and different event types can be uniformly converged and that consistency is maintained between the client and the cloud, the input parsing module, after obtaining the action target identifier, continues to generate an action constraint set based on preset parsing rules. This generation uses the input source identifier, event type identifier, and current collaboration state as input. First, the input parsing module retrieves the set of allowed action types from a pre-stored mapping table based on the input source identifier. This pre-stored mapping table is pre-configured by the interactive screen terminal during deployment and stores the list of action types allowed to be triggered by that input source using the input source identifier as the key. Next, the input parsing module retrieves the set of writable attribute fields from the pre-stored mapping table based on the event type identifier. This pre-stored mapping table is also pre-configured by the interactive screen terminal during deployment and stores the set of writable attribute fields using the event type identifier as the key. The input parsing module then reads the set of role permissions corresponding to the input event from the current collaboration state. The set of role permissions is parsed by the terminal from the session permission parameters sent from the cloud after the session is established and cached according to the session identifier. The input parsing module intersects the set of role permissions with the set of writable attribute fields to obtain the set of valid writable attribute fields, and encapsulates the set of allowed action types and the set of valid writable attribute fields into a set of action constraints and outputs it. When any mapping table does not match the corresponding key value or the set of role permissions is missing, the input parsing module outputs an empty set of action constraints and marks the input event as a restricted event so that it can be rejected for encapsulation or encapsulated into an action package that can only be replayed and not written in the subsequent action package generation stage, thereby avoiding uncontrollable state drift caused by unauthorized writing. To ensure that the change amount has a clear calculation rule between continuous trajectory inputs and discrete trigger inputs, and to achieve consistent numerical caliber across different terminals, the input parsing module continues to generate change amount data based on preset parsing rules after generating the action constraint set. These preset parsing rules are pre-configured by the interactive screen terminal as a set of event judgment rules, differential calculation rules, and dimensional accuracy normalization rules. The input parsing module first determines whether the input event is a continuous or discrete input event based on the event type identifier. When the input event is determined to be a continuous input event, the input parsing module reads the coordinate parameters of the input event under the same action target identifier and reads the coordinate parameters of the previous continuous input event from the terminal-side cache. The terminal-side cache stores the coordinate parameters of the most recent continuous input event indexed by the action target identifier for differential calculation. The input parsing module subtracts the two coordinate parameters dimension by dimension to calculate the difference to obtain the geometric change amount and writes the current coordinate parameters back to the terminal-side cache. When the input event is determined to be a discrete input event, the input parsing module reads the attribute identifier and attribute value from the event parameter set of the input event to obtain the attribute change amount. Discrete input events are those generated in a single trigger manner and do not depend on continuous sampling sequences. Their event parameter sets represent the attribute values ​​of a one-time action rather than continuous coordinate trajectories. The attribute values ​​are the parameter values ​​or parameter enumeration values ​​read from the event parameter set according to the attribute identifier, which are used as the new values ​​of the attribute. The input parsing module then performs unified unit conversion and numerical truncation on the geometric change and attribute change according to preset dimensions and precision rules to form change data. The preset dimensions and precision rules are pre-configured by the interactive screen terminal during deployment as a set of unit conversion factors and truncation precision parameters. The unit conversion factor is used to convert the coordinate difference to a unified display unit, and the truncation precision parameters are used to truncate or round the change by a specified decimal place or a specified step size to eliminate the resolution differences of different input sources. When the coordinate parameters of the previous continuous input event are missing or the cache is not initialized, the input parsing module sets the geometric change to zero and only writes back the coordinate parameters of the current time to complete the cache initialization. Finally, the input parsing module outputs an action description triple consisting of the action target identifier, the action constraint set, and the change data for the action package generation module to read and encapsulate. Through the above steps, the input parsing module can stably map the input event stream into action target identifiers by extracting field-by-event, mapping target localization patterns, and using candidate target uniqueness rules. It then generates an action constraint set to limit the executable range by retrieving the mapping table between the input source identifier and the event type identifier, and by intersecting the role permission set. Simultaneously, it generates change data by combining continuous event differencing with discrete event values, supplemented by dimensional and precision normalization rules. This results in a clearly structured and verifiable action description triplet, reducing the risk of out-of-bounds writing, target drift, and inconsistent numerical values ​​in subsequent action package generation and cloud adjudication. In practical applications, for example in collaborative whiteboard scenarios, when dragging and dropping objects, the input parsing module maps the touch input source identifier and drag event type identifier into a coordinate hit pattern and obtains a candidate target set through object boundary hit detection. Then, based on the most recent... The action target identifier is determined by the rule of minimizing the difference between selected timestamps. At the same time, the set of allowed action types and the set of writable attribute fields are retrieved from the mapping table and intersected with the set of role permissions in the current collaboration state to obtain the set of valid writable attribute fields. Then, the geometric change amount is obtained by subtracting the coordinate parameters of two consecutive touch events and truncating it according to the preset units and precision rules to form the change amount data. When the user clicks the toolbar to change the line width, the input parsing module maps the event type identifier to the object index direct pointing mode and reads the object index parameter to locate the target object. Then, it reads the attribute identifier and attribute value corresponding to the line width from the event parameter set to form the attribute change amount and performs unit and precision normalization. Finally, both types of input are uniformly output as action description triples for subsequent action package generation and consistent cloud adjudication. This allows the parsing process in multi-input mixed scenarios to still be executed on-premises and consistent between the end and the cloud.

[0020] An action package generation module is set on the interactive screen terminal and executed by the processor. It is used to generate action packages based on action description triples. The action package includes a session identifier, an action target identifier, an action constraint set, change quantity data, and target state verification information, and outputs a sequence of action packages to be synchronized. The purpose of this embodiment is to encapsulate the action description triples output by the input parsing module into action packets that can be sent, adjudicated, and traced on the interactive screen terminal side, and further organize multiple action packets into an ordered sequence of action packets to be synchronized under the same session identifier. This enables the cloud collaboration server to perform validity judgment and conflict handling based on the action target identifier, action constraint set, change quantity data, and target state verification information, and enables the communication module to drive confirmation, retransmission, and deduplication and rearrangement with the sending sequence number. The process is as follows: first, the action description triples are written into the action packet field according to the session identifier to form the smallest transmissible unit; then, the target object state summary is extracted from the current rendering state to generate target state verification information and written into the action packet to form a verifiable unit; finally, incremental sending sequence numbers are written into the action packets under the same session identifier and concatenated into a sequence to form a sortable and retransmittable unit. This implementation process includes the following steps: To ensure the source of action package fields is clear and that subsequent cloud-based adjudication and end-side communication can directly access the required fields, the action package generation module receives the action description triplet from the interactive screen terminal as generation input. It then reads the action target identifier, action constraint set, and change quantity data from the action description triplet. Simultaneously, it reads the session identifier from the current session context of the interactive screen terminal as attribution input. The session identifier is generated by the interactive screen terminal when establishing a two-way communication session or written to the local session cache from the session establishment response for the module to read. The action package generation module then allocates the action package data structure in memory and stores the session identifier, action target identifier, action constraint set, and change quantity data. The quantity data is written into the action package fields one by one according to the field name to output the action package. The action constraint set is written in the form of a combination of the set of allowed action types and the set of valid writable attribute fields, so that the cloud adjudication and distribution module can limit the writable attribute fields. The change quantity data is written in the form of geometric change quantity or attribute change quantity records for subsequent rendering state updates. When the action description triplet is missing the action target identifier or the action constraint set is empty, the action package generation module marks the action package as a restricted action package and writes it into the restricted flag field. At the same time, the session identifier and the original change quantity data are retained for subsequent recording and playback without entering the upload queue, thereby avoiding the cloud adjudication from mishandling of targetless or unauthorized actions. To enable the cloud to determine the target object's perceived state at the time of action package generation and to perform consistency checks and conflict resolution accordingly, the action package generation module, after outputting the action package, reads the state summary corresponding to the target object from the current rendering state of the interactive screen terminal as target state verification information based on the action target identifier. The current rendering state of the interactive screen terminal includes at least an object index table and an object attribute table. The action package generation module locates the target object in the object index table using the action target identifier and reads the set of attribute values ​​corresponding to the writable attribute fields related to the target object from the object attribute table as the summary input. The action package generation module then sorts the attribute value set by attribute identifier, performs unit conversion and numerical truncation according to preset units and precision rules, and concatenates them into a state summary string. The summary string is used as target state verification information and written into the target state verification information field of the action package, thereby outputting an action package containing target state verification information. The preset units and precision rules are pre-configured by the interactive screen terminal during deployment as a set of unit conversion factors and truncation precision parameters, and are consistent with the caliber used by the input parsing module, so as to ensure that the summary construction of the same attribute on the terminal side and the cloud side can be aligned. When there is no target object corresponding to the action target identifier in the current rendering state or the set of read attribute values ​​is empty, the action package generation module writes the target state verification information as empty and writes the target missing flag into the action package. At the same time, the action package is written to the local exception log for subsequent diagnosis, thereby avoiding inconsistent judgments caused by the cloud using erroneous summaries for difference comparison. To ensure that action packets can form a acknowledgable, deduplicatable, reorderable, and retransmittable sequence structure under the same session identifier, the action packet generation module, after receiving the action packet containing target state verification information, writes an incrementing sequence number to the action packets generated under the same session identifier in the order of generation and concatenates them into a sequence of action packets to be synchronized. The incrementing sequence number is generated by a session sequence number counter maintained locally by the action packet generation module. This counter stores the most recently allocated sequence number using the session identifier as an index and increments by one each time a new action packet is generated to obtain the current sequence number. The action packet generation module writes the current sequence number into the sequence number field of the action packet and appends the written action packet to the end of the sequence of action packets to be synchronized in the order of writing to output the action packets to be synchronized. The action packet generation module generates a sequence of action packets and simultaneously writes the start and end sequence numbers of the action packet sequence to be synchronized into the sequence metadata for subsequent window encapsulation and maintenance of the pending confirmation cache by the communication module. When the session sequence number counter is not initialized, the action packet generation module initializes the sending sequence number to a preset starting value and writes an initialization flag. The preset starting value is pre-configured by the terminal side or fixed to the minimum positive integer to ensure that the sequence number is monotonically increasing and comparable. When the action packet sequence to be synchronized reaches the preset queue length limit, the action packet generation module suspends receiving new action description triplets and reports the queue congestion status to the local collaborative presentation module. The local collaborative presentation module reduces the real-time update frequency or merges continuous geometric changes to reduce the action packet generation rate, thereby avoiding uncontrolled memory usage caused by unbounded sequence growth. Through the above steps, the action package generation module can encapsulate the action target identifier, action constraint set, and change quantity data into an action package using a session identifier. It also constructs target state verification information using the state summary of the target object in the current rendering state and writes it into the action package, thus providing a verifiable basis for cloud-based adjudication. Simultaneously, it connects multiple action packages into a sequence of action packages to be synchronized using an incrementally increasing sequence number within the session and outputs it to the communication module. This provides the foundation for subsequent sequence number alignment required for confirmation sequence parsing, confirmation cache maintenance, and retransmission processing. In practical applications, for example in a collaborative whiteboard scenario, each action package generated by a user continuously writing with a stylus... After the input event is processed by the input parsing module, which outputs an action description triplet, the action package generation module reads the session identifier and writes it into the action package field. At the same time, based on the action target identifier, it reads the line attribute value of the target object from the current rendering state and constructs a state summary string as the target state verification information, which is then written into the action package. An incrementing sending sequence number is written to each action package and appended to the sequence of action packages to be synchronized. The communication module encapsulates and sends the package according to the sending window. After receiving the confirmation sequence returned by the cloud, it deletes the confirmed and resends the unconfirmed. This ensures that the action package can still be adjudicated by the cloud under weak network jitter and ultimately drive consistent presentation across multiple terminals.

[0021] The local collaborative rendering module is set on the interactive screen terminal and executed by the processor. It is used to write the sequence of action packages to be synchronized into the local collaborative buffer and to perform real-time updates on the rendering state based on the sequence of action packages to be synchronized in order to output the local rendering result. The purpose of this embodiment is to enable interactive screen terminals, under the latency constraints of real-time interaction on the device side, to extract effective changes, locate target objects, and update the rendering state of the action packet sequence to be synchronized under the constraints of writable attribute domains. The updated rendering state is then converted into display frames in real time to drive screen refresh. Simultaneously, the order of the action packet sequence to be synchronized is maintained in the buffer for subsequent alignment with the cloud-confirmed sequence. The process involves first writing the action packet sequence to be synchronized into a local collaborative buffer to form traceable input; then filtering the change data according to the action constraint set to obtain effective change data and updating the current rendering state accordingly to output the local rendering result; subsequently, the action packets in the buffer are deduplicated and rearranged according to their sending sequence number to form an ordered action packet sequence to be synchronized. In the specific update process, the optimal update is further limited by consistency cost, writable attribute domain set constraints, and local rendering latency upper limit constraints. Approximate backoff iteration is triggered by update confidence thresholds and cost thresholds to ensure convergent updates with controllable latency. This implementation process includes the following steps: To ensure the traceability of input and the reproducibility of output in the end-side presentation link, the local collaborative presentation module receives the sequence of action packets to be synchronized as presentation input and appends the sequence of action packets to be synchronized to the local collaborative buffer in the order of writing to output the buffered write result. The local collaborative buffer is pre-allocated in the memory of the interactive screen terminal as a circular queue structure and stores the action packet records of this session using the session identifier as the distinguishing key. The upper limit of the buffer capacity is pre-configured by the terminal during deployment as an upper limit of the number of records or a upper limit of the byte capacity to control storage usage. Subsequently, the local collaborative presentation module reads the action target of each action packet from the sequence of action packets to be synchronized in ascending order of the sending sequence number. The system takes the set of knowledge, action constraints, and change data as update inputs. Based on the set of valid writable attribute fields in the action constraint set, it performs writable attribute field filtering on each change data entry to output valid change data. The filtering rule is to read the attribute identifier in each change data entry and determine whether the attribute identifier belongs to the set of valid writable attribute fields. If it belongs, the change record is retained and written into the valid change data; otherwise, the change record is discarded and written into the discard count for subsequent statistics. When the valid change data is empty, the local collaborative rendering module skips the current rendering state update and only marks the action package as an invalid change package and writes it into the buffer to avoid meaningless refreshes and state drift. After obtaining valid change data, the local collaborative rendering module locates the target object in the object index table of the current rendering state based on the action target identifier and reads the set of writable attribute fields and the current attribute value set of the target object as the application benchmark. Then, it applies the valid change data to the corresponding writable attribute fields of the target object one by one to update the current rendering state. The application rule is to locate the corresponding attribute of the target object according to its attribute identifier for each change record and perform an additive update or assignment update on the attribute value according to the change amount. The additive update is used for geometric change type attributes to realize coordinate translation or size increment, and the assignment update is used for attribute change type attributes to realize attribute overlay such as color and line width. After the update is completed, the updated attribute value set is written back to the current rendering state to output the updated rendering state. Subsequently, the local collaborative rendering module converts the updated rendering state into a display frame of the display screen and drives the display screen to refresh through the processor to output the local rendering result. The display frame is generated by the rendering engine according to the updated rendering state and written to the frame buffer. The display screen refresh is read from the frame buffer by the display control interface according to the refresh cycle and output to the display screen. Finally, the local collaborative presentation module performs deduplication and rearrangement on the sequence of action packets to be synchronized written to the local collaborative buffer according to their sending sequence number to output an ordered sequence of action packets to be synchronized. The deduplication rule is to retain the first occurrence of the action packet in the buffer index table with the sending sequence number as the key and delete the action packet record corresponding to the subsequent duplicate sending sequence number. The rearrangement rule is to relink the action packet records in the buffer in incrementing order of sending sequence number to form a continuous traversable sequence, thereby providing ordered input for subsequent consistent updates aligned with the acknowledgment sequence. When the buffer capacity is insufficient to write new action packets, the local collaborative presentation module first cleans up the action packet records that have been acknowledged by the communication module and retains the unacknowledgment action packet records. If it is still insufficient, it triggers the congestion flag and reduces the writing frequency of the local presentation module to continuous geometric changes to reduce the generation rate of subsequent action packets. To ensure that interpretable update results that meet latency constraints are still output even when the object state may be affected by both local real-time updates and subsequent cloud confirmation, the local collaborative rendering module further executes a consistency cost-driven candidate update selection process when locating the target object based on the action target identifier and updating the current rendering state: First, the target object is retrieved in the current rendering state using the action target identifier as an index, and its target state verification information and writable attribute domain set are read. The target state verification information is a state digest string written during action package generation and used to characterize the target object's verification summary during action package generation. The writable attribute domain set is the set of attribute identifiers allowed to be changed in the current collaborative state, obtained by intersecting the action constraint set. The local collaborative rendering module uses the target state verification information and the effective change amount data to be applied as computational input to construct a candidate update set and calculate the consistency cost for each candidate update to output the candidate update set. The candidate update set is generated by grouping the effective change amount data by attribute identifier to form a basic update, and then, in continuous input event scenarios, splitting the basic update into multiple candidate updates with different amplitudes according to a preset segmentation rule. The preset segmentation rule is determined by the terminal. The deployment process pre-configures the number of segment levels and the scaling factor for each level to generate multiple update magnitudes without changing the direction of change. The consistency cost is calculated by generating a candidate state summary string after applying the candidate update to the target object's attribute value set, and then performing a difference calculation between the candidate state summary string and the target state verification information according to the corresponding fields. The difference is the absolute difference of numerical fields or the inconsistency flag of enumerated fields. Finally, the consistency cost is obtained by summing the differences of each field according to the field weights. The field weights are pre-configured by the terminal during deployment or generated by the rendering engine based on the importance of the attribute to the visual impact and written to the configuration area for reading. When the target state verification information is missing or cannot be parsed, the local collaborative rendering module sets the consistency cost to the preset maximum value and only retains the candidate update with the smallest magnitude to proceed to the next step to avoid outputting excessive updates when the verification basis is missing. Subsequently, the local collaborative rendering module selects the optimal update from the candidate update set and updates the confidence level to ensure that the update meets the dual hard constraints of writable attribute domains and latency limits: for each candidate update, it is first broken down into several attribute change records, and each attribute change record is checked to see if its attribute identifier belongs to the writable attribute domain set. If any does not belong, the candidate update is determined not to meet the writable attribute domain set constraint and is directly eliminated; for candidate updates that meet the writable attribute domain set constraint, the estimated rendering time is calculated according to the pre-stored rendering time estimation rules, and it is determined whether it does not exceed the local rendering latency limit. The rendering time estimation rules are pre-configured by the terminal during deployment as a mapping table from attribute identifiers to unit rendering time and can be estimated and updated by historical statistics. The estimated rendering time is obtained by summing the unit rendering times corresponding to each attribute change record contained in the candidate update. The local rendering latency limit is pre-configured by the terminal during deployment as a millisecond-level threshold. To ensure the interaction frame rate; only among candidate updates that simultaneously satisfy the above two constraints, the local collaborative rendering module compares the consistency cost and selects the candidate update with the smallest consistency cost value as the optimal update; the local collaborative rendering module calculates the update confidence of the optimal update and performs a fusion update with the historical update confidence of the target object to output the update confidence. The calculation of the update confidence takes the consistency cost as input and converts it into a confidence value according to a preset mapping function. The mapping function is pre-configured by the terminal as a monotonically decreasing function to ensure that the smaller the consistency cost, the higher the confidence. The historical update confidence is maintained by the terminal for each target object in the current rendering state and written back after each update so that the credibility of continuous updates can be accumulated and reflected; when there is no candidate update in the candidate update set that simultaneously satisfies the two constraints, the local collaborative rendering module sets the optimal update to null and directly enters the approximate rollback process to avoid timeout or out-of-bounds writing; Finally, the local collaborative rendering module outputs the updated rendering state based on threshold judgment and an approximate backoff iteration mechanism to ensure process convergence and verifiable output: when the update confidence is not lower than a preset confidence threshold and the consistency cost is not higher than a preset cost threshold, the local collaborative rendering module applies the optimal update to the corresponding writable attribute domain of the target object to update the current rendering state and outputs the updated rendering state. The preset confidence threshold and preset cost threshold are pre-configured as numerical thresholds by the interactive screen terminal during deployment and can be selected according to the collaboration mode or network status as the current value. When either threshold is not met, the local collaborative rendering module truncates or quantizes the effective change data according to a preset approximation rule to generate approximate change data and returns to the candidate update set generation and optimal update selection process. The terminal pre-configures a set of rules for quantization step size rounding and amplitude truncation. The quantization step size is used to round numerical changes by a fixed step size to reduce fine-grained jitter, and the amplitude truncation limit is used to limit the maximum change in a single update to avoid excessive jumps. The stopping condition is pre-configured by the terminal during deployment as either the maximum number of iterations or the minimum change threshold. The maximum number of iterations is used to avoid infinite loops, and the minimum change threshold is used to stop directly and output the current rendering state when the change is small enough not to affect the rendering after approximation. When the stopping condition is met but the threshold judgment is not satisfied, the local collaborative rendering module outputs the current rendering state without applying an update and marks the action package as a delayed consistent update package, which is then handed over to the consistent update process driven by the subsequent confirmation sequence for processing. This ensures that the real-time rendering on the terminal is stable and that the state will not become uncontrollable due to untrusted updates. Through the above steps, the local collaborative rendering module can write the sequence of action packets to be synchronized into the local collaborative buffer and extract effective change data under the constraints of writable attribute domains. Then, it can locate the target object, update the current rendering state, and refresh the display screen to output the local rendering result. At the same time, it can deduplicate and rearrange the action packets in the buffer according to the sending sequence number to maintain order. During the update process, it can select the optimal update by minimizing the consistency cost, the constraints of the writable attribute domain set and the upper limit of the local rendering latency. Then, it can trigger an approximate backoff iteration until the stopping condition is met by the update confidence and cost threshold to ensure that the update is convergent and the latency is controllable, thereby reducing the rendering jitter and state drift risk under weak network and concurrent collaboration. In practical applications: For example, in a multi-user collaborative whiteboard scenario, a terminal user continuously drags an object, generating continuous geometric changes. The presentation module first writes the action package into the buffer and filters out change records that are not in the valid writable attribute domain. Then, it locates the target object using the action target identifier and generates candidate updates with multiple amplitude levels. It calculates the difference between the candidate state summary string after each candidate update is applied and the target state verification information, and accumulates the differences to obtain the consistency cost. At the same time, it estimates the rendering time to ensure that the expected rendering time does not exceed the local presentation latency limit. It selects the optimal update with the minimum consistency cost and satisfies the constraints and calculates the update confidence. When network fluctuations cause the target state verification information to deviate more from the current state, the confidence threshold or cost threshold judgment fails. The module generates an approximate change amount according to the quantization step size and amplitude truncation rules and iteratively selects the change amount. Finally, it maintains the local frame rate with a controllable update amplitude and waits for the subsequent cloud confirmation sequence to arrive before completing the consistent update. This allows the real-time presentation on the terminal side and the consistent presentation between terminals to be established simultaneously.

[0022] The cloud-based adjudication and distribution module is set up on and executed by the cloud collaboration server. It is used to receive the sequence of action packages to be synchronized, locate the target object for each action package based on the action target identifier, limit the writable attribute field according to the action constraint set, and perform validity judgment and conflict handling in combination with the target state verification information to generate the confirmation sequence. The confirmation sequence is then distributed to each interactive screen terminal participating in the same session, so that each interactive screen terminal performs a consistent update of the rendering state based on the confirmation sequence to output a consistent collaborative presentation result between the terminals. The purpose of this embodiment is to enable a cloud-based collaborative server to perform field extraction, target object consistency determination, writable attribute field filtering, and concurrent conflict handling for each action packet under constraints such as concurrent uploads of action packets from multiple interactive screen terminals, network out-of-order and duplicate behavior, and potential local real-time updates to the terminal state. The server then organizes the adjudication results into a confirmation sequence that can be consistently consumed by all interactive screen terminals. The underlying process involves first unpacking the sequence of action packets to be synchronized into a set of adjudication fields for each packet to form a unified adjudication input. Then, a state summary string is constructed based on the session rendering state maintained in the cloud and compared field-by-field with the target state verification information to obtain a consistency determination result. Simultaneously, writable attribute field filtering is performed on the changed data to obtain valid changed data. When concurrent action packets with the same action target identifier appear, the server selects to merge or adjudicate based on attribute intersection. Finally, the adjudicated action packets are grouped by session identifier, sorted in ascending order of sending sequence number, connected to form a confirmation sequence, and sent out. This process integrates terminal concurrency and link uncertainty into a unified, verifiable, and executable adjudication and sending process. The implementation process includes the following steps: To ensure that subsequent consistency checks, attribute domain filtering, and conflict resolution are all performed under the same field definition, the cloud-based adjudication and distribution module receives the sequence of action packets to be synchronized as adjudication input. It then reads each action packet sequentially from its order within the sequence, using the session identifier, action target identifier, action constraint set, change quantity data, target status verification information, and transmission sequence number as the initial adjudication fields. The session identifier assigns the action packet to a unique collaborative session; the action target identifier indicates the target object; the action constraint set includes at least a set of allowed action types and a set of valid writable attribute domains to limit the executable scope; the change quantity data represents the geometric or attribute change quantity to be written; the target status verification information represents the verification summary of the target object when the action packet was generated on the client side; and the transmission sequence number ensures that the action packets are aligned between the cloud and client sides. The cloud-based adjudication and distribution module then writes these fields into each packet. The adjudication records are compiled into a packet-by-packet adjudication field set. Each packet-by-packet adjudication record includes at least a session identifier field, an action target identifier field, an action constraint set field, a change quantity data field, a target status verification information field, and a transmission sequence number field. This allows subsequent adjudication steps to complete all calculations simply by reading the packet-by-packet adjudication field set. When any action packet lacks a session identifier or a transmission sequence number, the cloud adjudication distribution module marks the action packet as invalid and writes the invalidity reason field. Simultaneously, it removes the action packet from the packet-by-packet adjudication field set to prevent subsequent grouping and sorting from becoming unexecutable. When any action packet lacks an action target identifier or the target status verification information is empty, the cloud adjudication distribution module marks the action packet as a weakly verified action packet and retains its session identifier, action target identifier, action constraint set, and change quantity data for subsequent filtering and conflict handling processes. This ensures that limited adjudication can still be performed even when the endpoint cannot provide a complete verification summary. To ensure consistency between cloud-based adjudication and end-to-end verification, and to guarantee the effective implementation of conflict resolution, the cloud-based adjudication distribution module, upon receiving the packet-by-packet adjudication field set, first locates the target object in the cloud-maintained session rendering state based on the action target identifier of each packet-by-packet adjudication record. It then reads the set of attribute values ​​corresponding to the target object's currently writable attribute domains. The session rendering state is maintained by the cloud collaboration server for each session identifier and includes at least an object index table and an object attribute table. The rule for locating the target object is to retrieve a unique object entry from the object index table using the action target identifier and then point to the attribute set in the object attribute table. The cloud-based adjudication distribution module sorts the read attribute value set by attribute identifier, performs unit conversion and precision truncation, and concatenates them into a state summary string. The rules for unit conversion and precision truncation originate from a pre-configured rule set on the cloud side and are consistent with those on the interactive screen terminal side. This includes mappings from attribute identifiers to unit conversion factors and from attribute identifiers to truncation precision parameters. Unit conversion is used to convert attribute values ​​to uniform... One unit, precision truncation, is used to truncate or round attribute values ​​to a specified decimal place or a specified step size. Subsequently, the cloud-based adjudication module compares the state summary string with the target state verification information field by field to obtain the number of difference fields and the set of difference magnitudes, and outputs the consistency judgment result accordingly. The rule for field-by-field comparison is to calculate whether the corresponding fields of the same attribute identifier are equal or to calculate the absolute difference of the values. The number of difference fields is the count of fields with non-zero or unequal differences. The set of difference magnitudes is the set of absolute differences or inconsistency flags corresponding to each difference field. The consistency judgment result is based on whether the number of difference fields is zero or does not exceed the preset maximum allowed number of difference fields. The preset maximum allowed number of difference fields is pre-configured by the cloud as an integer threshold during deployment to distinguish between consistency and inconsistency. When the target object cannot be located or the set of attribute values ​​that can be read is empty, the cloud-based adjudication module marks the consistency judgment result as uncertain and sets the number of difference fields to the preset maximum value to avoid misjudging consistency when there is no state benchmark. After completing the consistency determination, the cloud-based adjudication module performs attribute domain filtering on the change data based on the writable attribute domain limitation information in the action constraint set to output valid change data. The filtering rule is to read each change record in the change data one by one and determine whether its attribute identifier belongs to the valid writable attribute domain set indicated by the writable attribute domain limitation information. If it belongs, the change record is retained; otherwise, it is discarded and added to the discard count. The cloud-based adjudication module merges similar items of the retained change records according to their attribute identifiers to obtain valid change data. The merging rule is to merge multiple change records with the same attribute identifier according to their change type. If it is a geometric change, the values ​​are accumulated to obtain the merged change amount. If it is an attribute change, the record with the larger sending sequence number is overwritten with the record with the smaller sending sequence number to obtain the merged change amount. This ensures that the same attribute identifier corresponds to only one valid change record in one adjudication output. When the valid change data is empty, the cloud-based adjudication module marks the packet-by-packet adjudication record as invalid and retains only its sending sequence number and invalid reason in the subsequent confirmation sequence so that the terminal side can delete the pending confirmation cache but does not perform a status update. When concurrent action packets corresponding to the same action target identifier appear under the same session identifier, the cloud-based adjudication and distribution module performs conflict handling on the concurrent action packets to output an adjudication action packet. The concurrency determination rule is that multiple packet-by-packet adjudication records with the same action target identifier exist within the same session identifier group, and their sending sequence numbers fall into the same concurrency window. The concurrency window is pre-configured by the cloud as a time window or a sequence number window and is used to limit the range of action packets considered concurrent. After determining concurrency, the cloud-based adjudication and distribution module extracts the attribute identifier set contained in the valid change amount data of each concurrent action packet and determines whether there is an intersection between the attribute identifier sets. The intersection determination rule is to perform a set intersection operation on any two sets and check whether the result is empty. When there is no intersection, the cloud-based adjudication and distribution module directly merges the valid change amount data of each concurrent action packet according to the attribute identifier to generate an adjudication action packet. The merge generation rule is to directly append and summarize each valid change record and remove duplicates by attribute identifier to form a merged valid change record. The change amount data is processed, and the session identifier and action target identifier of the adjudication action packet are set to the corresponding session identifier and action target identifier, and its target state verification information is set to the target state verification information carried by the concurrent action packet with the smallest sending sequence number to maintain the consistency of the verification benchmark. When there is an intersection, the cloud adjudication and distribution module simulates the application of its effective change amount data to each concurrent action packet based on the set of attribute values ​​corresponding to the current writable attribute domain to obtain candidate update results. The rule for simulation application is not to write back to the session rendering state, but to copy a set of target object attribute values ​​in memory and perform addition update or assignment update according to the attribute identifier to generate a set of candidate attribute values. The cloud adjudication and distribution module generates a candidate state summary string for each candidate update result and compares it field by field with the target state verification information carried by the concurrent action packet to obtain the number of candidate difference fields and the set of candidate difference magnitudes. The construction rule of the candidate state summary string is the same as that of the aforementioned state summary string and uses the same unit conversion and precision truncation rules. The cloud-based adjudication module selects the concurrent action packet corresponding to the candidate update result with the smallest number of candidate difference fields and the smallest cumulative amplitude of the candidate difference amplitude set as the adjudication action packet. The cumulative amplitude of the candidate difference amplitude set is the cumulative amplitude obtained by summing the numerical differences in the candidate difference amplitude set. If enumerated differences are included, each enumerated difference is included in the cumulative amplitude by a preset unit penalty value. The preset unit penalty value is pre-configured as a constant by the cloud to achieve a unified comparison between numerical differences and enumerated differences. When multiple candidate update results are the same in terms of the number of candidate difference fields and the cumulative amplitude, the concurrent action packet with the smallest sending sequence number is selected as the adjudication action packet to ensure adjudication determinism. When the target state verification information of any action packet in the concurrent action packet is empty, causing the candidate difference to be unable to be calculated, the number of candidate difference fields of that action packet is set to the preset maximum value to prevent it from being selected as the adjudication action packet. To ensure that all interactive screen terminals can consume the cloud-based adjudication results in the same order and use them as consistent update input, the cloud-based adjudication distribution module, after receiving the adjudication action packets, groups the packets according to session identifiers and sorts them in ascending order of sending sequence number to form an acknowledgment sequence. The grouping rule uses the session identifier as the key to assign the adjudication action packets to the corresponding session group; the sorting rule within each session group is to sort them in ascending order of sending sequence number; and the connection rule is to append the sorted results sequentially to form the sequence structure. The cloud-based adjudication distribution module writes the acknowledgment sequence into the distribution buffer and distributes it to all interactive screen terminals participating in the same session via the network communication link. The system outputs an acknowledgment sequence for consistent updates across all interactive screen terminals. The network communication link is a network transmission channel that establishes and maintains a bidirectional communication connection between the cloud collaboration server and the interactive screen terminal. When any interactive screen terminal fails to send an acknowledgment sequence or the acknowledgment sequence times out, the cloud adjudication and sending module marks the terminal as pending retry and resends the acknowledgment sequence according to the preset sending retry cycle. At the same time, the system records the latest sent sequence number upper bound of the terminal in the session rendering state for subsequent incremental sending. The preset sending retry cycle is pre-configured by the cloud as a time threshold during deployment to balance retry overhead and timeliness. Through the above steps, the cloud-based adjudication and distribution module can extract adjudication fields from each packet of the action packet sequence to be synchronized, construct a state summary string based on the session rendering state, compare it field by field with the target state verification information, and output a consistency judgment result. It also filters writable attribute domains based on the action constraint set and merges similar items to form effective change data. In concurrent action packet scenarios, it achieves conflict-free merging through attribute intersection judgment or selects an adjudicator through candidate update simulation and difference minimization criteria. Finally, it groups the adjudication action packets by session identifier, sorts them in ascending order by sending sequence number, and connects them to form a confirmation sequence, which is then distributed to each interactive screen terminal. This provides consistent, ordered, and verifiable adjudication input for end-to-end updates and reduces the risk of state drift and inter-end discrepancies under concurrent collaboration. In practical applications, for example, in collaborative white-screen interactions where multiple people simultaneously operate the same graphical object... In a scenario where two interactive screen terminals send action packets targeting the same action target identifier almost simultaneously under the same session identifier, the cloud-based adjudication and distribution module first performs writable attribute field filtering on the two packets to obtain valid change data. Then, it extracts the attribute identifier sets of the two packets and finds that there is an intersection. Therefore, based on the current attribute value set of the target object in the session rendering state, it simulates applying the change amounts of the two packets and generates their respective candidate state summary strings. It compares each of the target state verification information carried by each packet field to obtain the number of candidate difference fields and the set of candidate difference magnitudes. The action packet with the smallest number of difference fields and the smallest cumulative magnitude is selected as the adjudication action packet and sorted by the sending sequence number before being written into the confirmation sequence for distribution. This ensures that each terminal performs a consistent update to the rendering state after receiving the same confirmation sequence, thus maintaining consistency between terminals under concurrent and weak network conditions.

[0023] Furthermore, in the further explanation of this solution, the input sources corresponding to the touch component, stylus component, and external input access component are not limited to touch input, but can also include voice acquisition devices, vision acquisition devices, and external devices that control and interact with the interactive screen terminal. Among them, the voice command data output by the voice acquisition device, the image or video recognition results output by the vision acquisition device, and the control commands output by the external device can all be connected to the interactive screen terminal by the external input access component. After writing the input source identifier, event type identifier, event parameter set, and timestamp information according to the unified fields, an input event is formed, which then participates in the input event stream generation, action target identifier determination, action constraint set generation, change amount data calculation, action package generation, and cloud adjudication and distribution processing. Therefore, this solution is not only applicable to touch and stylus input scenarios, but also to voice, vision, and inter-device collaborative control scenarios. However, all types of input still follow a unified event-based processing and consistency update mechanism after entering the subsequent processing link. Working Principle: This solution breaks down multi-input collaboration into two closed-loop edge-cloud collaboration mechanisms: edge generation and cloud adjudication. The interactive screen terminal first samples, normalizes, and aligns touch, stylus, and external inputs into an input event stream. The input parsing module determines the action target identifier for each event, generates an action constraint set, calculates change data, and forms an action description triplet. The action package generation module encapsulates the triplet into an action package sequence with a session identifier, target state verification information, and an incrementing sending sequence number. The communication module sends the action package and receives the cloud confirmation sequence, deletes confirmed requests, resends unconfirmed requests, and outputs ordered confirmations according to the confirmation sequence number. The local collaborative presentation module filters writable attribute fields based on the action constraint set, updates the rendering state in real time, and refreshes the display. The cloud adjudication and distribution module performs consistency checks on the cloud session rendering state, filters unauthorized changes, handles concurrent conflicts, generates a confirmation sequence ordered by sending sequence number, and distributes it, enabling each terminal to complete consistent edge-cloud collaborative updates based on the same confirmation result. For example, multiple people operate on the same whiteboard simultaneously: A writes with a pen, B drags the same graphic; both ends first display their operation locally, and then upload the corresponding action package to the cloud; the cloud checks whether the concurrent modifications to the same target are changing different attributes, and merges them if possible, and if there is a conflict, it decides based on the result of "more in line with the target state verification information", and then sends back a unified confirmation sequence; after both ends receive the same confirmation sequence, they align the local display with the cloud decision, and finally everyone sees the same screen.

[0024] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cloud-based collaborative multi-input interactive screen system, comprising an interactive screen terminal, a communication module, an input parsing module, an action package generation module, and a local collaborative presentation module, characterized in that: An interactive screen terminal includes a processor, a memory, a display screen, and a touch component, a stylus component, and an external input access component that are electrically connected to the processor, respectively, for collecting input events from the touch component, the stylus component, and the external input access component and forming an input event stream; The communication module, located on the interactive screen terminal and electrically connected to the processor, is used to establish a two-way communication connection with the cloud collaboration server through a network communication link, so as to send the action packet sequence corresponding to the input event stream and receive the confirmation sequence. The input parsing module is set in the interactive screen terminal and executed by the processor. It is used to parse the input event stream according to the preset parsing rules, determine the action target identifier, action constraint set and change amount data based on each input event, and output an action description triplet. An action package generation module is set on the interactive screen terminal and executed by the processor. It is used to generate action packages based on action description triples. The action package includes a session identifier, an action target identifier, an action constraint set, change quantity data, and target state verification information, and outputs a sequence of action packages to be synchronized. The local collaborative rendering module is set up on the interactive screen terminal and executed by the processor. It is used to write the sequence of action packages to be synchronized into the local collaborative buffer and to perform real-time updates on the rendering state based on the sequence of action packages to be synchronized in order to output the local rendering result.

2. The cloud-based collaborative multi-input interactive screen system according to claim 1, characterized in that: It also includes: a cloud-based adjudication module; The cloud-based adjudication and distribution module, located on and executed by the cloud collaboration server, receives a sequence of action packages to be synchronized. For each action package, it locates the target object based on the action target identifier, limits the writable attribute fields according to the action constraint set, and performs validity judgment and conflict handling in conjunction with the target state verification information to generate the confirmation sequence. The confirmation sequence is then distributed to each interactive screen terminal participating in the same session, enabling each interactive screen terminal to perform a consistent update of its rendering state based on the confirmation sequence to output a consistent collaborative presentation result between the terminals.

3. The cloud-based collaborative multi-input interactive screen system according to claim 2, characterized in that: The cloud-based adjudication and distribution module is used to receive the sequence of action packets to be synchronized as adjudication input, and to extract the session identifier, action target identifier, action constraint set, change quantity data and target status verification information for each action packet to form a set of adjudication fields for each packet. Based on the action target identifier, the target object is located in the session rendering state maintained in the cloud and the set of attribute values ​​corresponding to the current writable attribute domain of the target object is read. The set of attribute values ​​is sorted by attribute identifier, and then the unit conversion and precision truncation are performed and concatenated into a state summary string. Then, the state summary string is compared with the target state verification information field by field to obtain the number of difference fields and the set of difference magnitudes, and the consistency judgment result is output accordingly. Based on the writable attribute domain limitation information in the action constraint set, filter out change records in the change quantity data one by one whose attribute identifier does not belong to the writable attribute domain, and merge similar items of the retained change records according to attribute identifier to obtain effective change quantity data. When concurrent action packets corresponding to the same action target identifier appear under the same session identifier, first extract the attribute identifier set contained in the effective change amount data of each concurrent action packet and determine whether there is an intersection between the attribute identifier sets; when there is no intersection, directly merge the effective change amount data of each concurrent action packet according to the attribute identifier and generate a decision action packet; when there is an intersection, for each concurrent action packet, based on the attribute value set corresponding to the current writable attribute domain, apply the effective change amount data of the concurrent action packet to the corresponding attribute identifier to obtain candidate update results, and generate a candidate state summary string for the candidate update results and compare it field by field with the target state verification information carried by the concurrent action packet to obtain the number of candidate difference fields and the set of candidate difference magnitudes. Select the concurrent action packet corresponding to the candidate update result with the smallest number of candidate difference fields and the smallest cumulative magnitude of the candidate difference magnitude set as the decision action packet; The adjudication action packets are grouped by session identifier, sorted in ascending order of transmission sequence number, and connected to form an acknowledgment sequence. This sequence is then sent via the network communication link to each interactive screen terminal participating in the same session, so as to output an acknowledgment sequence for each interactive screen terminal to perform consistent updates.

4. The multi-input interactive screen system based on cloud collaboration according to claim 3, characterized in that: In the interactive screen terminal, the process of collecting input events from the touch component, the stylus component, and the external input access component and forming an input event stream includes: The processor acquires raw input samples from the touch component, stylus component, and external input access component, and writes an input source identifier, channel identifier, and local incrementing sequence number for each raw input sample to form a normalized input record set. The normalized input record set is then written with a relative timestamp within the source according to the input source identifier, and cross-source timing alignment is performed within a preset alignment window to obtain an aligned input record set. The aligned input record set is then concatenated into an input event stream according to a preset sorting rule and written to a local buffer, thereby outputting the input event stream.

5. A cloud-based collaborative multi-input interactive screen system according to claim 4, characterized in that: The communication module is configured to perform the following steps to send the action packet sequence corresponding to the input event stream and receive the confirmation sequence: Based on the session identifier and the interactive screen terminal identifier, a connection request is initiated to the cloud collaboration server and a session establishment response is received to generate a two-way communication session bound to the session identifier; Each action packet in the sequence of action packets to be sent is written with a sending sequence number and a sending window identifier. The written action packets are sorted according to the sending sequence number and then encapsulated into an uploading data frame to output an uploading data frame sequence. The sequence of data frames to be sent is transmitted to the cloud collaboration server via the network communication link, and a cache of pending confirmations corresponding to the sending sequence number is maintained locally to output the set of sent pending confirmations. Receive the confirmation sequence from the cloud collaboration server and parse it to obtain the confirmation sequence number set. Based on the confirmation sequence number set, delete the confirmed action packets from the sent pending confirmation set and resend the unconfirmed action packets to output the updated action packet sequence sending status. The received confirmation sequence is deduplicated and rearranged according to its confirmation number set and then provided to the interactive screen terminal for subsequent consistency update processing, so as to output an ordered confirmation sequence.

6. The multi-input interactive screen system based on cloud collaboration according to claim 5, characterized in that: The input parsing module includes: receiving an input event stream as parsing input in the interactive screen terminal; reading the input source identifier, event type identifier, event parameter set, and timestamp information for each input event in the input event stream; and outputting a set of event feature fields required for event-by-event parsing accordingly. The input parsing module further includes performing target determination processing on the event feature field set based on preset parsing rules. The preset parsing rules include: mapping the input source identifier and the event type identifier to a target positioning mode, wherein the target positioning mode is limited to an object index pointing mode or a coordinate hit mode; in the object index pointing mode, reading the object index parameter from the event parameter set and determining a unique candidate target accordingly; in the coordinate hit mode, reading the coordinate parameter from the event parameter set and performing a hit detection in the object boundary set of the current rendering state to obtain a candidate target set; when the candidate target set contains multiple candidate targets, selecting the candidate target with the smallest difference between its most recently selected timestamp and the input event timestamp as the unique target and outputting the corresponding action target identifier.

7. A cloud-based collaborative multi-input interactive screen system according to claim 6, characterized in that: The input parsing module further includes generating an action constraint set based on the preset parsing rules. The preset parsing rules further include: retrieving a set of allowed action types from a pre-stored mapping table based on the input source identifier; retrieving a set of writable attribute fields from a pre-stored mapping table based on the event type identifier; reading a set of role permissions corresponding to the input event from the current collaboration state and intersecting it with the set of writable attribute fields to obtain a set of valid writable attribute fields; and encapsulating the set of allowed action types and the set of valid writable attribute fields into an action constraint set and outputting it. The input parsing module further includes generating change amount data based on the preset parsing rules. The preset parsing rules further include: when an input event is determined to be a continuous input event, reading the coordinate parameters of the input event under the same action target identifier and calculating the difference with the coordinate parameters of the previous continuous input event to obtain the geometric change amount; when an input event is determined to be a discrete input event, reading the attribute identifier and attribute value from the event parameter set of the input event to obtain the attribute change amount; performing unified unit conversion and numerical truncation on the geometric change amount and the attribute change amount according to preset dimensions and precision rules to form the change amount data, and outputting an action description triplet composed of the action target identifier, the action constraint set, and the change amount data.

8. A cloud-based collaborative multi-input interactive screen system according to claim 7, characterized in that: The action package generation module is used to receive the action description triple as the generation input, and write the action target identifier, action constraint set and change quantity data into the action package field based on the session identifier to output the action package; Based on the action target identifier, read the state summary corresponding to the target object in the current rendering state of the interactive screen terminal as the target state verification information, and write the target state verification information into the action package to output the action package containing the target state verification information; Action packets generated under the same session identifier are written with incrementing sequence numbers in the order of generation and concatenated into a sequence of action packets to be synchronized, so as to output the sequence of action packets to be synchronized.

9. A cloud-based collaborative multi-input interactive screen system according to claim 8, characterized in that: The local collaborative presentation module is used to receive the sequence of action packets to be synchronized as presentation input, and write the sequence of action packets to be synchronized into the local collaborative buffer in the writing order to output the buffer writing result. Read the action target identifier and change amount data sequentially from the action package sequence to be synchronized, and filter the change amount data by writable attribute domain according to the action constraint set to output valid change amount data; Based on the action target identifier, the target object in the current rendering state is located, and the effective change data is applied to the corresponding writable attribute field of the target object to update the current rendering state, thereby outputting the updated rendering state; The updated rendering state is converted into a display frame on the screen and the screen is driven to refresh to output the local rendering result; The sequence of action packets to be synchronized written to the local cooperative buffer is deduplicated and rearranged according to its sending sequence number to output an ordered sequence of action packets to be synchronized.

10. A cloud-based collaborative multi-input interactive screen system according to claim 9, characterized in that: When the local collaborative rendering module locates the target object in the current rendering state based on the action target identifier and updates the current rendering state, it includes performing the following steps to output the updated rendering state: Step 1: Using the action target identifier as an index, retrieve the target object in the current rendering state and read the target state verification information and writable attribute domain set of the target object. Calculate the consistency cost based on the target state verification information and the effective change amount data to be applied, and generate a candidate update set to output the candidate update set. Step 2: In the candidate update set, the candidate update with the minimum consistency cost and that satisfies the constraints of the writable attribute domain set and the local rendering latency limit is selected as the optimal update. The update confidence of the optimal update is calculated and merged with the historical update confidence of the target object to output the update confidence. Step 3: When the update confidence is not lower than the preset confidence threshold and the consistency cost is not higher than the preset cost threshold, the optimal update is applied to the corresponding writable attribute domain of the target object to update the current rendering state. If the conditions are not met, the effective change data is truncated or quantized according to the preset approximation rules to generate approximate change data, and the candidate update generation and optimal update selection are repeated until the stopping condition is met, thereby outputting the updated rendering state.

Images