A modular hardware-based distributed information kiosk system

Abstract

The application relates to the technical field of digital information transmission, and discloses a distributed information interconnection console system based on modular hardware, which comprises a session initialization module, a streaming media sending module, a streaming media receiving module, a state feedback module and a signaling arbitration module. The session initialization module generates a cross-domain session context and binds an access control token; the streaming media sending module records a request timestamp, locks a control signaling receiving channel and extracts a synchronization reference frame; the streaming media receiving module accesses a multicast tree, buffers video data in a display blanking state and generates a vertical synchronization interrupt; the state feedback module generates a state authentication code, calculates a decoding pipeline delay and outputs an authorized data packet; and the signaling arbitration module verifies a physical rendering output state, calculates a net network delay, calibrates an arbitration time window, completes control signaling recombination and injection. The application can improve the visual control synchronization, concurrent stability and access security under distributed networking.

Description

Technical Field

[0001] This invention relates to the field of digital information transmission technology, specifically to a distributed information interconnection console system based on modular hardware. Background Technology

[0002] The command center relies on the console as the operational terminal for scheduling multi-domain data. To achieve physical separation of humans and machines, the industry often adopts a distributed information interconnection network, centrally deploying the core host in a remote data center, with the front end communicating and collaborating across nodes via Ethernet. In this architecture, the console is mostly built with modular hardware, does not rely on the local computing power of general-purpose computers, and directly sends, receives, and processes multimedia streams and signaling messages carried by Internet Protocol by relying on dedicated digital signal processors and hardware logic resources.

[0003] In existing business interaction scenarios, front-end hardware nodes mainly receive streaming media data from remote hosts by establishing a standard multicast tree structure, and rely on conventional transport layer protocols to package and send back local peripheral control data, thereby supporting operators to complete multi-source access and cross-organization business collaboration in a physically isolated state.

[0004] However, conventional network communication mechanisms do not fully consider the real-time operating status of the underlying hardware. On the one hand, due to network jitter and differences in the decoding buffer cycle within each receiving node, multicast distribution mechanisms that rely solely on network packet arrival time cannot account for the latency of the terminal's internal decoding pipeline. This can lead to screen tearing and inconsistencies between control responses and displayed images when multiple nodes are collaboratively stitched together. On the other hand, in scenarios where multiple seats concurrently send control messages to the same device, existing systems lack a packet arbitration mechanism based on link latency calculations, which can easily cause signaling queuing conflicts and command timing errors. Furthermore, conventional surface routing strategies cannot combine access control tokens, physical rendering status, and status authentication codes for signaling access verification, leaving the Ethernet transmission link vulnerable to unauthorized nodes injecting control commands.

[0005] Therefore, this invention proposes a distributed information interconnection console system based on modular hardware to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a distributed information interconnection console system based on modular hardware, which solves the problems of asynchronous visual and control timing, concurrent signaling conflicts, and unauthorized control caused by multi-node collaboration in a distributed architecture.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a distributed information interconnection console system based on modular hardware, comprising: The session initialization module is used to generate a cross-domain session context in response to cross-domain collaboration requests and extract the access control token bound to the cross-domain session context; The streaming media sending module is used to record the request timestamp, lock the control signaling receiving channel according to the access control token, and extract the first complete keyframe in the sending buffer queue as the synchronization reference frame. The streaming media receiving module is used to access the multicast tree according to the cross-domain session context, buffer streaming media video data to the receiving buffer in the display blanking state, and generate a vertical synchronization interrupt based on the synchronization reference frame. The status feedback module is used to generate a status authentication code in conjunction with the access control token when the vertical synchronization interrupt is triggered, calculate the decoding pipeline delay using the number of queued frames in the receive buffer, and output an authorized data packet containing the rendering sequence number, the status authentication code and the decoding pipeline delay. The signaling arbitration module is used to receive the authorized data packet and record the local synchronization timestamp, verify the physical rendering output status of the target decoding node according to the status authentication code and the rendering sequence number, calculate the net network latency by combining the request timestamp, the local synchronization timestamp and the decoding pipeline latency, and complete the control signaling reassembly and injection based on the net network latency to calibrate the arbitration time window.

[0008] Preferably, in the session initialization module, the steps of generating a cross-domain session context in response to a cross-domain collaboration request and extracting the access control token bound to the cross-domain session context include: Read the local hardware clock timestamp and hardware device identification code, concatenate them, and perform a hash operation to generate a session identifier; Extract the target operating device information from the cross-domain collaboration request, and retrieve the source end encoding node identifier and the source end encoding node multicast address from the local static routing table based on the target operating device information; Generate a random byte sequence as the access control token; It also includes: using a node shared key to perform symmetric encryption on a data field containing the access control token and the session identifier to obtain an encrypted encapsulated data field, and establishing a memory mapping association between the encrypted encapsulated data field and the cross-domain session context.

[0009] Preferably, in the streaming media sending module, the step of recording the request timestamp and locking the control signaling receiving channel according to the access control token includes: Read the nanosecond-level time count value of the real-time clock chip as the request timestamp, and store the request timestamp in the status identifier area; Write the access control token as a feature matching field into the access control list; The system receives control data packets, extracts identity credential data from the network transmission protocol header extension field of the control data packets, compares the identity credential data with the access control list, and triggers a data discard command to lock the control signaling receiving channel when the comparison matches.

[0010] Preferably, in the streaming media sending module, the step of extracting the first complete keyframe in the sending buffer queue as the synchronization reference frame includes: Read the media transmission timestamp of the multimedia frame data in the transmission buffer queue; The value of the media transmission timestamp is compared with the value of the request timestamp, and the first multimedia frame data with the media transmission timestamp not earlier than the request timestamp and the frame encoding type is an intra-frame coded frame is located as the synchronization reference frame. It also includes: extracting the start sequence number, end sequence number, and media transmission timestamp of the synchronization reference frame, encapsulating them to generate a multicast notification message, and sending the multicast notification message to the multicast address of the source encoding node.

[0011] Preferably, in the streaming media receiving module, the step of buffering streaming media video data to the receiving buffer in the display blanking state, according to the cross-domain session context access multicast tree, includes: Extract the source encoding node multicast address from the cross-domain session context, send an Internet Group Management Protocol (IGM) join message to establish a multicast mapping, and push the received streaming video data into the receiving buffer. A register control command is sent to the display controller to cut off the video signal transmission between the main video memory and the physical output interface and initiate the display blanking state; It also includes: addressing the first video frame data whose time sequence is no earlier than the synchronization reference frame time attribute in the receiving buffer, extracting the media transmission timestamp of the first video frame data, resetting the internal time counter of the digital signal processor, and establishing a hardware decoding clock reference.

[0012] Preferably, in the streaming media receiving module, the step of generating a vertical synchronization interrupt based on the synchronization reference frame includes: After verifying that all fragmented data packets of the synchronization reference frame are completely residing in the receiving buffer, the digital signal processor is called to perform decoding operations on the synchronization reference frame to flip the pixel matrix data and write it into the main video memory. When the display controller scans and outputs the pixel matrix data in the main video memory, triggering a vertical synchronization signal level transition, the vertical synchronization interrupt is captured and generated by the interrupt controller. It also includes: issuing a state reset command to remove the display blanking state after the vertical synchronization interruption is triggered, and not sending control data packets carrying the access control token to the source encoding node when the control signaling injection is in an unauthorized state.

[0013] Preferably, in the status feedback module, the step of generating a status authentication code in conjunction with the access control token when the vertical synchronization interrupt is triggered includes: When the vertical synchronization interruption is captured, the physical address of the video frame in physical rendering state is obtained, and the real-time transmission protocol start number is queried based on the physical address of the video frame using the frame attribute mapping table as the rendering sequence number. The rendering sequence number, the session identifier in the cross-domain session context, and the access control token are respectively converted into binary fields and bit-width consistency processing is performed. Perform a binary XOR operation on the rendering sequence number, the session identifier, and the access control token after the bit width is consistent, and input the XOR operation result into a secure hash algorithm to generate a one-way hash digest as the status authentication code.

[0014] Preferably, in the status feedback module, the step of calculating the decoding pipeline delay using the number of queued frames in the receive buffer includes: The write pointer number and read pointer number of the receive buffer are read, and the difference between the write pointer number and the read pointer number is calculated to determine the number of queued frames; When the write pointer number is less than the read pointer number, the write pointer number is added to the frame descriptor queue capacity and then subtracted from the read pointer number to obtain the number of queued frames. Extract the chip hardware base frame period configured according to the display refresh rate; The decoding pipeline delay is obtained by multiplying the number of queued frames with the chip hardware base frame period input hardware multiplier.

[0015] Preferably, in the signaling arbitration module, the step of calculating the net network latency by combining the request timestamp, the local synchronization timestamp, and the decoding pipeline latency includes: The total round-trip time is obtained by subtracting the absolute time count of the requested timestamp from the absolute time count of the local synchronization timestamp. Subtract the decoding pipeline delay from the total round-trip delay to obtain the delay subtraction result; The result of subtracting the delay from the zero value is compared with the maximum value, and the maximum value of the comparison output is determined as the net network delay.

[0016] Preferably, in the signaling arbitration module, the step of completing control signaling reassembly and injection based on the net network delay to calibrate the arbitration time window includes: Obtain the network jitter compensation coefficient corresponding to the port negotiation rate matching of the network interface, multiply the network jitter compensation coefficient by half of the net network delay to obtain the product result, and superimpose the product result onto the preset polling reference time window constant to obtain the arbitration time window; Within the polling interval of each arbitration time window, receive multi-node control data packets sent by nodes holding valid access control tokens within the authorized control node identifier set, extract the signaling timestamps embedded in the multi-node control data packets, and sort the control signaling according to their numerical values. For absolute coordinate control signaling, the control signaling with the largest timestamp value is retained. For paired event control signaling, the control signaling is retained in sequence according to the original event occurrence number. The retained control signaling is converted into data and injected into the source device using the Universal Serial Bus human-machine interface device protocol.

[0017] This invention provides a distributed information interconnection console system based on modular hardware. It has the following advantages: 1. This invention extracts keyframes as a synchronization benchmark and calculates the decoding pipeline latency by combining vertical synchronization interruptions and the number of queued frames, thus deducting the latency impact caused by the internal buffer of the target decoding node. In a network environment deploying modular hardware, the system utilizes a hardware decoding mechanism to achieve time benchmark alignment of independently transmitted streaming media data on the terminal side, thereby alleviating the screen tearing problem of the information interconnection console in distributed multi-node collaborative retrieval scenarios. In a communication architecture where core nodes are centrally located to meet the requirements of human-machine separation, this method can improve the stability of long-distance transmission and display output of audio and video multicast.

[0018] 2. This invention verifies the physical rendering status of the terminal and calculates the net network latency. Based on the net network latency, an arbitration time window is calibrated to complete the deduplication, sorting, and reassembly of concurrent control signaling. For modular hardware network layouts with boundary isolation, this processing method can reduce signaling queuing conflicts and command timing errors caused by multiple nodes issuing interactive commands concurrently. It supports distributed routing nodes in collaborating with the front-end information interconnection console to perform cross-domain signal capture and push, and maintains the continuity of concurrent data calls while simplifying the layout of the terminal lines.

[0019] 3. This invention constructs a session context in response to cross-domain collaboration requests and generates an access control token. The access control token is dynamically written into the access control list as a feature matching field to lock the control signaling reception channel. Simultaneously, it verifies the physical rendering output status of the target decoding node using a status authentication code. This method can establish a controlled multicast mapping relationship across physical network segments and block the forwarding of unauthorized control messages to the source device in the modular hardware transmission link. When multi-level users perform resource scheduling within a distributed network topology, this system can provide the front-end information interconnection console with a control and admission mechanism based on access control tokens and status authentication codes, thereby forming a unified communication link with controlled permissions. Attached Figure Description

[0020] Figure 1 This is a system architecture diagram of the distributed information interconnection console based on modular hardware of the present invention; Figure 2 This is a flowchart of the distributed information interconnection console method based on modular hardware of the present invention; Figure 3 This is a schematic diagram of session initialization and token binding according to the present invention; Figure 4 This is a schematic diagram of signaling interception and reference frame extraction according to the present invention; Figure 5 This is a schematic diagram of the media stream buffer and hardware decoding of the present invention; Figure 6 This is a schematic diagram of the status authentication code and delay calculation of the present invention; Figure 7 This is a schematic diagram of the multi-node signaling arbitration and injection of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 This invention provides a distributed information interconnection console system based on modular hardware, which is deployed between the console terminal and the source device.

[0023] The hardware operating environment includes encoding nodes, decoding nodes, and network switching equipment. Source devices are connected to the encoding nodes via video cables and a universal serial bus; the console terminal's display devices and peripherals are connected to the decoding nodes; and the encoding and decoding nodes transmit network data packets through the network switching equipment.

[0024] The encoding and decoding nodes are equipped with field-programmable gate arrays and digital signal processors to perform hardware-level data processing.

[0025] The digital signal processor is specifically configured in the underlying hardware architecture with a very long instruction word (VLSI) computing core for parallel execution of multimedia instructions, a direct memory access controller connected to the VLSI computing core via an internal bus, a multiply-accumulate array for performing frame pixel matrix multiply-accumulate operations, and an on-chip high-speed static random access memory matrix for mapping the physical addresses of the video buffer queue.

[0026] Field-programmable gate arrays (FPGAs) are physically configured with lookup table resources, hardware triggers, and programmable internal wiring matrices for implementing cryptographic algorithms.

[0027] The distributed information interconnection console system based on modular hardware includes the following modules.

[0028] The session initialization module is used to respond to cross-domain collaboration requests, generate a cross-domain session context containing a session identifier, and extract the access control token bound to the cross-domain session context.

[0029] The streaming media sending module is used to record request timestamps, lock the control signaling receiving channel according to the access control token, and extract the first complete keyframe in the sending buffer queue as the synchronization reference frame.

[0030] The streaming media receiving module is used to access the multicast tree according to the cross-domain session context, buffer streaming media video data to the receiving buffer in the display blanking state, and perform hardware decoding to generate a vertical synchronization interrupt based on the synchronization reference frame.

[0031] The status feedback module is used to generate a status authentication code in conjunction with the access control token when the vertical synchronization interrupt is triggered, calculate the decoding pipeline latency using the number of queued frames in the receive buffer, and output an authorized data packet containing the rendering sequence number, status authentication code, and decoding pipeline latency.

[0032] The signaling arbitration module records the local synchronization timestamp of the arrival time of the authorized data packet, verifies the physical rendering output status of the target decoding node based on the status authentication code and rendering sequence number, calculates the net network latency by combining the request timestamp, local synchronization timestamp and decoding pipeline latency, and dynamically calibrates the arbitration time window based on the net network latency, so as to complete the reassembly and injection of control signaling of multiple nodes within the arbitration time window.

[0033] Reference Figure 2 The present invention also provides a method for a distributed information interconnection console based on modular hardware. This method is executed by the aforementioned distributed information interconnection console system based on modular hardware, and specifically includes the following steps: S1 responds to cross-domain collaboration requests, generates a cross-domain session context containing a session identifier, and extracts the access control token bound to the cross-domain session context; S2, record the request timestamp, lock the control signaling receiving channel according to the access control token, and extract the first complete keyframe in the send buffer queue as the synchronization reference frame; S3 accesses the multicast tree based on the cross-domain session context, buffers streaming video data to the receiving buffer in the display blanking state, and performs hardware decoding based on the synchronization reference frame to generate a vertical synchronization interrupt. S4 generates a status authentication code by combining the access control token when the vertical synchronization interrupt is triggered, calculates the decoding pipeline latency using the number of queued frames in the receive buffer, and outputs an authorized data packet containing the rendering sequence number, status authentication code and decoding pipeline latency. S5 records the local synchronization timestamp of the arrival time of the authorized data packet, verifies the physical rendering output status of the target decoding node based on the status authentication code and rendering sequence number, calculates the net network latency by combining the request timestamp, local synchronization timestamp and decoding pipeline latency, and dynamically calibrates the arbitration time window based on the net network latency, so as to complete the reassembly and injection of control signaling of multiple nodes within the arbitration time window.

[0034] To illustrate how each functional module performs the aforementioned steps, the internal processing flow of each functional module is described below.

[0035] Reference Figure 3 In this embodiment, the session initialization module performs step S1, which includes the following steps: S101 responds to cross-domain collaboration requests.

[0036] The session initialization module continuously listens to the service port of the control plane through the internal network communication interface, such as port 8080 of the Transmission Control Protocol (TCP).

[0037] Upon receiving a network packet from the console terminal, the session initialization module unpacks and parses the packet according to the Transmission Control Protocol (TCP), reading the packet payload data segment. This unpacking and parsing can be performed using existing TCP packet parsing mechanisms.

[0038] Optionally, the message payload data segment adopts a JavaScript object representation data format. The session initialization module retrieves key-value pairs from the message payload data segment, extracts the data values ​​corresponding to the operator identifier field and the target device identifier field as keys, obtains the operator identity information that triggered this operation and the target operating device information to be taken over, and confirms the response to the cross-domain collaboration request accordingly.

[0039] Cross-domain collaborative requests refer to resource takeover signaling initiated by a control node in one physical region to a host node in another physical region in a distributed network environment.

[0040] S102, Generate a cross-domain session context containing a session identifier.

[0041] The session initialization module allocates a data storage area in local memory. To ensure the uniqueness of the session state, the session initialization module reads the local hardware clock timestamp of the current clock chip and the hardware device identification code of the motherboard, concatenates the local hardware clock timestamp and the hardware device identification code, performs a hash operation, and generates a session identifier.

[0042] Based on the target operating device information extracted in step S101, the session initialization module uses this target operating device information as a query keyword to retrieve the local static routing table. Optionally, this static routing table is configured and distributed by the administrator through the background management interface during the device network initialization phase and is permanently stored in the non-volatile memory of each node. This static routing table records the static mapping relationship between the media access control addresses, node logical identifiers, and multicast protocol addresses of all network access devices.

[0043] By retrieving the static routing table, the session initialization module matches and obtains the corresponding source encoding node identifier and the source encoding node multicast address.

[0044] Based on the operator identity information obtained above, the session initialization module matches and retrieves the corresponding target decoding node identifier in the user equipment mapping table. Optionally, the user equipment mapping table is statically constructed by the system administrator during the deployment phase at the command center console, by binding the operator identity identifier to the device logical identification code of the physical workstation decoding node one-to-one through the identity authentication management backend, and sending the binding relationship data to the local non-volatile memory of the session initialization module.

[0045] When the cross-domain collaboration request carries the collaboration control node identifier, the session initialization module will also write the target decoding node identifier and the collaboration control node identifier into the authorized control node identifier set; when the cross-domain collaboration request does not carry the collaboration control node identifier, the authorized control node identifier set only includes the target decoding node identifier.

[0046] After obtaining the above basic parameters, the session initialization module encapsulates and splices the session identifier, source encoding node identifier, source encoding node multicast address, target decoding node identifier, and authorization control node identifier set in the allocated data storage area according to a structured protocol format, such as JavaScript object notation or Extensible Markup Language (XML) format, and constructs a cross-domain session context in memory.

[0047] Cross-domain session context is structured data that maintains the full lifecycle state of a single cross-domain collaborative operation in memory, used to indicate the network routing path for multiplexing control commands and audio / video streaming media data.

[0048] S103, extract the access control token bound to the cross-domain session context, and bind the encrypted encapsulated data field containing the access control token to the cross-domain session context and output it.

[0049] Optionally, the session initialization module triggers an internal hardware random number generator to generate a random byte sequence of 128 to 512 bits in length. The length of the random byte sequence can be 128, 192, 256, 384, or 512 bits, and this random byte sequence is used as an access control token. The access control token is used to verify the terminal device's control signaling injection permissions in a distributed network state.

[0050] To ensure secure signaling interaction, the session initialization module invokes a symmetric encryption algorithm from existing technologies. It uses the shared node key—written during device initialization to the identifiers of the source encoding node, target decoding node, and authorized control node—as the encryption key to encrypt and encapsulate data fields containing access control tokens and session identifiers. The shared node key can be 128, 192, or 256 bits long. The data encryption and encapsulation process of the symmetric encryption algorithm can be configured according to existing cryptographic standards.

[0051] After encryption and encapsulation, the session initialization module obtains the encrypted encapsulated data field carrying the access control token, and binds the encrypted encapsulated data field to the cross-domain session context and outputs it.

[0052] After obtaining the encrypted encapsulated data field carrying the access control token, the session initialization module creates a new dedicated data field in the data structure of the already constructed cross-domain session context, writes the encrypted encapsulated data field into the dedicated data field, and establishes a mapping association between the encrypted encapsulated data field and the memory address of the cross-domain session context.

[0053] After the mapping association is completed, the session initialization module resolves the corresponding destination Internet Protocol address based on the source encoding node identifier, the target decoding node identifier, and the cooperating control node identifier from the set of authorized control node identifiers recorded in the cross-domain session context.

[0054] Based on the destination Internet Protocol address, the session initialization module will send a data packet carrying the cross-domain session context and encrypted encapsulated data field to the corresponding source encoding node, target decoding node, and cooperative control node in the authorized control node identifier set (excluding the target decoding node) through the underlying network interface card. When the authorized control node identifier set only includes the target decoding node, the data packet will only be sent to the source encoding node and the target decoding node.

[0055] Reference Figure 4 In this embodiment, the streaming media sending module performs step S2, which includes the following steps: S201 uses the local hardware clock to record the request timestamp at the moment the cross-domain collaboration request is received.

[0056] The streaming media transmission module is configured with an underlying Ethernet communication interface for receiving network packets sent by external nodes. After the underlying driver of the Ethernet communication interface captures a data packet carrying a cross-domain session context and an encrypted encapsulated data field, the streaming media transmission module uses the node's shared key to decapsulate the encrypted encapsulated data field, obtains an access control token, and then issues a hardware read command to the local motherboard controller.

[0057] Optionally, the streaming media sending module can call the real-time clock chip corresponding to the local hardware clock through a general input / output interface to read the nanosecond-level time count value of the current moment in the internal register of the real-time clock chip.

[0058] After obtaining the nanosecond-level time count value of the current moment, the streaming media sending module defines this nanosecond-level time count value as a request timestamp and stores the request timestamp in the status identifier area in memory. The request timestamp is used to mark the absolute starting point of this cross-domain resource takeover operation on the physical timeline.

[0059] S202, at the local application layer, refuse to receive control packets carrying access control tokens to lock the control signaling reception channel.

[0060] The control signaling receiving channel refers to the logical data communication link built inside the streaming media sending module. It is used to receive external network control messages, parse the external network control messages into general serial bus instructions, and then inject them into the source device.

[0061] To prevent operators from issuing erroneous commands before the audio and video streaming media data has been physically aligned, the streaming media sending module establishes data interception rules at the application layer of the network protocol stack.

[0062] The streaming media sending module extracts the access control token obtained from the aforementioned decapsulation and dynamically writes it into the access control list of the local firewall component as a feature matching field. The configuration of the access control list at the application layer of the network protocol stack can refer to existing network security access standards.

[0063] In network receiving operation mode, when the underlying network card receives a control data packet containing Universal Serial Bus control data, the streaming media sending module decapsulates the network transmission protocol header of the control data packet and extracts the identity credential data carried in the extended field of the network transmission protocol header.

[0064] The streaming media sending module compares the extracted identity credential data with the feature matching fields in the access control list, bit by bit. If the identity credential data matches the access control token, the streaming media sending module determines that the control data packet belongs to the cross-domain collaborative session to be established.

[0065] Based on this determination, the streaming media transmission module triggers a data discard instruction at the application layer, performing a rejection operation on the control data packets. Through this data discard strategy, control messages can no longer be converted into Universal Serial Bus (USB) instructions, and the control signaling receiving channel enters a locked state, thereby blocking remote input instructions from controlling the source device.

[0066] While executing the packet interception logic, the streaming media sending module maintains the normal network packet release and distribution mechanism. If the external data packet received by the underlying network card does not carry an access control token, such as a keep-alive probe packet used to maintain link connection or a background operation and maintenance management data packet, the streaming media sending module determines that the external data packet does not belong to the processing object of this cross-domain collaborative control method.

[0067] For regular network packets that do not carry access control tokens, the streaming media sending module, based on existing Internet Protocol delivery rules, directly allows them to pass and transfers them to the corresponding upper-layer processing process for parsing.

[0068] S203, extract the first complete keyframe in the send buffer queue whose time sequence is no earlier than the requested timestamp as the synchronization reference frame, and send a multicast notification message containing the sequence number range of the synchronization reference frame's time attribute.

[0069] The streaming media transmission module internally maintains a transmission buffer queue built on a first-in, first-out (FIFO) queue structure. This transmission buffer queue is used to temporarily store multimedia frame data that has completed video hardware encoding and is awaiting network transmission, in chronological order.

[0070] The streaming media sending module sequentially traverses each multimedia frame data in the queued state of the sending buffer. Optionally, the streaming media sending module reads the media transmission timestamp written during the encapsulation encoding of each multimedia frame data. The generation and encapsulation specifications of intra-frame encoded frames and real-time transmission protocol timestamps during video encoding can be configured with reference to existing audio and video digital encoding communication standards.

[0071] The media transmission timestamp is generated by the local hardware clock of the source encoding node, or it is calculated from the real-time transmission protocol timestamp according to the corresponding media clock frequency, so that the media transmission timestamp and the request timestamp recorded in step S201 are on the same time base.

[0072] The streaming media sending module compares the read media transmission timestamp value with the request timestamp value recorded in step S201.

[0073] Through the above size comparison operation, the streaming media sending module addresses and locates the first multimedia frame data whose media transmission timestamp value is greater than or equal to the requested timestamp value and whose frame encoding type is intra-coded. Intra-coded frames are complete keyframes that can achieve independent image rendering without relying on other reference frames.

[0074] The streaming media sending module extracts the first multimedia frame data it addresses and marks it as a synchronization reference frame in memory. The synchronization reference frame serves as the time reference point for timing alignment of the multimedia data stream, and is used to instruct the target decoding node to establish the rendering clock of the physical picture starting from this multimedia frame data.

[0075] For earlier multimedia frame data whose media transmission timestamp value is less than the requested timestamp value during the traversal and comparison process, the streaming media sending module does not perform interception and marking operations, but instead pushes the earlier multimedia frame data to the underlying network sending interface for normal forwarding according to the conventional transmission protocol.

[0076] After marking the synchronization reference frame, the streaming media sending module extracts the start sequence number, end sequence number, and media transmission timestamp from the real-time transmission protocol header of the synchronization reference frame.

[0077] The starting sequence number is the real-time transmission protocol sequence number of the first fragment data packet of the synchronization reference frame, and the ending sequence number is the real-time transmission protocol sequence number of the last fragment data packet of the synchronization reference frame. The sequence number range of the synchronization reference frame is a closed interval consisting of the starting sequence number and the ending sequence number.

[0078] The streaming media sending module uses the extracted start sequence number, end sequence number, and media transmission timestamp as payload data, and encapsulates them to generate a multicast notification message in User Datagram Protocol (UDP) format.

[0079] After generating the multicast notification message, the streaming media sending module sends the multicast notification message to the multicast address of the source encoding node in the local area network, and repeatedly sends the multicast notification message according to a preset notification period until a preset number of retransmissions or a preset retransmission duration is reached; preferably, the preset notification period is 10 milliseconds, with an allowable configuration range of 5 milliseconds to 50 milliseconds; the preset number of retransmissions is 5 times, with an allowable configuration range of 3 times to 20 times; the preset retransmission duration is 200 milliseconds, with an allowable configuration range of 50 milliseconds to 500 milliseconds, in order to notify the target decoding node in the network environment to obtain the reference identifier for timing alignment.

[0080] Reference Figure 5 In this embodiment, the streaming media receiving module performs step S3, which includes the following steps: S301, send a network multicast join message to access the source-end encoding node multicast tree associated with the cross-domain session context, and push streaming video data into the receive buffer.

[0081] The streaming media receiving module parses the cross-domain session context and reads the source encoding node multicast address recorded in the cross-domain session context.

[0082] Based on the acquired source-end encoding node multicast address, the streaming media receiving module sends an Internet Group Management Protocol (IGMP) join message to the network switching equipment within the local area network via the underlying network interface. The sending of the IMP join message and the establishment of group membership can be configured according to existing multicast routing protocol standards.

[0083] After receiving an Internet Group Management Protocol (IGMP) join message, the network switching device establishes a multicast mapping in its internal routing table. Optionally, the internal routing table is constructed by the network switching device based on the IMP listening mechanism; the network switching device records the mapping relationship between physical switching ports and multicast group addresses by parsing the group membership report messages sent by each node in the local area network.

[0084] After connecting to the multicast tree of the source encoding nodes, the streaming media receiving module begins receiving streaming video data distributed by the source encoding nodes. To mitigate network transmission jitter, the streaming media receiving module allocates a circular buffer space in its local static random access memory, defining this circular buffer space as the receive buffer area.

[0085] The streaming media receiving module extracts the streaming media video data received by the underlying network card and pushes the streaming media video data into the receiving buffer in sequence according to the network arrival time for queuing and temporary storage.

[0086] S302, activates the display blanking state to prevent the physical rendering output of unsynchronized images, and establishes a local hardware decoding clock reference by addressing video frame data whose time sequence is no earlier than the synchronization reference frame in the receive buffer.

[0087] In the initial stage of data buffering, the receive buffer may temporarily store historical video data generated before the cross-domain collaborative request took effect. To avoid operator misjudgment caused by the historical video data being refreshed on the physical screen, the streaming media receiving module sends a register control command to the local display controller to cut off the effective video signal transmission between the main video memory and the physical output interface, and initiate the display blanking state.

[0088] In the blanking state, the physical display remains black or the image is frozen, thereby preventing the physical rendering output of unsynchronized images.

[0089] While maintaining the blanking state, the streaming media receiving module performs network transmission protocol frame header parsing on the streaming media video data in the receiving buffer. The streaming media receiving module extracts the sequence number range carried in the multicast notification message in step S203 and compares the sequence number range with the real-time transmission protocol sequence number of each streaming media video data in the receiving buffer.

[0090] During the address matching process, when the time sequence of a video frame in the receiving buffer is detected to be earlier than the time attribute of the synchronization reference frame, the streaming media receiving module determines that the video frame is an invalid old frame and directly performs the removal and discard operation on the invalid old frame in the receiving buffer.

[0091] When the streaming media receiving module locates the first video frame whose time sequence is equal to or later than the synchronization reference frame's time attribute based on the numerical comparison results, it extracts the media transmission timestamp of that first video frame. The streaming media receiving module then uses this extracted timestamp to reset the internal time counter of its local digital signal processor, thus establishing the local hardware decoding clock reference.

[0092] S303: After confirming the existence of complete video frame data corresponding to the synchronization reference frame, hardware decoding is performed to generate a vertical synchronization interrupt and the display blanking state is lifted, while the control signaling injection remains in an unauthorized state.

[0093] The streaming media receiving module continuously monitors the fragmentation and aggregation integrity of each data packet in the receiving buffer through a frame boundary verification mechanism. Before confirming the existence of a complete video frame data corresponding to the synchronization reference frame, the streaming media receiving module continues to send control commands to the local display controller to maintain the display blanking state.

[0094] Once the streaming media receiving module verifies that all fragmented data packets of the synchronization reference frame have been completely stored in the receiving buffer, the streaming media receiving module calls the local digital signal processor to load the complete synchronization reference frame data and perform hardware decoding operations.

[0095] After the local digital signal processor completes the data decoding operation, it generates the restored uncompressed pixel matrix data and writes the uncompressed pixel matrix data into the main video memory of the local display controller.

[0096] When the local display controller completely scans and outputs a frame of uncompressed pixel matrix data from the main video memory to the external physical display panel, a level transition signal is generated at the vertical sync signal output terminal of the local display controller. The streaming media receiving module captures this level transition signal through the underlying interrupt controller and generates a vertical sync interrupt.

[0097] Vertical synchronization interrupt refers to a low-level hardware-level interrupt signal triggered by the display control hardware when it completes the scanning and refresh of a full physical frame. It is used to indicate that the digital video frame data has been physically displayed and output.

[0098] After the vertical synchronization interrupt is triggered, the streaming media receiving module sends a status reset command to the local display controller to release the display blanking state and restore the normal screen output of the external physical display.

[0099] Although the external physical display has already shown the aligned real-time image, the control data packet carrying the access control token is still subject to the locking constraint of the control signaling receiving channel before the source encoding node completes the network status verification. It cannot be converted into a universal serial bus instruction and injected into the source device.

[0100] The streaming media receiving module can collect peripheral input data generated by connected peripheral input devices, but in the unauthorized state of control signaling injection, it does not send control data packets carrying access control tokens to the source encoding node, nor does it change the lock state of the control signaling receiving channel of the source encoding node, thereby achieving time-series isolation between visual alignment and logical authorization.

[0101] Reference Figure 6 In this embodiment, the status feedback module executes step S4, which includes the following steps: S401 extracts the rendering sequence number corresponding to the physically rendered video frame when the vertical synchronization interrupt is triggered.

[0102] During the process of the streaming media receiving module calling the local digital signal processor to write the decoded frame data into the main display memory, once the local display controller completes the scanning and output of one frame, the vertical synchronization signal output terminal triggers a vertical synchronization interrupt.

[0103] The status feedback module is configured with an interrupt response routine. When a vertical synchronization interrupt is detected, the status feedback module is activated and executes the interrupt response routine.

[0104] Optionally, the status feedback module sends a read command to the register bus of the local display controller to obtain the physical address of the video frame currently in physical rendering state.

[0105] Based on the physical address of the video memory, the status feedback module retrieves the locally maintained frame attribute mapping table. Optionally, the frame attribute mapping table is constructed by the streaming media receiving module during the process of pushing streaming media video data into the receiving buffer and subsequently flipping it to write to the main video memory; the frame attribute mapping table records the association between the physical base address of each frame data and the start sequence number of the real-time transmission protocol in the header of the corresponding network data packet.

[0106] The status feedback module uses the frame attribute mapping table to query the real-time transmission protocol start number corresponding to the physically rendered video frame in the initial network reception phase.

[0107] The status feedback module defines the extracted real-time transmission protocol start number as the rendering sequence number. The rendering sequence number is used to uniquely identify the image frame currently being presented by the physical display panel at the data stream level.

[0108] S402 performs an XOR hash operation on the rendering sequence number, the session identifier in the cross-domain session context, and the access control token to generate a status authentication code.

[0109] After obtaining the rendering sequence number, the status feedback module retrieves the previously constructed cross-domain session context from the local memory area, extracts the session identifier and associated encrypted encapsulated data fields from the cross-domain session context, and decapsulates the encrypted encapsulated data fields using the node shared key to obtain the access control token.

[0110] To ensure the security of cross-domain signaling interaction and prevent data replay attacks, the status feedback module calls the local cryptographic operation configuration to perform combined operations on the obtained rendering sequence number, session identifier, and access control token.

[0111] Optionally, the status feedback module employs a secure hash algorithm, such as SHA-256, to perform an XOR hash operation. The specific XOR hash operation logic follows the formula below: ; In the formula, This indicates the generated status authentication code; This represents the one-way hash function corresponding to an existing secure hash algorithm; Indicates the rendering serial number; Indicates the session identifier; Indicates an access control token; Represents the binary XOR operator. The status authentication code integrates node network identity, cross-domain collaborative session state, and physical display progress data, and is used to verify the true physical rendering alignment state of the target decoding node to the source encoding node.

[0112] The status authentication code is calculated by combining the rendering sequence number, session identifier, and access control token, and is used to prove to the source encoding node the physical rendering alignment status of the target decoding node.

[0113] Optionally, before performing the XOR operation, the status feedback module converts the rendering sequence number, session identifier, and access control token into binary fields respectively; wherein, the rendering sequence number is represented in 64-bit unsigned integer format, the session identifier is represented in 256-bit binary digest format, and the access control token is represented in its original length when it was generated, with the original length being 128 to 512 bits.

[0114] When the lengths of the binary fields involved in the XOR operation are inconsistent, the status feedback module performs bit-width standardization on each binary field before performing the XOR operation. Specifically, the rendering sequence number is converted to 256 bits by padding with zeros at the high-order bits, the session identifier remains at 256 bits, and the access control token is converted to 256 bits when its length is less than 256 bits, retains its original length when its length is equal to 256 bits, and is converted to 256 bits through SHA-256 digest operation when its length is greater than 256 bits. The status feedback module uses the bit-width-standardized rendering sequence number, session identifier, and access control token as inputs for the XOR operation.

[0115] S403 reads the number of queued frames in the receiving buffer that are in a waiting state, multiplies the number of queued frames by the preset chip hardware base frame period to calculate the decoding pipeline delay, and outputs an authorized data packet containing the status authentication code, rendering sequence number and decoding pipeline delay.

[0116] Within the same processing cycle of generating the status authentication code, the status feedback module reads the status of the receive buffer queue maintained by the streaming media receiving module. Optionally, the status feedback module reads the write pointer and read pointer of the underlying queue of the receive buffer.

[0117] The receive buffer includes a data buffer and a frame descriptor queue. Each frame descriptor in the queue corresponds to a complete video frame that has been fragmented and aggregated and is awaiting decoding or display. The frame descriptor records the starting storage address, data length, real-time transmission protocol sequence number range, and media transmission timestamp of the corresponding video frame data.

[0118] The status feedback module calculates the difference between the write pointer number and the read pointer number in the frame descriptor queue, and determines this difference as the number of queued frames waiting in the receive buffer. When the write pointer number is less than the read pointer number, the status feedback module adds the frame descriptor queue capacity to the write pointer number and then subtracts the read pointer number to obtain the number of queued frames.

[0119] Simultaneously, the status feedback module retrieves the preset chip hardware base frame period from the local read-only memory. The preset chip hardware base frame period is determined based on the display refresh rate configuration parameters currently read by the digital signal processor, and the calculation relationship is as follows: the preset chip hardware base frame period equals one second divided by the display refresh rate; when the display refresh rate is 60 Hz, the preset chip hardware base frame period is 16.67 milliseconds; when the display refresh rate is 120 Hz, the preset chip hardware base frame period is 8.33 milliseconds.

[0120] The status feedback module inputs the acquired number of queued frames and the preset chip hardware base frame period into the internal hardware multiplier for multiplication, generating the decoding pipeline delay. The specific decoding pipeline delay calculation formula is as follows: ; In the formula, Indicates the decoding pipeline delay; Indicates the number of queued frames; This indicates the preset chip hardware base frame period.

[0121] Decoding pipeline latency characterizes the buffering dwell time of streaming video data during the hardware processing stage within the target decoding node.

[0122] After completing the decoding pipeline delay calculation, the status feedback module uses the status authentication code, rendering sequence number, and decoding pipeline delay as payload data, and encapsulates the payload data into an authorized data packet according to the User Datagram Protocol (UDP) format.

[0123] After data encapsulation is completed, the status feedback module sends the authorized data packet to the corresponding source encoding node through the underlying network interface.

[0124] Reference Figure 7 In this embodiment, the signaling arbitration module executes step S5, which includes the following steps: S501 records the local synchronization timestamp and extracts the sending record of the streaming media sending module and compares its position with the rendering sequence number in the authorized data packet.

[0125] After receiving the authorized data packet at the underlying network interface, the signaling arbitration module and the streaming media sending module use the same local hardware clock counter on the source encoding node; the signaling arbitration module reads the absolute time count value of the arrival time of the authorized data packet and records the absolute time count value as a local synchronization timestamp.

[0126] After completing the time recording, the signaling arbitration module unpacks the authorized data packet and extracts the rendering sequence number from the payload data.

[0127] The signaling arbitration module accesses the transmission buffer queue of the streaming media transmission module and retrieves the video frame transmission record corresponding to the current session. The video frame transmission record includes the Real-Time Protocol (RTP) start sequence number and RTP end sequence number for each video frame. Based on the video frame transmission record, the signaling arbitration module locates the start sequence number position of the synchronization reference frame in the transmission buffer queue and determines the RTP sequence number range corresponding to the transmitting end sliding window starting from the synchronization reference frame. The extracted rendering sequence number is then compared with this RTP sequence number range to confirm the relative network progress of the current playback frame at the target decoding node.

[0128] S502, when it is determined that the rendering sequence number is within the sliding window of the sending end starting from the synchronization reference frame and the status authentication code is valid, the physical rendering output status is confirmed to be completed and the lock on the reception of the access control token is released.

[0129] If the value of the rendered sequence number falls within the range of real-time transmission protocol sequence numbers corresponding to the sliding window of the transmitting end starting from the synchronization reference frame, the signaling arbitration module determines that the rendered sequence number is within the sliding window of the transmitting end starting from the synchronization reference frame.

[0130] The sending end sliding window has a length of 32 video frame units, with a configurable range of 16 to 64 video frame units. The sending end sliding window is used to verify the physical rendering progress of the target decoding node and is not used for the collection and arbitration of multi-node control signaling.

[0131] After determining the position of the sliding window at the sending end, the signaling arbitration module extracts the status authentication code from the authorization data packet. Optionally, the signaling arbitration module uses the locally stored cross-domain session context, access control token, and rendering sequence number from the authorization data packet to perform local cryptographic hash calculation using the same secure hash algorithm as the target decoding node.

[0132] The signaling arbitration module performs a consistency comparison between the locally calculated status authentication code and the received status authentication code.

[0133] If the sliding window position determination at the sending end is valid and the status authentication code is consistent, the signaling arbitration module confirms that the target decoding node has completed the physical rendering output status.

[0134] Physical rendering output status refers to the physical display connected to the target decoding node having completed the physical display output of the image data of the synchronization reference frame or subsequent frames of the synchronization reference frame.

[0135] After confirming the physical rendering output status, the signaling arbitration module issues a run command to the local application layer firewall to remove the access control token from the denied access match in the firewall's access control list, thereby unlocking the access control token reception, restoring the underlying access permission for the corresponding control data packet, and sending control authorization notifications to the target decoding node and the cooperating control nodes in the authorized control node identifier set.

[0136] If the rendering sequence number is detected to be outside the sending end's sliding window, or if the status authentication code is inconsistent, the signaling arbitration module maintains the current lock state for receiving the access control token and directly discards the authorized data packet, blocking unsynchronized or illegal control signals.

[0137] S503 calculates net network latency by combining the request timestamp and the decoding pipeline latency.

[0138] The signaling arbitration module extracts the recorded request timestamp from the status identifier area in the source encoding node's memory and extracts the decoding pipeline latency from the authorized data packet.

[0139] The signaling arbitration module subtracts the request timestamp from the local synchronization timestamp to calculate the total round-trip time. The specific formula for calculating the total round-trip time is as follows: ; In the formula, Indicates the total round-trip time; Indicates local synchronization timestamp; This indicates a request for a timestamp.

[0140] After obtaining the total round-trip time, in order to eliminate the delay calculation error caused by the internal hardware buffer of the target decoding node, the signaling arbitration module subtracts the decoding pipeline delay from the total round-trip time.

[0141] Considering hardware counting granularity errors and decoding pipeline delay conversion errors, the signaling arbitration module takes the maximum value from the subtraction result and zero. The specific net network delay calculation formula is as follows: ; In the formula, Indicates net network latency; Indicates the decoding pipeline delay; This represents the function that takes the maximum value.

[0142] By taking the maximum value as described above, the signaling arbitration module deducts the buffer delay caused by the internal hardware processing of the target decoding node, and avoids abnormally negative results due to hardware counting granularity errors and decoding pipeline delay conversion errors. The net network latency calculated in this way reflects the transmission lag status under the current physical network link environment.

[0143] S504 dynamically calibrates the arbitration time window based on net network latency.

[0144] The signaling arbitration module retrieves the current port negotiation rate of the underlying network communication interface and matches the corresponding network jitter compensation coefficient based on the port negotiation rate.

[0145] Optionally, the network jitter compensation coefficient is configured as follows: when the port negotiation rate is greater than or equal to 1 Gbit / s, the network jitter compensation coefficient is 0.1; when the port negotiation rate is greater than or equal to 100 Mbit / s and less than 1 Gbit / s, the network jitter compensation coefficient is 0.2; and when the port negotiation rate is less than 100 Mbit / s, the network jitter compensation coefficient is 0.3.

[0146] The signaling arbitration module multiplies the network jitter compensation coefficient by half of the net network latency.

[0147] The signaling arbitration module reads the polling reference time window constant written by the device initialization phase based on the local area network basic communication delay baseline configuration from the configuration file. The polling reference time window constant has a value range of 5 milliseconds to 20 milliseconds, with a preferred value of 10 milliseconds.

[0148] The signaling arbitration module superimposes the product result onto the polling baseline time window constant to calculate the dynamically calibrated arbitration time window. The specific formula for calculating the arbitration time window is as follows: ; In the formula, This indicates the dynamically calibrated arbitration time window; This represents the preset polling baseline time window constant; This represents the network jitter compensation coefficient determined by the port negotiation rate; Represents net network delay; constant 2 is used to convert round-trip delay into one-way network transmission state parameters.

[0149] The calculated arbitration time window is used to collect and filter external control instructions in batches under concurrent control environment.

[0150] S505 completes the reassembly and injection of control signaling for multiple nodes within the arbitration time window.

[0151] After unlocking the control signaling receiving channel, the signaling arbitration module activates its internal concurrent processing thread.

[0152] During each polling interval, which lasts for an arbitration time window period, the signaling arbitration module receives multi-node control data packets sent by nodes in the network that are within the set of authorized control node identifiers that have received control authorization notices and hold valid access control tokens.

[0153] Multi-node control data packets carry signaling timestamps; for control signaling events such as keyboard key presses and mouse key presses, multi-node control data packets also carry the original event occurrence sequence number.

[0154] The signaling arbitration module disassembles the multi-node control data packets, extracts the signaling timestamps embedded in each message, and sorts the extracted control signaling according to the value of the signaling timestamps.

[0155] For different types of control signaling commands, the signaling arbitration module executes differentiated filtering logic: for absolute coordinate or state-overlay control signaling, such as mouse displacement coordinates, the signaling arbitration module retains the latest control signaling with the largest timestamp value; for paired event control signaling, such as keyboard key press and release operations, the signaling arbitration module performs sequential verification and retention according to the original event occurrence sequence number of the control data packet to avoid discarding old messages and causing the key state to remain in the pressed state.

[0156] The signaling arbitration module reassembles the HMI report data of the retained control signaling according to the Universal Serial Bus (USB) HMI device protocol standard; the conversion and injection process of the USB HMI report data can be configured with reference to the existing Universal Peripheral Bus (UPPC) communication standard.

[0157] After reassembly, the signaling arbitration module injects the reassembled control signaling into the underlying driver interface of the source device. For out-of-order packets that lag behind the current arbitration time window, the signaling arbitration module performs data clearing and discarding operations.

[0158] Differentiated data filtering rules are used to enable source devices to respond to effective collaborative operation commands in multi-node concurrent control scenarios.

[0159] To further illustrate the implementation process and technical effects of the present invention, the following description is provided in conjunction with specific application scenarios. The specific numerical values ​​and scenario parameters described below are only used to explain the implementation principle of the present invention and do not limit the scope of protection of the present invention.

[0160] Deploy a distributed information interconnection console system based on modular hardware in the command center network environment, and operators can initiate cross-domain collaboration requests from the console terminal.

[0161] The session initialization module responds to cross-domain collaboration requests by generating a cross-domain session context containing a 256-bit session identifier in memory. The module then triggers an internal hardware random number generator to generate a 256-bit random byte sequence as an access control token. Finally, it maps and binds an encrypted encapsulated data field containing the access control token and session identifier to the cross-domain session context.

[0162] The streaming media transmission module uses a local hardware clock to record the absolute time count of the current moment, storing 1000.00 milliseconds as the request timestamp in the status identifier area of ​​memory. The module dynamically writes the parsed access control token as a feature matching field into the access control list of the local firewall component, and performs a rejection operation on control packets carrying the access control token, locking the control signaling reception channel. The module iterates through the transmission buffer queue, locates the first multimedia frame data with a media transmission timestamp greater than or equal to 1000.00 milliseconds and a frame encoding type of intra-frame encoded frame, and extracts this first multimedia frame data as the synchronization reference frame. The module extracts the start sequence number 1000, end sequence number 1005, and media transmission timestamp from the real-time transmission protocol header of the synchronization reference frame, encapsulates them to generate a multicast notification message, sends it to the multicast address of the source encoding node, and repeats the message 5 times at a preset notification period of 10 milliseconds.

[0163] The streaming media receiving module sends an Internet Group Management Protocol (IGMP) join message to access the multicast tree of the source-end encoding node associated with the cross-domain session context, and receives a multicast notification message containing the synchronization reference frame sequence number range and the media transmission timestamp. The streaming media receiving module issues a register control command to the local display controller to cut off the effective video signal transmission between the main video memory and the physical output interface, initiates the display blanking state, and simultaneously pushes the streaming media video data received by the underlying network card into the receive buffer. The streaming media receiving module addresses and locates video frame data in the receive buffer whose time sequence equals the synchronization reference frame's time attribute. After confirming the existence of complete video frame data corresponding to the synchronization reference frame, it calls the local digital signal processor to perform hardware decoding operations on the synchronization reference frame. The local digital signal processor completes the data decoding operation and flips the uncompressed pixel matrix data, writing it to the main video memory of the local display controller. The vertical synchronization signal output terminal of the local display controller generates a level transition signal, generating a vertical synchronization interrupt.

[0164] When the status feedback module detects a vertical synchronization interruption, it uses the frame attribute mapping table to retrieve the rendering sequence number (1000) corresponding to the physically rendered video frame. The status feedback module performs an XOR hash operation on the rendering sequence number, session identifier, and access control token to generate a status authentication code. The status feedback module reads the receive buffer queue status, calculates the difference between the write pointer number and the read pointer number of the frame descriptor queue, and determines that there are 3 frames waiting in the receive buffer. The current display refresh rate of the digital signal processor is 60 Hz, and the preset chip hardware basic frame period is 16.67 milliseconds. The status feedback module multiplies the number of queued frames (3) with the preset chip hardware basic frame period (16.67 milliseconds) in its internal hardware multiplier to generate a decoding pipeline latency of 50.01 milliseconds. The status feedback module encapsulates the status authentication code, rendering sequence number, and 50.01 milliseconds decoding pipeline latency as payload data into an authorization data packet, which is then sent to the corresponding source encoding node through the underlying network interface.

[0165] After receiving the authorization data packet at the underlying network interface, the signaling arbitration module reads the absolute time count of the arrival time of the authorization data packet and records 1060.00 milliseconds as the local synchronization timestamp. The signaling arbitration module accesses the streaming media sending module's sending buffer queue, retrieves the video frame sending record corresponding to the current session, and determines the real-time transmission protocol sequence number range corresponding to the sending end sliding window with a length of 32 video frame units, starting from the synchronization reference frame, based on the video frame sending record. It confirms that the rendering sequence number 1000 falls within this real-time transmission protocol sequence number range. The signaling arbitration module uses the same secure hash algorithm as the target decoding node to perform local cryptographic hash calculation, and compares the locally calculated status authentication code with the received status authentication code. If the comparison is successful, the signaling arbitration module confirms the completion of the physical rendering output status, removes the access control token from the firewall's access control list's denied match, unlocks the reception of the access control token, and sends a control authorization notification to the target decoding node and the cooperating control nodes within the authorized control node identifier set. The signaling arbitration module subtracts the request timestamp of 1000.00 milliseconds from the local synchronization timestamp of 1060.00 milliseconds, calculating the total round-trip latency to be 60.00 milliseconds. The signaling arbitration module then subtracts the decoding pipeline latency of 50.01 milliseconds from the total round-trip latency of 60.00 milliseconds, resulting in a latency difference of 9.99 milliseconds. Considering hardware counting granularity errors, the maximum value is taken between 9.99 milliseconds and zero, resulting in a net network latency of 9.99 milliseconds.

[0166] The signaling arbitration module retrieves the current port negotiation rate of the underlying network communication interface, which is 1 Gbit / s, and matches a network jitter compensation coefficient of 0.1 based on this rate. The module reads the polling baseline time window constant, written during device initialization based on the LAN basic communication delay configuration, from the configuration file; this constant is 10.00 milliseconds. The module multiplies the network jitter compensation coefficient of 0.1 by half the net network latency (4.995 milliseconds), resulting in a product of 0.4995 milliseconds. This product is then added to the polling baseline time window constant of 10.00 milliseconds to calculate a dynamically calibrated arbitration time window of 10.4995 milliseconds. Within each polling interval of this 10.4995 millisecond arbitration time window period, the module receives multi-node control data packets sent by nodes within the set of authorized control node identifiers that have received control authorization notifications and hold valid access control tokens, and extracts the signaling timestamps embedded in each packet. For absolute coordinate control signaling, the signaling arbitration module retains the latest control signaling with the largest timestamp value, reassembles the HMI device report data according to the Universal Serial Bus HMI device protocol standard, injects the reassembled control signaling into the underlying driver interface of the source device, and performs data clearing and discarding operations on out-of-order messages that lag behind the current arbitration time window period.

[0167] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result using substantially the same method falls within the scope of the present invention.

Claims

1. A distributed information interconnection control console system based on modular hardware, characterized in that, include: The session initialization module is used to generate a cross-domain session context in response to cross-domain collaboration requests and extract the access control token bound to the cross-domain session context; The streaming media sending module is used to record the request timestamp, lock the control signaling receiving channel according to the access control token, and extract the first complete keyframe in the sending buffer queue as the synchronization reference frame. The streaming media receiving module is used to access the multicast tree according to the cross-domain session context, buffer streaming media video data to the receiving buffer in the display blanking state, and generate a vertical synchronization interrupt based on the synchronization reference frame. The status feedback module is used to generate a status authentication code in conjunction with the access control token when the vertical synchronization interrupt is triggered, calculate the decoding pipeline delay using the number of queued frames in the receive buffer, and output an authorized data packet containing the rendering sequence number, the status authentication code and the decoding pipeline delay. The signaling arbitration module is used to receive the authorized data packet and record the local synchronization timestamp, verify the physical rendering output status of the target decoding node according to the status authentication code and the rendering sequence number, calculate the net network latency by combining the request timestamp, the local synchronization timestamp and the decoding pipeline latency, and complete the control signaling reassembly and injection based on the net network latency to calibrate the arbitration time window.

2. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the session initialization module, the steps of generating a cross-domain session context in response to a cross-domain collaboration request and extracting the access control token bound to the cross-domain session context include: Read the local hardware clock timestamp and hardware device identification code, concatenate them, and perform a hash operation to generate a session identifier; Extract the target operating device information from the cross-domain collaboration request, and retrieve the source end encoding node identifier and the source end encoding node multicast address from the local static routing table based on the target operating device information; Generate a random byte sequence as the access control token; It also includes: using a node shared key to perform symmetric encryption on a data field containing the access control token and the session identifier to obtain an encrypted encapsulated data field, and establishing a memory mapping association between the encrypted encapsulated data field and the cross-domain session context.

3. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the streaming media sending module, the steps of recording the request timestamp and locking the control signaling receiving channel according to the access control token include: Read the nanosecond-level time count value of the real-time clock chip as the request timestamp, and store the request timestamp in the status identifier area; Write the access control token as a feature matching field into the access control list; The system receives control data packets, extracts identity credential data from the network transmission protocol header extension field of the control data packets, compares the identity credential data with the access control list, and triggers a data discard command to lock the control signaling receiving channel when the comparison matches.

4. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the streaming media sending module, the step of extracting the first complete keyframe in the sending buffer queue as the synchronization reference frame includes: Read the media transmission timestamp of the multimedia frame data in the transmission buffer queue; The value of the media transmission timestamp is compared with the value of the request timestamp, and the first multimedia frame data with the media transmission timestamp not earlier than the request timestamp and the frame encoding type is an intra-frame coded frame is located as the synchronization reference frame. It also includes: extracting the start sequence number, end sequence number, and media transmission timestamp of the synchronization reference frame, encapsulating them to generate a multicast notification message, and sending the multicast notification message to the multicast address of the source encoding node.

5. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the streaming media receiving module, the step of buffering streaming media video data to the receiving buffer in the display blanking state, based on the cross-domain session context access to the multicast tree, includes: Extract the source encoding node multicast address from the cross-domain session context, send an Internet Group Management Protocol (IGM) join message to establish a multicast mapping, and push the received streaming video data into the receiving buffer. A register control command is sent to the display controller to cut off the video signal transmission between the main video memory and the physical output interface and initiate the display blanking state; It also includes: addressing the first video frame data whose time sequence is no earlier than the synchronization reference frame time attribute in the receiving buffer, extracting the media transmission timestamp of the first video frame data, resetting the internal time counter of the digital signal processor, and establishing a hardware decoding clock reference.

6. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the streaming media receiving module, the step of generating a vertical synchronization interrupt based on the synchronization reference frame includes: After verifying that all fragmented data packets of the synchronization reference frame are completely residing in the receiving buffer, the digital signal processor is called to perform decoding operations on the synchronization reference frame to flip the pixel matrix data and write it into the main video memory. When the display controller scans and outputs the pixel matrix data in the main video memory, triggering a vertical synchronization signal level transition, the vertical synchronization interrupt is captured and generated by the interrupt controller. It also includes: issuing a state reset command to remove the display blanking state after the vertical synchronization interruption is triggered, and not sending control data packets carrying the access control token to the source encoding node when the control signaling injection is in an unauthorized state.

7. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the status feedback module, the step of generating a status authentication code in conjunction with the access control token when the vertical synchronization interrupt is triggered includes: When the vertical synchronization interruption is captured, the physical address of the video frame in physical rendering state is obtained, and the real-time transmission protocol start number is queried based on the physical address of the video frame using the frame attribute mapping table as the rendering sequence number. The rendering sequence number, the session identifier in the cross-domain session context, and the access control token are respectively converted into binary fields and bit-width consistency processing is performed. Perform a binary XOR operation on the rendering sequence number, the session identifier, and the access control token after the bit width is consistent, and input the XOR operation result into a secure hash algorithm to generate a one-way hash digest as the status authentication code.

8. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the status feedback module, the step of calculating the decoding pipeline delay using the number of queued frames in the receive buffer includes: The write pointer number and read pointer number of the receive buffer are read, and the difference between the write pointer number and the read pointer number is calculated to determine the number of queued frames; When the write pointer number is less than the read pointer number, the write pointer number is added to the frame descriptor queue capacity and then subtracted from the read pointer number to obtain the number of queued frames. Extract the chip hardware base frame period configured according to the display refresh rate; The decoding pipeline delay is obtained by multiplying the number of queued frames with the chip hardware base frame period input hardware multiplier.

9. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the signaling arbitration module, the step of calculating the net network latency by combining the request timestamp, the local synchronization timestamp, and the decoding pipeline latency includes: The total round-trip time is obtained by subtracting the absolute time count of the requested timestamp from the absolute time count of the local synchronization timestamp. Subtract the decoding pipeline delay from the total round-trip delay to obtain the delay subtraction result; The result of subtracting the delay from the zero value is compared with the maximum value, and the maximum value of the comparison output is determined as the net network delay.

10. The distributed information interconnection control console system based on modular hardware according to claim 1, characterized in that, In the signaling arbitration module, the steps for completing control signaling reassembly and injection based on the net network delay to calibrate the arbitration time window include: Obtain the network jitter compensation coefficient corresponding to the port negotiation rate matching of the network interface, multiply the network jitter compensation coefficient by half of the net network delay to obtain the product result, and superimpose the product result onto the preset polling reference time window constant to obtain the arbitration time window; Within the polling interval of each arbitration time window, receive multi-node control data packets sent by nodes holding valid access control tokens within the authorized control node identifier set, extract the signaling timestamps embedded in the multi-node control data packets, and sort the control signaling according to their numerical values. For absolute coordinate control signaling, the control signaling with the largest timestamp value is retained. For paired event control signaling, the control signaling is retained in sequence according to the original event occurrence number. The retained control signaling is converted into data and injected into the source device using the Universal Serial Bus human-machine interface device protocol.

Images