Low-delay vehicle-mounted rear-row dual-screen wireless bidirectional audio-video touch linkage system and method

Abstract

The application relates to a low-delay vehicle-mounted rear-row double-screen wireless bidirectional audio-video touch linkage method and system, which comprises a linkage same-display mode, a master-slave switching mode and an independent working mode. In the linkage same-display mode, a sending end encapsulates video data into a single-package layered structure for wireless sending, and a receiving end can add a touch subframe to the data structure and then reversely send the data structure to the sending end to realize double-screen synchronous same-display and cross-screen touch reverse control. In the master-slave switching mode, two SOC ends realize non-service interruption switching through a transition overlap state, gradient bandwidth scheduling and clock phase smooth compensation. In the independent working mode, the receiving end detects a local signal, enters a priority switching process and makes the two ends work independently. The application realizes low-delay audio-video synchronous transmission, non-interruption master-slave switching, multi-source independent working and safe and compliant signal source identification, and is especially suitable for multi-scene applications such as double-screen linkage movie watching, conference office work, multi-person game and the like in the rear row of a passenger car.

Description

Technical Field

[0001] This invention relates to the field of vehicle display technology, and more specifically, to a low-latency vehicle rear dual-screen wireless two-way audio and video touch linkage system. Background Technology

[0002] With the improvement of automotive intelligence and the growth of entertainment needs for rear-seat passengers, in-vehicle dual-screen interactive systems are gradually becoming standard features in high-end models. In actual use, dual-screen systems need to cope with a variety of complex application scenarios, especially adapting to multi-scenario applications such as movie viewing, meeting and office work, and multiplayer games in passenger vehicles.

[0003] However, existing in-vehicle rear dual-screen interaction technology has significant shortcomings in the following aspects: First, wireless screen projection solutions only support one-way video transmission and cannot achieve two-way interaction. Existing native wireless screen projection technologies (such as Miracast, AirPlay, and Google Cast) are mainly geared towards consumer electronics scenarios, supporting only one-way transmission of video data from the signal source device to the display device, and cannot enable reverse touch control from the display device to the signal source device. In the scenario of a dual-screen rear seat in a car, this means that rear passengers cannot control the video being played (such as pause, fast forward, or volume adjustment) by touching the screen, nor can they perform interactive operations such as touch page turning and annotation in a meeting scenario, severely limiting the practicality and user experience of the rear dual-screen system.

[0004] Second, touch data and video data are transmitted independently, resulting in uncontrollable end-to-end latency. Some solutions attempt to transmit touch data through independent channels to achieve reverse control, but touch messages and video streams follow different transmission paths, leading to uncontrollable latency differences between touch response and screen refresh. In movie-watching scenarios, the screen may respond after a user touches the pause button, with a delay of hundreds of milliseconds. In gaming scenarios, the asynchrony between touch operation and screen feedback leads to significant "responsiveness" issues, severely impacting the immersive experience.

[0005] Third, service interruptions occur during master-slave switching. In dual-screen scenarios, when switching the identity of the master and slave devices (e.g., transferring control from one screen to another), existing solutions typically require disconnecting the current wireless connection and then re-establishing a new one. This process can cause issues such as screen freezing, touch malfunction, and intermittent audio, lasting from hundreds of milliseconds to several seconds. During vehicle operation, frequent interruptions not only affect user experience but can also distract the driver due to sudden screen changes, potentially leading to safety hazards.

[0006] Fourth, the lack of an effective arbitration mechanism when two independent signal sources are connected simultaneously: When two terminal devices simultaneously connect to independent signal sources (e.g., one screen displays navigation, and the other displays a game), existing solutions lack effective channel isolation and command arbitration mechanisms. Touch commands from both ends may be sent to the same signal source simultaneously, causing conflicts. Competition for wireless channel resources can lead to screen stuttering or glitches, and crosstalk between adjacent devices reduces connection stability. These problems are particularly prominent in multiplayer game scenarios, directly impacting the game's playability and fairness.

[0007] Fifth, the current solution relies excessively on automatic logic for mode switching, lacking a means for manual intervention. Existing solutions overly depend on automatic logic (such as signal detection and load balancing) for mode switching decisions, lacking a way for users to manually intervene. When the automatic logic's judgment does not match the user's actual intention (for example, the system mistakenly identifies a brief signal jitter as a condition for last-in-first-out switching), the user cannot easily lock the current mode, reducing the system's controllability and adaptability.

[0008] Therefore, there is an urgent need for a method and system for wireless interaction between dual rear screens in vehicles that can achieve low-latency two-way audio and video touch control linkage, support uninterrupted master-slave switching, have multi-source independent working capabilities, and also accommodate manual intervention. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage method and a low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage system, in view of the above-mentioned defects of the prior art.

[0010] The technical solution adopted by this invention to solve its technical problem is: A low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method is constructed and applied to a system including a first terminal device and a second terminal device. The first terminal device and the second terminal device can dynamically act as master and slave transceivers and are connected through a wireless link. The method includes a linkage display mode, a master-slave switching mode and an independent working mode. The linked display mode includes the following steps: Step S1: In the first terminal device or the second terminal device, the one that first detects a valid input signal is used as the transmitter, and the other is used as the receiver; the scaler unit of the transmitter identifies the input signal, completes the image preprocessing, and sends the preprocessed video data to the local wireless image transmission SOC unit through the DP video interface. In step S2, the wireless image transmission SOC unit of the transmitting end enters the transmitting state, encapsulates the video data into a single-packet layered structure, which includes a common frame header, video data, and an idle time slot for reverse carrying of touch subframes, and transmits it wirelessly through the antenna; the touch subframes generated by the receiving end are filled into the idle time slots through the reverse link and transmitted back to the transmitting end. Step S3: The wireless image transmission SOC unit at the receiving end enters the receiving state, receives and parses the common frame header, and extracts the video data. In step S4, the separated video data is processed by the Scaler unit of the receiving end and then output by the display module. The touch coordinates collected by the touch module of the receiving end are corrected by the FPGA and then encapsulated into touch subframes by the MCU of the receiving end. These subframes are then filled into the idle time slots of the video frame of the sending end through the reverse link and sent back to the sending end. After the sending end separates the touch subframes, they are sent to the MCU unit of the sending end for parsing and execution, realizing cross-screen touch reverse control. The master-slave switching mode includes the following steps: Step S5: In the linked display mode, when it is necessary to switch master and slave identities, the wireless image transmission SOC units at both ends complete the master-slave switch without service interruption through gradient bandwidth scheduling and clock phase smoothing compensation. The independent working mode includes the following steps: Step S6: In the linked display mode, when the receiving end detects a valid local signal input, it triggers the last-input priority switching process: The scaler unit of the receiving end sends a switching command to the local wireless image transmission SOC via UART. The wireless image transmission SOC sends a mode switching warning packet with CRC check to the sending end. After the sending end replies with an ACK response, the wireless image transmission SOCs at both ends synchronously enter the transition overlap state, complete the handover of the transmit and receive paths and clock phase smoothing compensation, so that both ends can work independently.

[0011] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein in step S2, the single-packet layered encapsulation structure adopts an audio and video touch frame nested asynchronous layered packaging algorithm. definition: For video frame period, This is the reporting cycle for capacitive touchscreens. The wireless transmission time for a single frame of video data; The remaining idle time slot length after a single frame of video transmission satisfies ; The maximum number of touch messages that can be embedded within a single frame's idle time slot, satisfying the round-down constraint. ; For the common frame header length, The length of a single video frame. The length of the payload of the nth touch subframe; The hierarchical data packet corresponding to the kth video frame period The structure is as follows: ; in, The common frame header includes the frame number, layer identifier, video payload length, and number of embedded touch subframes; This is the video encoded data for the k-th frame; To collect touch messages from the receiving end and transmit them back to the sending end via the reverse link, the reverse time slot of TDD time division duplex is nested within the idle time slot of the sending end's video frame. The actual number of touch subframes embedded in this frame, satisfying... ; When the idle time slot of a single frame is insufficient to accommodate the touch message to be sent, a cross-frame buffering and retransmission mechanism is adopted. The touch message fills the idle time slot of the subsequent video frame in the order of arrival, and the buffer queue length does not exceed 2 frames. The receiver parses the layer identifier and length field of the common frame header and performs a hard decision to split the payload into two types based on the offset: .

[0012] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein the master-slave switching in step S5 and the last-in priority switching in step S6 both adopt a single SOC multi-state smooth transition dynamic master-slave transceiver switching mechanism: The wireless image transmission SOC unit has four basic operating states and one transitional overlapping state. The four basic operating states include: Standby state S0, where there is no valid signal input and the SOC is in low-power standby mode; Transmitting state S1, where local video data is encoded and wirelessly uploaded, while simultaneously receiving touch message feedback from the other end; Receiving state S2, where the other end's video data is decoded and output for display, while simultaneously uploading local touch message feedback; Hybrid state S3, where time-division multiplexing occurs within a single frame period, with the first half receiving video and the second half transmitting touch data, supporting bidirectional interaction in linked mode; and Transitional overlapping state. Used for simultaneous operation of the transmit and receive paths during switching; Switching triggers are determined by a comprehensive score, which includes the overall switching trigger score. The calculation formula is: ; in, The score represents the validity of the local input signal; it is 1 when there is a stable and valid signal and -1 when there is no signal. The score for user role requirements is -1 when the peer requests master control and 1 when the local peer requests master control. This represents the local computing power load score; the lower the load, the higher the score, ranging from [-1, 1]. For the corresponding weights, the sum of the weights must be 1; Set the decision threshold ,when And when not in the sending state, a switch to the sending master state is triggered; when Furthermore, when not in the receiving state, a switch to the receiving slave state is triggered; the current state is maintained during the intermediate interval to suppress ping-pong switching; Before the handover is triggered, the local SOC sends a handover warning packet with CRC check. Only after receiving the ACK response from the peer can it enter the transition overlap state. Before the handover, a snapshot of the full state register is saved. If the handover times out, it immediately rolls back to the original working state and retryes at exponentially increasing intervals.

[0013] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein, in the transition overlapping state Inside, set the duration of overlapping windows. ,satisfy: ; in, This refers to the transmission time of the remaining data to be completed in the original transmission path. The time from pre-startup of the receiving path to stable output. Timing redundancy margin reserved for automotive scenarios; Within the overlap window, the transmit path bandwidth decreases linearly with time, while the receive path bandwidth increases linearly with time, and the total bandwidth throughout does not exceed the bandwidth limit of the SOC shared hardware. , respectively represented as: ; Where t is the running time within the overlapping window, and its value ranges from 0 to... .

[0014] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein when the main clock reference source changes before and after switching, a main and backup dual PLL architecture is used for clock phase smoothing compensation, and the backup PLL tracks the target clock in advance to measure the initial phase difference. Then, step-type phase compensation is adopted, with a single-step compensation amount. satisfy: ; in, The total number of compensation steps is the maximum clock jitter allowed by the system. The master clock source is switched after the phase converges.

[0015] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch linkage method of the present invention, wherein, before executing step S1, the wireless image transmission SOC unit adopts an RSSI dynamic threshold and group ID dual-layer anti-crosstalk pairing algorithm. The device will only enter the pairing process if the received broadcast packet signal strength meets the following conditions: ; in, The actual signal strength of the received broadcast packet, expressed in dBm; The default threshold for the enclosed space in a vehicle is -65dBm. For congestion dynamic adjustment, to meet C represents the congestion level of Wi-Fi data packets in the channel within 1 second, with a value ranging from 0 to 100. After pairing is complete, the two ends exchange a unique encrypted packet ID. All subsequent data packets will carry this ID in their headers. The receiving end will only process the data after verifying that the IDs match.

[0016] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention includes, in step S1, the image preprocessing includes generating rotation parameters, and in step S4, the FPGA unit adopts a screen rotation touch coordinate matrix correction algorithm. After the screen is physically rotated by an angle θ, the original acquired coordinates Target coordinates corrected to fit the image The transformation formula is: ; in is the radian value of the physical rotation of the screen; k is the screen scaling factor, which is equal to the ratio of the output resolution to the original input resolution; W and H are the horizontal and vertical physical resolutions of the touch screen, respectively.

[0017] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein in step S4, when there are concurrent touch commands from both ends, the MCU unit adopts a multi-touch command weighted arbitration algorithm. The priority score S for a single touch command is: ; in The score is normalized to timestamps; the earlier the instruction arrives, the higher the score, with different weights. Prioritize low latency; The probability of touch trajectory continuation is calculated based on the motion trend prediction of the touch points in the previous 5 frames, with a value ranging from 0 to 1, and a weight. ; The arbitration rules are as follows: only the instruction with the highest score is sent to the signal source at any given time; the instruction with the second highest score is buffered for a maximum of 2 frame cycles, and if there is an idle time slot during the buffering period, it will be sent again.

[0018] The low-latency in-vehicle rear dual-screen wireless bidirectional audio and video touch control linkage method of the present invention, wherein in step S4, when the input signal is the original vehicle head unit HDMI / DP signal, the touch control reverse control path is locked to prevent cross-screen touch control back to the vehicle head unit; It also includes manual forced switching rules: users can manually lock the linkage display mode or the independent working mode by pressing a button. The manual command has higher priority than all automatic polling and last-in priority logic.

[0019] A low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage system is used to execute the low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage method as described above, wherein it includes a first terminal device and a second terminal device; the first terminal device and the second terminal device can dynamically act as master and slave transceivers to each other and are connected through a wireless link; Both the first terminal device and the second terminal device include: The multi-input interface module includes HDMI / DP and Type-C interfaces for connecting to the original vehicle system signal or portable device signal; The Scaler unit is used for image preprocessing and multi-channel signal switching; FPGA unit, used for screen rotation and touch coordinate correction; Display module and touch module; The MCU unit is used for touch message processing and command arbitration. Buttons and motor module are used for manual mode switching and physical screen rotation; The wireless image transmission SOC unit supports TDD (Time Division Duplex) dynamic switching of transmit and receive states for wireless transmission and reception of audio, video, and touch data.

[0020] The beneficial effects of this invention are as follows: 1. Achieve low-latency synchronous display on both screens and cross-screen touch control; Through the linkage and simultaneous display mode described in steps S1 to S4 of claim 1, the sending end encapsulates the video data into a single-packet layered structure with a reverse idle time slot and sends it through a wireless link. The receiving end outputs the video after hard-decision separation. The touch commands generated by the receiving end are transmitted back to the sending end through the reverse link, and are parsed and executed by the sending end, realizing cross-screen touch control. Compared with the shortcomings of existing wireless screen projection solutions that only support unidirectional video transmission and are not synchronized with touch and video, this invention encapsulates touch data and video data in the same data packet and transmits them synchronously based on the timestamp of the same common frame header. The end-to-end audio and video delay difference with touch is determined by the intra-frame offset and is constant, fundamentally solving the problem of asynchrony between touch response and screen refresh, especially suitable for scenarios with high real-time requirements for touch, such as watching movies, conferencing, and gaming.

[0021] 2. Achieve master-slave failover without service interruption; Through the master-slave switching mode described in step S5 of claim 1, when it is necessary to switch master and slave identities, the two SOC units enter a transitional overlap state, and the handover of the transmit and receive paths is completed through gradient bandwidth scheduling and clock phase smoothing compensation. Compared with the shortcomings of existing solutions that require disconnecting the current connection and then re-establishing it, resulting in screen freezing and touch failure, this invention keeps the transmit and receive paths running simultaneously in the transitional overlap state. The transmit path bandwidth decreases linearly with time, and the receive path bandwidth increases linearly with time. The total bandwidth throughout does not exceed the bandwidth limit of the shared hardware of the SOC. With clock phase smoothing compensation, the master-slave switching process is achieved with zero interruption and imperceptibility, meeting the requirements of seamless switching in multiple scenarios.

[0022] 3. Enables conflict-free independent operation of dual independent signal sources; Through the independent operating mode described in step S6 of claim 1, when the receiving end detects a valid local signal input, it triggers the last-in-first-out handover process: the receiving end sends a mode switching warning packet with CRC check to the transmitting end, and after the transmitting end replies with an ACK response, both SOCs synchronously enter the transition overlap state, completing the handover of the transmit and receive paths and clock phase smoothing compensation, allowing both ends to work independently. Compared with the shortcomings of existing solutions that lack an effective arbitration mechanism and are prone to command conflicts and channel contention when both ends simultaneously access independent signal sources, this invention ensures the reliability and orderliness of the handover through the warning-acknowledgment handshake mechanism, guarantees the continuity of the handover process through the transition overlap state, and achieves complete isolation of the two signal sources through the independent operating mode, preventing mutual interference. It is especially suitable for scenarios such as multiplayer games that require independent operation of both ends.

[0023] 4. Adapts to multiple scenarios and allows for flexible switching; By organically combining the three working modes described in claim 1—simultaneous display mode, master-slave switching mode, and independent working mode—this invention can flexibly adapt to various typical application scenarios of dual screens in the rear seats of passenger vehicles: the simultaneous display mode is suitable for scenarios such as simultaneous viewing of movies on both screens and sharing of meeting materials; the master-slave switching mode is suitable for scenarios such as transfer of control; and the independent working mode is suitable for scenarios such as each screen running games or office work independently. The three modes can be automatically or manually switched according to signal access and user needs, achieving full scenario coverage from single-end input to dual-end independence, significantly improving the practicality and user experience of the in-vehicle rear-seat dual-screen system. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort: Figure 1 This is a flowchart of a preferred embodiment of the low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage method; Figure 2 This is a block diagram illustrating the principle of a low-latency in-vehicle rear dual-screen wireless two-way audio and video touch linkage system according to a preferred embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, a clear and complete description will be provided below in conjunction with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0026] The preferred embodiment of the present invention provides a low-latency in-vehicle rear-seat dual-screen wireless bidirectional audio and video touch control linkage method, applicable to a system including a first terminal device and a second terminal device. The first terminal device and the second terminal device can dynamically act as master and slave transceivers, connected via a wireless link, such as... Figure 1 As shown, it includes simultaneous display mode, master-slave switching mode and independent working mode; The simultaneous display mode includes the following steps: Step S1: In the first terminal device or the second terminal device, the one that first detects a valid input signal is used as the transmitter, and the other is used as the receiver; the scaler unit of the transmitter identifies the input signal, completes the image preprocessing, and sends the preprocessed video data to the local wireless image transmission SOC unit through the DP video interface. In step S2, the wireless image transmission SOC unit of the transmitting end enters the transmitting state, encapsulates the video data into a single-packet layered structure, which includes a common frame header, video data, and an idle time slot for reverse carrying of touch subframes, and transmits it wirelessly through the antenna; the touch subframes generated by the receiving end are filled into the idle time slots through the reverse link and transmitted back to the transmitting end. Step S3: The wireless image transmission SOC unit at the receiving end enters the receiving state, receives and parses the common frame header, and extracts the video data. In step S4, the separated video data is processed by the Scaler unit of the receiving end and then output by the display module. The touch coordinates collected by the touch module of the receiving end are corrected by the FPGA and then encapsulated into touch subframes by the MCU of the receiving end. These subframes are then filled into the idle time slots of the video frame of the sending end through the reverse link and sent back to the sending end. After the sending end separates the touch subframes, they are sent to the MCU unit of the sending end for parsing and execution, realizing cross-screen touch reverse control. The master-slave switching mode includes the following steps: Step S5: In the linkage and display mode, when it is necessary to switch the master and slave identities, the wireless image transmission SOC units at both ends complete the master and slave switching without service interruption through gradient bandwidth scheduling and clock phase smooth compensation. The independent working mode includes the following steps: Step S6: In the simultaneous display mode, when the receiving end detects a valid local signal input, the last-input priority switching process is triggered. The scaler unit at the receiving end sends a switching command to the local wireless transmission SOC via UART. The wireless transmission SOC sends a mode switching warning packet with CRC check to the transmitting end. After the transmitting end replies with an ACK response, the wireless transmission SOCs at both ends synchronously enter the transition overlap state, completing the handover of the transmit and receive paths and clock phase smoothing compensation, so that both ends can work independently.

[0027] By employing a synchronized display mode, this method achieves fully synchronized dual-screen display and cross-screen touch control, solving the problem that existing wireless projection solutions only support unidirectional video transmission and cannot achieve bidirectional interaction. Through a master-slave switching mode, it enables seamless identity switching, resolving the issue of screen freezing and touch failure caused by disconnection and reconnection during switching in existing solutions. Furthermore, by using an independent working mode, it ensures that both ends operate independently without interference, addressing the lack of an effective arbitration mechanism when both ends simultaneously access independent signal sources. This organic combination of three working modes allows this method to flexibly adapt to various in-vehicle rear-seat application scenarios such as movie watching, conferencing, and gaming.

[0028] Preferably, in step S2, the single-packet layered encapsulation structure adopts an audio / video touch frame nested asynchronous layered packaging algorithm. Conventional solutions establish two independent Wi-Fi links for video and touch, resulting in significant channel contention and latency differences of 5-10ms, leading to touch drift. Existing layered transmission patents are mostly media stream layering under the RTP protocol, without optimizing intra-frame time slot nesting for vehicle-mounted TDD single-channel scenarios, resulting in large fluctuations in touch latency.

[0029] This algorithm is based on the video frame period of a single TDD channel and constructs a single-packet layered encapsulation structure (video payload body + nested touch subframes in the idle time slot at the end of the frame). There is only one channel contention throughout the process, eliminating the time delay difference between the two links at the physical layer. The receiving end uses hard decision-based layer separation through common frame header identifiers to synchronously trigger video decoding and touch parsing.

[0030] definition: This is the video frame period, which corresponds to approximately 16.67ms at a frame rate of 60fps. This is the capacitive touch reporting cycle, which corresponds to approximately 8.33ms at a 120Hz sampling rate. The wireless transmission time for a single frame of video data is typically 12ms for 1080P H.265 encoding. The remaining idle time slot length after a single frame of video transmission satisfies ; The maximum number of touch messages that can be embedded within a single frame's idle time slot, satisfying the round-down constraint. ; For the common frame header length, The length of a single video frame. The length of the payload of the nth touch subframe; The hierarchical data packet corresponding to the kth video frame period The structure is as follows: ; in, The common frame header includes the frame number, layer identifier, video payload length, and number of embedded touch subframes; This is the video encoded data for the k-th frame; To collect touch messages from the receiving end and transmit them back to the sending end via the reverse link, the reverse time slot of TDD time division duplex is nested within the idle time slot of the sending end's video frame. The actual number of touch subframes embedded in this frame, satisfying... ; When the idle time slot of a single frame is insufficient to accommodate the touch message to be sent, a cross-frame buffering and retransmission mechanism is adopted. The touch message fills the idle time slot of the subsequent video frame in the order of arrival, and the buffer queue length does not exceed 2 frames. The receiver parses the layer identifier and length field of the common frame header and performs a hard decision to split the payload into two types based on the offset: .

[0031] The separated video payload is sent to the Scaler for decoding and display, while the touch subframe is sent to the MCU for parsing and execution. Both types of data are sent synchronously based on the same common frame header timestamp. The end-to-end delay difference is determined by the intra-frame offset, eliminating random jitter caused by dual-link channel contention.

[0032] Example of numerical substitution calculation: Substituting typical in-vehicle 1080P 60fps scene parameters: video frame period ms, the time taken to transmit a single frame of video ms, therefore, a single frame's idle time slot: ; Touch reporting cycle ms, therefore, the number of touch messages that can be embedded in a single frame idle time slot is: ; This means that the idle time slots in a single frame are insufficient to accommodate a complete touch message, so a cross-frame buffering mechanism is used, and the total idle time slots for two consecutive frames are... ms, can hold 1 touch message, maximum touch buffer latency is ms, which is far below the human perception threshold of 50ms in game scenarios.

[0033] Latency difference verification: Video and touch data share the same frame header and the same wireless transmission time slot. The hardware logic delay difference of separate processing at the receiving end is less than 1ms. Therefore, the end-to-end audio / video-touch latency difference is stably controlled within 1ms, which is far better than the fluctuation level of 5~10ms of the dual-link solution.

[0034] By nesting touch subframes within the idle time slots of video frames, video data and touch data are transmitted in the same frame. Both are sent synchronously based on the timestamp of the same common frame header. The end-to-end latency difference between audio / video and touch is determined by the intra-frame offset and is constant, fundamentally solving the problem of asynchrony between touch response and screen refresh.

[0035] The cross-frame buffering and resuming mechanism ensures zero-loss transmission in scenarios with dense touch messages. The design of a buffer queue length of no more than 2 frames ensures reliability while avoiding excessive latency.

[0036] Compared to the shortcomings of existing solutions where touch and video are transmitted independently, resulting in uncontrollable latency differences, this algorithm compresses the latency difference between touch and video to the intra-frame offset level, significantly improving the responsiveness and real-time performance of touch operations, and is especially suitable for scenarios with high touch accuracy requirements, such as games and drawing.

[0037] Preferably, the master-slave handover in step S5 and the last-in-first handover in step S6 both adopt a single-SOC multi-state smooth transition dynamic master-slave transmit / receive handover mechanism: Traditional solutions use two different SOCs at the transceiver end, with fixed master and slave identities. They can only transmit video unidirectionally from the master to the slave and cannot share the screen in the reverse direction. Existing single SOC switching patents mostly use simple state machine hard switching, which directly shuts down the transmission path and then opens the reception path. The switching process causes service interruption and does not consider the conflict problem of single SOC sharing hardware resources. In vehicle scenarios, frame loss and touch drop are likely to occur.

[0038] This mechanism sets up four basic working states and one transitional overlapping state within the SOC, constructing a closed-loop handover process of (pre-announcement handshake - overlapping transition - confirmation of effectiveness - abnormal rollback). Combined with shared resource gradient scheduling and dual PLL phase smoothing compensation, it achieves zero service interruption during the handover process. At the same time, it has built-in (last-in priority) triggering logic to adapt to dual-screen independent signal access scenarios. The four basic working states include: Standby state S0: No valid signal input is present, and the SOC is in low-power standby mode. In the sending state S1, the encoded local video data is wirelessly uploaded, and the touch message feedback from the other end is received simultaneously. In the receiving state S2, the decoded video data from the other end is output and displayed, and at the same time, the local touch message is sent back to the sending end through the TDD reverse time slot. In hybrid mode S3, time-division multiplexing is used within a single frame period. The first half receives video and the second half sends touch input, supporting two-way interaction in linkage mode. Transitional overlapping state This is used to allow the transmit and receive paths to operate simultaneously during the handover process, sharing hardware resources through gradient bandwidth scheduling. Switching triggers are determined by a comprehensive score, which includes the overall switching trigger score. The calculation formula is: ; in, The score represents the validity of the local input signal; it is 1 when there is a stable and valid signal and -1 when there is no signal. The score for user role requirements is -1 when the peer requests master control and 1 when the local peer requests master control. This represents the local computing power load score; the lower the load, the higher the score, ranging from [-1, 1]. For the corresponding weights, the sum of the weights must be 1; Set the decision threshold ,when And when not in the sending state, a switch to the sending master state is triggered; when Furthermore, when not in the receiving state, a switch to the receiving slave state is triggered; the current state is maintained during the intermediate interval to suppress ping-pong switching; Before the handover is triggered, the local SOC sends a handover warning packet with CRC check. Only after receiving the ACK response from the other end can it enter the transition overlap state. Before the handover, a snapshot of the full state register is saved. If the handover times out, it immediately rolls back to the original working state and retryes at exponentially increasing intervals.

[0039] By setting four basic operating states and one transitional overlapping state, fine-grained management of the SOC's operating state is achieved. Different states correspond to different transmit and receive modes, enabling the SOC to flexibly switch between low-power standby, transmit, receive, and mixed interaction. The comprehensive scoring decision mechanism takes into account three dimensions: signal validity, user role requirements, and computing load. Through weighting and threshold settings, reasonable switching decisions are achieved, and the hysteresis design in the intermediate interval effectively suppresses the ping-pong switching phenomenon.

[0040] The pre-announcement-response handshake mechanism ensures the reliability and orderliness of the handover, while the full-state register snapshot and timeout rollback mechanism provide comprehensive fault tolerance protection. Compared with the shortcomings of existing solutions, which have a single handover decision and lack fault tolerance mechanisms, this mechanism achieves intelligent, reliable, and uninterrupted master-slave identity handover.

[0041] Preferably, in the transitional overlapping state Inside, set the duration of overlapping windows. ,satisfy: ; in, This refers to the transmission time of the remaining data to be completed in the original transmission path. The time from pre-startup of the receiving path to stable output. Timing redundancy margin reserved for automotive scenarios; Within the overlap window, the transmit path bandwidth decreases linearly with time, while the receive path bandwidth increases linearly with time, and the total bandwidth throughout does not exceed the bandwidth limit of the SOC shared hardware. , respectively represented as: ; Where t is the running time within the overlapping window, and its value ranges from 0 to... .

[0042] By setting the overlap window duration T_overlap and ensuring that it is greater than the sum of the original transmission path's termination time and the receiving path's pre-start time plus timing redundancy margin, it is ensured that the original transmission path's data can be completely terminated and the new receiving path can be stably started during the handover process, without any data loss or interruption window.

[0043] Within the overlap window, the design of linearly decreasing transmit bandwidth and linearly increasing receive bandwidth ensures that the total bandwidth never exceeds the bandwidth limit B_total of the SOC shared hardware, avoiding packet loss or congestion caused by bandwidth overload. Compared to the shortcomings of existing solutions where sudden bandwidth changes during handover cause screen stuttering or glitches, this solution's gradient bandwidth scheduling achieves smooth handover of transmit and receive paths, guaranteeing continuous and smooth visuals during the handover process.

[0044] Preferably, when the master clock reference source changes before and after the switch, a master-slave dual PLL architecture is used for clock phase smoothing compensation. The backup PLL tracks the target clock in advance and measures the initial phase difference. Then, step-type phase compensation is adopted, with a single-step compensation amount. satisfy: ; in, The total number of compensation steps is the maximum clock jitter allowed by the system. The master clock source is switched after the phase converges.

[0045] By employing a dual-PLL architecture with primary and backup PLLs, the backup PLL pre-tracks the target clock and measures the initial phase difference, eliminating the need for re-locking during primary clock source switching and significantly reducing switching time. Stepped phase compensation limits the single-step compensation amount to the system's maximum allowable clock jitter range, avoiding excessive clock jitter caused by a single large phase adjustment and ensuring the continuity and stability of audio and video data during switching. The design of switching the primary clock source only after phase convergence ensures that the clock phase is fully aligned at the moment of switching, eliminating phase jumps. Compared to existing solutions where phase jumps during clock switching cause audio / video desynchronization or data errors, this solution achieves a seamless and smooth transition of clock phase.

[0046] Example of numerical substitution calculation: Substitute the parameters for a typical in-vehicle dual-screen scenario: SOC shared hardware total bandwidth It is 1Gbps; The corresponding duration of data to be received in the transmission path before the switchover ms; Receiver path pre-start stabilization time ms; Timing redundancy margin ms; Overlapping window calculation: ; The project rounding setting sets the overlap window duration to 35ms, which is approximately two video frame cycles, thus satisfying the timing constraints.

[0047] Bandwidth allocation for critical nodes within the window: Overlap start time (t=0): Transmit bandwidth 1Gbps, receive bandwidth 0; Window point (t=17.5ms): Transmit and receive bandwidth 500Mbps each; At the end of the window (t=35ms): the sending bandwidth is 0, the receiving bandwidth is 1Gbps, and the total bandwidth is always 1Gbps throughout the process, with no hardware resource conflicts or data packet loss.

[0048] Clock phase compensation calculation: Initial phase difference before and after switching ns, the maximum clock jitter allowed by the system. If the single-step compensation amount is taken as the maximum value of 0.5ns, then the total number of compensation steps is: ; Each compensation step has a 1ms interval, and phase convergence can be completed within 20ms. The entire process can be completed within a 35ms overlap window, with a clock jitter of no more than 0.5ns and an audio / video synchronization error fluctuation of less than 1.5ms.

[0049] Total switching time verification: The interaction time for the preview package is about 1ms, the overlap transition is 35ms, the confirmation of effectiveness is about 1ms, and the total switching time is about 37ms, which is slightly longer than 2 video frame cycles. Moreover, the data plane is continuously transmitted during the overlap period, with no service interruption gaps. Users do not experience any lag or touch loss, which is far superior to the 50~100ms interruption time of the hard handover solution.

[0050] Preferably, before performing step S1, the wireless image transmission SOC unit adopts a dual-layer anti-crosstalk pairing algorithm of RSSI dynamic threshold and packet ID. The device will only enter the pairing process if the received broadcast packet signal strength meets the following conditions: ; in, The actual signal strength of the received broadcast packet, expressed in dBm; The default threshold for the enclosed space in a vehicle is -65dBm. For congestion dynamic adjustment, to meet C represents the congestion level of Wi-Fi data packets in the channel within 1 second, with a value ranging from 0 to 100. After pairing is complete, the two ends exchange a unique encrypted packet ID. All subsequent data packets will carry this ID in their headers. The receiving end will only process the data after verifying that the IDs match.

[0051] By employing a dual-layer anti-crosstalk pairing algorithm combining RSSI dynamic thresholding and packet ID, precise pairing and anti-crosstalk between devices are achieved at both the physical and data link layers. RSSI dynamic thresholding leverages the signal attenuation characteristics of the enclosed space of an in-vehicle environment, combined with dynamic adjustment based on congestion levels, to effectively filter broadcast packet interference from adjacent vehicles or other wireless devices within the vehicle. The packet ID mechanism assigns a unique encrypted identifier to each end after pairing, ensuring that the receiving end only processes data packets carrying the matching ID during subsequent communication, completely eliminating the risk of misprocessing crosstalk data. Compared to existing solutions where fixed threshold pairing is prone to mispairing in congested environments or data crosstalk due to lack of ID verification, this solution achieves highly reliable device pairing and data isolation.

[0052] Example of numerical substitution calculation: When other Wi-Fi devices are present near the carriage and the channel congestion level C=30, the dynamic adjustment amount is: ; The entry threshold for matching has been raised to: ; Only two devices in close proximity within the same group can meet the signal strength requirement. Devices in adjacent seats or adjacent vehicles have signal strengths below the threshold and cannot enter the pairing process. With secondary verification of the group ID, the probability of crosstalk is reduced to less than one in a million.

[0053] Preferably, the image preprocessing in step S1 includes generating rotation parameters, and the FPGA unit in step S4 adopts a screen rotation touch coordinate matrix correction algorithm. After the screen is physically rotated by an angle θ, the original acquired coordinates Target coordinates corrected to fit the image The transformation formula is: ; in is the radian value of the physical rotation of the screen; k is the screen scaling factor, which is equal to the ratio of the output resolution to the original input resolution; W and H are the horizontal and vertical physical resolutions of the touch screen, respectively.

[0054] By employing a screen rotation touch coordinate matrix correction algorithm, the original touch coordinates after physical rotation are accurately mapped to the target coordinates of the adapted screen, resolving the issue of misalignment between touch position and displayed content after screen rotation. The coordinate transformation uses the screen center as the rotation reference point, combined with the screen scaling factor k, simultaneously adapting to both rotation and scaling screen transformation scenarios. Compared to existing solutions that either do not correct touch coordinates after screen rotation or only perform simple translation corrections, leading to touch deviations, this algorithm achieves precise touch coordinate correction under combined rotation and scaling transformations, ensuring the accuracy of touch operation positions.

[0055] Example of numerical substitution calculation: The screen's physical resolution is W=1920, H=1080. When projecting in portrait mode, the screen rotates 90 degrees physically, corresponding to the following radian. Portrait mode images are scaled proportionally to fit the screen; scaling factor. Get the original touch point coordinates .

[0056] Step 1: Center translation ; The second step is matrix rotation, substituting... / 2 , / 2 : ; Third step: Scaling correction ; Step 4: Move back to the center of the screen. ; The final corrected coordinates are (1564.5, 540), which perfectly match the pixel positions of the rotated screen, completely eliminating touch offset.

[0057] Preferably, in step S4, when there are concurrent touch commands from both ends, the MCU unit adopts a multi-touch command weighted arbitration algorithm. The priority score S for a single touch command is: ; in The score is normalized to timestamps; the earlier the instruction arrives, the higher the score, with different weights. Prioritize low latency; The probability of touch trajectory continuation is calculated based on the motion trend prediction of the touch points in the previous 5 frames, with a value ranging from 0 to 1, and a weight. ; The arbitration rules are as follows: only the instruction with the highest score is sent to the signal source at any given time; the instruction with the second highest score is buffered for a maximum of 2 frame cycles, and if there is an idle time slot during the buffering period, it will be sent again.

[0058] A weighted arbitration algorithm for multi-touch commands is used to achieve reasonable ordering and selective delivery of commands during concurrent dual-end touch. The timestamp-normalized score T_s ensures that commands arriving first are processed first, reflecting time fairness. The touch trajectory continuation probability P_pred predicts the continuation probability of the current command based on historical touch point movement trends, ensuring that continuous operations such as swiping and dragging are not interrupted. The weighting ratios α=0.7 and β=0.3 ensure both low latency priority and operational continuity. The design of buffering the second-highest-scoring command for 2 frames allows for the retransmission of delayed commands during idle time slots, preventing command loss. Compared to existing solutions where direct collisions between dual-end touches lead to unresponsive or random signal sources, this algorithm achieves orderly, fair, and continuous multi-end touch arbitration.

[0059] Example of numerical substitution calculation: Both ends simultaneously report two touch commands: Instruction 1: Earlier arrival time, timestamp score. This is a single-point click, with no continuous trajectory, and the probability of trajectory continuation is [not specified]. ; Instruction 2: Arrival time is slightly late, timestamp score. For continuous sliding operations, the probability of trajectory continuation is... ; Substitute into the formula to calculate the score: Score for Instruction 1: ; Score for Instruction 2: ; Ultimately, instruction 2, with a higher score, is sent first; instruction 1, with the second highest score, enters the cache queue and is retransmitted during the idle time slot of the next frame. This ensures the continuity of the swipe operation trajectory and prevents the loss of click instructions, thus improving the smoothness of dual-device touch interaction in multiplayer game scenarios.

[0060] Preferably, in step S4, when the input signal is the original vehicle head unit HDMI / DP signal, the touch reverse control path is locked to prevent cross-screen touch control of the vehicle head unit. Through a signal source type recognition mechanism, the system can automatically distinguish between the original vehicle infotainment system's HDMI / DP signals and portable device Type-C signals. When a signal is input from the vehicle infotainment system, it automatically locks the touch-controlled reverse path, preventing cross-screen touch input from returning to the vehicle infotainment system. This fundamentally eliminates the risk of rear-seat passengers accidentally operating the system and affecting driving safety. Simultaneously, when a signal is input from a portable device, the cross-screen touch control function remains enabled, ensuring a complete user experience.

[0061] It also includes manual forced switching rules: users can manually lock the linkage display mode or the independent working mode by pressing a button. The manual command has higher priority than all automatic polling and last-in priority logic.

[0062] The manual forced switching rule grants users the highest priority control over the working mode. Users can lock the current mode with a single button press, blocking all interference from automatic polling and last-in-first-out logic. This solves the problem of inconvenient intervention when automatic logic misjudges or does not conform to user intent. Compared to the shortcomings of existing solutions that unilaterally open or close anti-control and lack manual intervention methods, this solution achieves a balance between security compliance and user experience, as well as an organic combination of automatic logic and manual control.

[0063] A low-latency in-vehicle rear-seat dual-screen wireless two-way audio and video touch control linkage system is provided for executing the aforementioned low-latency in-vehicle rear-seat dual-screen wireless two-way audio and video touch control linkage method, such as... Figure 2 As shown, it includes a first terminal device (device A) and a second terminal device (device B); the first terminal device and the second terminal device can dynamically act as master and slave transceivers to each other and are connected through a wireless link; Both the first terminal device and the second terminal device include: The multi-input interface module includes HDMI / DP and Type-C interfaces for connecting to the original vehicle system signal or portable device signal; Scaler unit 100 is used for image preprocessing and multi-channel signal switching; FPGA unit 200 is used for screen rotation and touch coordinate correction; Display module 300 and touch module 400; MCU unit 500 is used for touch message processing and command arbitration; Buttons and motor module are used for manual mode switching and physical screen rotation; The Wireless Image Transmission SOC Unit 600 supports TDD (Time Division Duplex) dynamic switching of transmit and receive states for wireless transmission and reception of audio, video, and touch data.

[0064] like Figure 2 As shown, this system includes two terminals (Device A and Device B) with completely symmetrical structures that can dynamically act as master and slave terminals for each other. The connection logic of a single device module is as follows: The multi-input interface module includes HDMI / DP and Type-C input interfaces; the Type-C is equipped with a conversion IC to complete PD power supply negotiation and DP Alt Mode video conversion from HDMI to the Scaler unit; the HDMI / DP directly outputs native video signals to the Scaler, fully compatible with the original vehicle head unit distribution link.

[0065] The Scaler core processing unit (system main controller) has four built-in sub-modules: The image preprocessing sub-module performs black border cropping, resolution scaling, and image rotation parameter preprocessing; the multi-path signal switching matrix controls three types of path switching via IO logic: local input directly connected to local display, local input sent to wireless image transmission SOC for encoding and uploading, and remote wireless decoding signal accessed for local display; the input polling detection sub-module identifies the validity of interface signals and signal source type in real time, automatically switching trigger modes; and the motor drive sub-module outputs drive signals to control the screen rotation motor. The Scaler uses a high-speed video interface... The HDMI and DP interfaces interact with the wireless image transmission SOC to exchange audio and video data streams, and the UART serial port interacts with the SOC to exchange control commands and status information.

[0066] The FPGA screen & touch dual correction unit's video branch receives the pre-processed screen from the Scaler, performs high-precision screen rotation, adapts to multiple interfaces such as LVDS / eDP / MIPI / VBO, and drives the display module; the touch correction branch receives the raw touch coordinates uploaded by the MCU, performs matrix distortion correction based on the screen rotation angle and screen scaling ratio, and outputs accurate coordinates to solve the touch misalignment problem after rotation.

[0067] The display module receives video signals from the FPGA to output the image; the capacitive touch module is attached to the screen and collects the original touch coordinates, which are then uploaded to the MCU via I2C.

[0068] The MCU touch management unit integrates a touch message encapsulation / parsing module and a multi-instruction weighted arbitration module; it interfaces with the touch module and FPGA calibration data, converting coordinates into USB HID standard messages to achieve local direct transmission or remote wireless encapsulation and uploading; it performs priority arbitration for concurrent touch commands from both ends to avoid conflicts. The MCU connects to the wireless image transmission SOC via a USB interface to achieve high-speed transparent transmission of touch messages.

[0069] The button and motor modules are connected to the Scaler, supporting manual mode switching and parameter adjustment. Manual commands have higher priority than automatic logic. The motor is controlled by the Scaler, driving the display + touch module to rotate physically, perfectly matching the screen rotation angle.

[0070] The wireless image transmission SOC unit integrates a MAC controller, audio and video codec engine, layered packet / unpacking hardware accelerator, TDD time division duplex timing controller, master and backup dual PLL clock units and master-slave switching controller within a single SOC chip; it integrates a basic RF transceiver core on the chip, and externally connects a WIFI module, high-power PA (power amplifier), LNA (low noise amplifier) ​​and RF switch to connect to the vehicle antenna to achieve long-distance high-reliability transmission.

[0071] Interface connection logic: The SOC receives video data output from the Scaler via the DP video interface and decoded video data returned via HDMI. It interacts with the Scaler via the UART interface to exchange control commands such as register configuration and operating status. It interacts with the MCU via the USB interface to control HID message data. It supports TDD time-division duplex dynamic switching of transmit and receive states. Both ends use the same SOC chip and hardware BOM design, which can flexibly switch master and slave roles, breaking through the limitations of traditional fixed transceiver chips. By integrating the aforementioned low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method into a system composed of a first terminal device and a second terminal device, a complete hardware implementation solution was constructed. Each terminal device is equipped with a multi-input interface module, compatible with both HDMI / DP and Type-C mainstream signal sources, covering two major access scenarios: original vehicle infotainment systems and portable devices.

[0072] Among them, the Scaler unit is responsible for image preprocessing and multi-path switching, the FPGA unit is dedicated to image rotation and touch coordinate correction, the MCU unit focuses on touch message processing and command arbitration, the button and motor module provides manual control and physical rotation functions, and the wireless image transmission SOC unit, as the communication core, supports TDD time division duplex dynamic switching.

[0073] Each functional module has a clear division of labor and works collaboratively to achieve core functions such as low-latency synchronous transmission of audio, video, and touch, uninterrupted master-slave switching, independent operation of multiple sources, safe and compliant signal source identification, and manual forced intervention. Compared with the shortcomings of existing solutions where functional modules are scattered and have low integration, this system achieves a highly integrated dedicated hardware design, providing a complete end-to-end solution for wireless interaction between dual screens in the rear seats of vehicles.

[0074] The specific applicable scenarios and corresponding processing procedures are explained below: Scenario 1: Single-ended HDMI / DP vehicle head unit signal input (retain the original vehicle head unit distribution mode, disable touch control) Taking device A connected to the car's infotainment system via HDMI / DP and device B having no input as an example: 1. The Scaler identifies the vehicle's infotainment system signal, preprocesses and trims the black borders, generates rotation parameters, and sends them to the FPGA; the FPGA completes the screen rotation and simultaneously sends the rotation matrix parameters to the touch correction module.

[0075] 2. The scaler's drive motor physically rotates the screen to match the image angle.

[0076] 3. The Scaler switches the path to send the video to the local wireless SOC through the DP video interface. The SOC enters the S1 transmit state and encapsulates the data according to the intra-frame nested layered packetization rules for wireless transmission.

[0077] 4. Device B's SOC enters the S2 receiving state, and after layered unpacking, the video is transmitted back to the Scaler. Simultaneously, image processing and motor rotation are completed, achieving complete dual-screen display. The Scaler identifies the vehicle's source and locks the touch reverse control path, which complies with the original vehicle's safety specifications.

[0078] 5. If device B is connected to an HDMI / DP signal, the symmetrical reverse process is executed to achieve reverse video distribution.

[0079] Scenario 2: Input on a single-ended Type-C portable device (with full bidirectional cross-screen touch functionality enabled) Taking device A connected to the phone's Type-C port and device B having no input as an example: 1. The video synchronization process is the same as in scenario 1, with complete synchronization of dual-screen images and rotation angles.

[0080] 2. The Scaler enables full-path touch control on the MCU, and the FPGA loads the corresponding rotation coordinate correction matrix.

[0081] 3. Local touch control: After the touch coordinates on the A end are corrected by the FPGA, they are converted into USB HID messages by the MCU and transmitted back to the mobile phone via Type-C to achieve local control.

[0082] 4. Cross-screen touch control: After touch coordinate calibration at the B end, the MCU encapsulates the data into a standard touch message and sends it to the B end SOC via the USB interface. The B end SOC sends the touch message back to the A end in the TDD reverse time slot. The A end SOC embeds the touch subframe in the idle time slot of the next video frame (or receives it in an independent uplink time slot), unpacks it, extracts the touch subframe, sends it to the A end MCU, and sends it back to the mobile phone after weighted arbitration, thus realizing cross-screen touch control.

[0083] 5. Dual-end simultaneous touch: The MCU calculates the priority score of two instructions, only issues the instruction with the highest score, and buffers the instruction with the second highest score for a short time to avoid instruction conflicts.

[0084] Scenario 3: Both ends are connected to independent signal sources (dual-screen independent working mode) 1. Device A connects to the signal first, and the system enters the linkage mode; Device B subsequently detects a valid local input and triggers the (last-in, first-out) logic.

[0085] 2. The B-side Scaler sends a switching command to the local SOC via UART. The SOC first sends a mode switching warning packet with CRC check to the A-side.

[0086] 3. After end A replies with an ACK response, both SOCs synchronously enter the overlapping transition state, and the handover of the transmit and receive paths is completed through gradient bandwidth scheduling, while clock phase smoothing compensation is also completed.

[0087] 4. After the handover is completed, the B-end SOC switches to S1 transmitting mode and closes the wireless receiving channel, while the A-end switches to S2 receiving mode and closes the wireless uploading channel. The two terminals independently complete the screen, touch control, and motor control without interfering with each other.

[0088] 5. If the signal at either end is disconnected, it will automatically switch back to the linkage and sharing mode.

[0089] Scenario 4: Manual priority forced switching Users can force lock the linkage / independent mode by pressing a button, and manual commands have higher priority than all automatic polling logic; at the same time, it supports manual adjustment of RSSI pairing threshold to adapt to different in-vehicle space environments.

[0090] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A low-latency in-vehicle rear-seat dual-screen wireless bidirectional audio and video touch control linkage method, applied to a system including a first terminal device and a second terminal device, wherein the first terminal device and the second terminal device can dynamically act as master and slave transceivers, connected via a wireless link, characterized in that... This includes simultaneous display mode, master-slave switching mode, and independent working mode; The linked display mode includes the following steps: Step S1: In the first terminal device or the second terminal device, the one that first detects a valid input signal is used as the transmitter, and the other is used as the receiver; the scaler unit of the transmitter identifies the input signal, completes the image preprocessing, and sends the preprocessed video data to the local wireless image transmission SOC unit through the DP video interface. In step S2, the wireless image transmission SOC unit of the transmitting end enters the transmitting state, encapsulates the video data into a single packet layered structure, which includes a common frame header, video data, and an idle time slot for reverse carrying of touch subframes, and transmits it wirelessly through the antenna. Step S3: The wireless image transmission SOC unit at the receiving end enters the receiving state, receives and parses the common frame header, and extracts the video data. In step S4, the separated video data is processed by the Scaler unit of the receiving end and then output by the display module. The touch coordinates collected by the touch module of the receiving end are corrected by the FPGA unit and then encapsulated into touch subframes by the MCU unit of the receiving end. These subframes are then filled into the idle time slots of the video frame of the sending end through the reverse link and sent back to the sending end. After the sending end separates the touch subframes, they are sent to the MCU unit of the sending end for parsing and execution, realizing cross-screen touch control. The master-slave switching mode includes the following steps: Step S5: In the linked display mode, when it is necessary to switch master and slave identities, the wireless image transmission SOC units at both ends complete the master-slave switch without service interruption through gradient bandwidth scheduling and clock phase smoothing compensation. The independent working mode includes the following steps: Step S6: In the linked display mode, when the receiving end detects a valid local signal input, it triggers the last-input priority switching process: The scaler unit of the receiving end sends a switching command to the local wireless image transmission SOC via UART. The wireless image transmission SOC sends a mode switching warning packet with CRC check to the sending end. After the sending end replies with an ACK response, the wireless image transmission SOCs at both ends synchronously enter the transition overlap state, complete the handover of the transmit and receive paths and clock phase smoothing compensation, so that both ends can work independently.

2. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, In step S2, the single-packet layered encapsulation structure adopts an audio / video touch frame nested asynchronous layered packaging algorithm. definition: For video frame period, This is the reporting cycle for capacitive touchscreens. The wireless transmission time for a single frame of video data; The remaining idle time slot length after a single frame of video transmission satisfies ; The maximum number of touch messages that can be embedded within a single frame's idle time slot, satisfying the round-down constraint. ; For the common frame header length, The length of a single-frame video payload; The hierarchical data packet corresponding to the kth video frame period The structure is as follows: ； in, The common frame header includes the frame number, layer identifier, video payload length, and number of embedded touch subframes; This is the video encoded data for the k-th frame; The touch message collected by the receiving end and transmitted back to the sending end via the reverse link is nested in the idle time slot of the video frame at the sending end through the reverse time slot of TDD time division duplex. The actual number of touch subframes embedded in this frame, satisfying... ; When the idle time slot of a single frame is insufficient to accommodate the touch message to be sent, a cross-frame buffering and retransmission mechanism is adopted. The touch message fills the idle time slot of the subsequent video frame in the order of arrival, and the buffer queue length does not exceed 2 frames. The receiver parses the layer identifier and length field of the common frame header and performs a hard decision to split the payload into two types based on the offset: 。 3. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, The master-slave handover in step S5 and the last-in-first handover in step S6 both adopt a single-SOC multi-state smooth transition dynamic master-slave transmit / receive handover mechanism: The wireless image transmission SOC unit has four basic operating states and one transitional overlapping state. The four basic operating states include: Standby state S0, where there is no valid signal input and the SOC is in low-power standby mode; Transmit state S1, where local video data is encoded and wirelessly uploaded, while simultaneously receiving touch message feedback from the other end; Receive state S2, where the other end's video data is decoded and output for display, while simultaneously uploading local touch message feedback; Hybrid state S3, where time-division multiplexing occurs within a single frame period, with the first half receiving video and the second half transmitting touch data, supporting bidirectional interaction in linked mode; and Transitional overlapping state. Used for simultaneous operation of the transmit and receive paths during switching; Switching triggers are determined by a comprehensive score, which includes the overall switching trigger score. The calculation formula is: ； in, The score represents the validity of the local input signal; it is 1 when there is a stable and valid signal and -1 when there is no stable and valid signal. The score for user role requirements is -1 when the peer requests master control and 1 when the local peer requests master control. This represents the local computing power load score; the lower the load, the higher the score, ranging from [-1, 1]. For the corresponding weights, the sum of the weights must be 1; Set the decision threshold ,when And when not in the sending state, a switch to the sending master state is triggered; when Furthermore, when not in the receiving state, a switch to the receiving slave state is triggered; the current state is maintained during the intermediate interval to suppress ping-pong switching; Before the handover is triggered, the local SOC sends a handover warning packet with CRC check. Only after receiving the ACK response from the other end can it enter the transition overlap state. Before the handover, a snapshot of the full state register is saved. If the handover times out, it immediately rolls back to the original working state and retryes at exponentially increasing intervals.

4. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 3, characterized in that, In the transitional overlapping state Inside, set the duration of overlapping windows. ,satisfy: ； in, This refers to the transmission time of the remaining data to be completed in the original transmission path. The time from pre-startup of the receiving path to stable output. Timing redundancy margin reserved for automotive scenarios; Within the overlap window, the transmit path bandwidth decreases linearly with time, while the receive path bandwidth increases linearly with time, and the total bandwidth throughout does not exceed the bandwidth limit of the SOC shared hardware. , respectively represented as: ； Where t is the running time within the overlapping window, and its value ranges from 0 to... .

5. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 3, characterized in that, When the master clock reference source changes before and after the switchover, a master-slave dual PLL architecture is used for clock phase smoothing compensation. The backup PLL tracks the target clock in advance and measures the initial phase difference. Then, step-type phase compensation is adopted, with a single-step compensation amount. satisfy: ； in, The total number of compensation steps is the maximum clock jitter allowed by the system. The master clock source is switched after the phase converges.

6. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, Before executing step S1, the wireless image transmission SOC unit adopts a dual-layer anti-crosstalk pairing algorithm of RSSI dynamic threshold and packet ID. The device will only enter the pairing process if the received broadcast packet signal strength meets the following conditions: ； in, The actual signal strength of the received broadcast packet, expressed in dBm; The default threshold for the enclosed space in a vehicle is -65dBm. For congestion dynamic adjustment, to meet C represents the congestion level of Wi-Fi data packets in the channel within 1 second, with a value ranging from 0 to 100. After pairing is complete, the two ends exchange a unique encrypted packet ID. All subsequent data packets will carry this ID in their headers. The receiving end will only process the data after verifying that the IDs match.

7. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, The image preprocessing in step S1 includes generating rotation parameters, and the FPGA unit in step S4 uses a screen rotation touch coordinate matrix correction algorithm. After the screen is physically rotated by an angle θ, the original acquired coordinates Target coordinates corrected to fit the image The transformation formula is: ； in is the radian value of the physical screen rotation; k is the image scaling factor, which is equal to the ratio of the output resolution to the original input resolution; W and H represent the horizontal and vertical physical resolutions of the touchscreen, respectively.

8. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, In step S4, when there are concurrent touch commands from both ends, the MCU unit adopts a multi-touch command weighted arbitration algorithm. The priority score S for a single touch command is: ； in The score is normalized to timestamps; the earlier the instruction arrives, the higher the score, with different weights. Prioritize low latency; The probability of touch trajectory continuation is calculated based on the motion trend prediction of the touch points in the previous 5 frames, with a value ranging from 0 to 1, and a weight. ; The arbitration rules are as follows: only the instruction with the highest score is sent to the signal source at any given time; the instruction with the second highest score is buffered for a maximum of 2 frame cycles, and if there is an idle time slot during the buffering period, it will be sent again.

9. The low-latency in-vehicle rear dual-screen wireless two-way audio and video touch control linkage method according to claim 1, characterized in that, In step S4, when the input signal is the original vehicle HDMI / DP signal, the touch reverse control path is locked to prevent cross-screen touch control of the vehicle. It also includes manual forced switching rules: users can manually lock the linkage display mode or the independent working mode by pressing a button. The manual command has higher priority than all automatic polling and last-in priority logic.

10. A low-latency in-vehicle rear-seat dual-screen wireless bidirectional audio and video touch control linkage system, used to execute the low-latency in-vehicle rear-seat dual-screen wireless bidirectional audio and video touch control linkage method as described in any one of claims 1 to 9, characterized in that, It includes a first terminal device and a second terminal device; the first terminal device and the second terminal device can dynamically act as master and slave transceivers to each other and are connected through a wireless link; Both the first terminal device and the second terminal device include: The multi-input interface module includes HDMI / DP and Type-C interfaces for connecting to the original vehicle system signal or portable device signal; The Scaler unit is used for image preprocessing and multi-channel signal switching; FPGA unit, used for screen rotation and touch coordinate correction; Display module and touch module; The MCU unit is used for touch message processing and command arbitration. Buttons and motor module are used for manual mode switching and physical screen rotation; The wireless image transmission SOC unit supports TDD (Time Division Duplex) dynamic switching of transmit and receive states for wireless transmission and reception of audio, video, and touch data.

Images