Vehicle-mounted multimedia system supporting multi-screen interaction and seamless content circulation and control method

Abstract

The invention discloses a vehicle-mounted multimedia system supporting multi-screen interaction and content seamless circulation and a control method, and particularly relates to the technical field of vehicle-mounted multimedia. Frame clock signals, rendering time delay, interaction events and content situation characteristics of a main screen, an auxiliary screen and a rear-row display screen are accurately collected; the real-time monitoring and analysis of the time sequence and semantic state of the system are realized, the trend of time sequence-semantic dislocation coupling instability is dynamically evaluated and predicted by applying a drift operator and a semantic deviation operator in combination with a weighting formula, a self-correction mechanism is started based on the instability trend, the screen refresh phase and frame synchronization proportion is dynamically adjusted, and the stability of the system is improved. And on the basis of the recovered historical behavior track and the current state code, a context inheriting semantic path is effectively reconstructed, semantic coherence in the interaction process is ensured, a generated comprehensive control instruction is broadcasted to all screen nodes through a central control bus, and seamless collaboration of content and interaction operation among multiple screens is ensured.

Description

Technical Field

[0001] This invention relates to the field of in-vehicle multimedia technology, and more specifically, to an in-vehicle multimedia system and control method that supports multi-screen interaction and seamless content flow. Background Technology

[0002] As in-vehicle intelligent cockpit systems evolve towards multi-screen collaboration and cross-domain interaction, in-vehicle multimedia control methods that support multi-screen interaction and seamless content flow are gradually becoming one of the core technologies in human-machine interaction design. These systems establish a unified content status management and timing control mechanism among the main screen, secondary screen, instrument panel, and rear entertainment screen to achieve multi-terminal collaborative display and operational continuity of navigation, entertainment, and communication content. However, during multi-screen parallelism, asynchronous refresh, and cross-screen state migration, the internal time synchronization and semantic inheritance of the system often fail to maintain consistency, resulting in a potential phenomenon of "timing-semantic coupling instability."

[0003] Specifically, when there are slight differences in refresh rates, rendering latency, and communication clocks among various displays, the system's time base will drift, causing misalignment between interaction sampling and content presentation on different screens. This time drift not only leads to visual asynchrony of cross-screen content but also directly interferes with the accuracy of state encoding and context inheritance. Since the context inheritance module typically relies on time-series features to reconstruct user intent, once time base drift disrupts the continuity of interaction events, the system may misjudge user behavior during semantic encoding, parsing the originally coherent sequence of operations into multiple independent fragments, thus creating semantic ambiguity. This semantic ambiguity, in turn, exacerbates the accumulation of time drift. When the context state is not correctly restored, the content state reconstruction module will calculate the content refresh rate and target screen synchronization time based on the incorrect semantic state, further misaligning the screen rendering plan with the actual clock. To correct this deviation, the system often triggers repeated synchronization instructions and rendering callbacks, creating a non-linear positive feedback loop between time drift and semantic deviation. This not only causes visual anomalies such as multi-screen flickering and content jumps but also triggers system-level instability due to thread rescheduling and repeated cache reconstruction. Summary of the Invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide an in-vehicle multimedia system and control method that support multi-screen interaction and seamless content flow, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The in-vehicle multimedia system supports multi-screen interaction and seamless content flow, including a data acquisition module, a drift and deviation calculation module, a coupling evaluation module, a self-correction startup module, a context recovery module, and a control command broadcasting module; The data acquisition module is used to collect frame clock signals, rendering latency, interactive events and content status features from the main screen, secondary screen and rear display screen, and to build a temporal and semantic joint monitoring chain to form a unified feature stream in both time and semantic domains. The drift and deviation calculation module is used to apply drift operators and semantic deviation operators to calculate cross-screen temporal drift and contextual ambiguity on a unified feature flow, and form a synergistic index through a weighted formula; The coupling evaluation module is used to define a temporal-semantic coupling evolution equation based on the synergy index and evaluate whether the system exhibits a temporal-semantic misalignment coupling instability trend. The self-correction startup module is used to trigger the time drift self-correction mechanism based on the instability trend of the time-semantic misalignment coupling, and dynamically adjust the screen refresh phase and frame synchronization ratio. The context recovery module is used to recover the context inheritance semantic path by encoding historical behavior trajectories and current states after time drift self-correction. The control command broadcasting module is used to combine the corrected screen refresh phase and context inheritance semantic path to generate comprehensive control commands. These comprehensive control commands are broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

[0006] In a preferred embodiment, drift operators and semantic deviation operators are applied to a unified feature flow to calculate cross-screen temporal drift and contextual ambiguity, and a synergy index is formed through a weighted formula, as follows: The drift operator is used to calculate the clock signal difference between each pair of screens: ,That For cross-screen time drift, and These are the time alignment signals for the source screen and the target screen, respectively. The semantic deviation operator is used to calculate the contextual ambiguity across screens, and the contextual ambiguity is calculated based on the extracted interaction event data and content status features: ,in For contextual ambiguity, For the degree of difference in interactive events; The content difference is used to calculate the content difference between the source screen and the target screen; Cross-screen time drift With contextual ambiguity Weighted merging to form a synergy index ,in As a synergy index, The weighting coefficient for cross-screen time drift. is the weighting coefficient for contextual ambiguity, and and All are greater than 0.

[0007] In a preferred embodiment, a temporal-semantic coupling evolution equation is defined based on the synergy index to evaluate whether the system exhibits a temporal-semantic misalignment coupling instability trend, as follows: By tracking the synergy index The changes over multiple time windows yielded the evolution sequence of the synergy index over time: ,in For discrete time points, This is the synergy index calculated at discrete time points; Analyze the evolution trend of the synergy index and calculate its rate of change over time: ; The temporal rate of change of the synergy index and its historical dependency are incorporated into the temporal-semantic coupled evolution equation, which is modeled as follows: ,in This represents the risk index of temporal-semantic coupling instability. The rate of change of the synergistic index over time; It is the historical dependence coefficient; It is a time delay term, representing the historical cooperative exponential state of the system. It is the perceived latency.

[0008] In a preferred embodiment, the system is evaluated for a temporal-semantic coupling instability trend based on the temporal-semantic coupling evolution equation. Specifically, this is achieved by observing the temporal-semantic coupling instability risk index. Changes in [the system] can be used to determine whether there is a risk of system instability. Time-Semantic Coupling Instability Risk Index Greater than the preset instability threshold If the value is 10, it indicates a tendency for coupling instability, and the current system is judged to have entered the instability risk zone. Time-Semantic Coupling Instability Risk Index Less than or equal to the preset instability threshold When this time, it indicates that the system remains stable.

[0009] In a preferred embodiment, based on the temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio, as follows: When the time drift self-correction mechanism is triggered, the current cross-screen time drift needs to be recalculated. With contextual ambiguity ; Based on the current cross-screen time drift With contextual ambiguity The synchronization adjustment coefficient used to adjust the screen refresh phase and frame synchronization weight is calculated. The first adjustment coefficient is obtained by weighted synthesis of cross-screen time drift and context ambiguity. Second adjustment coefficient ,in and The weighting coefficients for current cross-screen time drift and contextual ambiguity. and The maximum cross-screen time drift and contextual ambiguity allowed by the system; first adjustment factor. Second adjustment coefficient The dynamic adjustment range used to determine the screen refresh phase and frame synchronization weight.

[0010] In a preferred embodiment, according to the first adjustment coefficient Second adjustment coefficient The screen refresh phase and frame synchronization ratio are dynamically adjusted to correct timing and semantic misalignments. The specific adjustments are as follows: Refresh Phase Adjustment: Based on cross-screen time drift and the first adjustment factor Correct the screen refresh phase: ,in For the screen The current refresh phase, This is the adjusted refresh phase; Frame synchronization weight adjustment: based on contextual ambiguity Second adjustment coefficient Adjust the frame synchronization ratio of the screen: ,in For the screen Current frame synchronization ratio This is the adjusted frame synchronization weight.

[0011] In a preferred embodiment, after time drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and the current state, as follows: After time drift self-correction, the historical behavior trajectory is obtained by collecting the timestamps, event types, triggering screen nodes, and corresponding content changes of all interaction events: ,in For historical behavioral trajectory, This refers to the interaction event data at the m-th time point t; Based on historical behavioral patterns Extract historical behavioral semantic mappings ; By encoding the current screen state in real time, the current screen content features are generated. and current interaction events ; semantic mapping through historical behavior and current screen content features This restores the semantic path between the current state and historical behavior.

[0012] In a preferred embodiment, restoring the semantic path between the current state and historical behavior involves calculating the mapping relationship between the current screen state and historical behavior, and restoring the context inheritance semantic path based on this relationship. The specific mapping formula is as follows: ,in To recover the semantic path, For at a certain point in time At any given moment, the characteristics of the content on the screen, This represents the similarity between the current state code and the historical state content, calculated using cosine similarity.

[0013] In a preferred embodiment, the in-vehicle multimedia control method supporting multi-screen interaction and seamless content flow includes the following steps: Collect frame clock signals, rendering latency, interactive events, and content status features from the main screen, secondary screen, and rear display screens; construct a temporal and semantic joint monitoring chain; and form a unified feature stream with both temporal and semantic domains. The drift operator and semantic deviation operator are applied to the unified feature flow to calculate cross-screen time drift and contextual ambiguity, and a synergistic index is formed by weighting formula; Based on the aforementioned synergy index, a temporal-semantic coupling evolution equation is defined to assess whether the system exhibits a temporal-semantic misalignment coupling instability trend. Based on the aforementioned temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio. After time drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and current state. By combining the corrected screen refresh phase and context inheritance semantic path, a comprehensive control command is generated. The comprehensive control command is broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

[0014] The technical effects and advantages of this invention are as follows: 1. This invention solves the time synchronization and semantic inheritance problems existing in multi-screen collaborative systems by constructing a vehicle multimedia control method for multi-screen interaction and seamless content flow, significantly improving the human-computer interaction experience of the in-vehicle intelligent cockpit. By accurately collecting frame clock signals, rendering latency, interaction events, and content status characteristics of the main screen, secondary screen, and rear display screen, it achieves real-time monitoring and analysis of the system's timing and semantic state. By applying drift operators and semantic deviation operators, it calculates cross-screen time drift and contextual ambiguity, and generates a collaborative index using a weighted formula, thereby dynamically evaluating and predicting the trend of timing-semantic misalignment coupling instability. Furthermore, based on the instability trend, it initiates a self-correction mechanism to dynamically adjust... The screen refresh phase and frame synchronization ratio ensure time synchronization and semantic consistency of content display between screens. Based on this, historical behavior trajectories and current state codes are restored to effectively reconstruct the context inheritance semantic path and ensure semantic coherence during interaction. Finally, the generated integrated control commands are broadcast to all screen nodes through the central control bus to ensure seamless collaboration of content and interactive operations between multiple screens. This method not only avoids abnormal phenomena such as visual flickering and content jumping in traditional multi-screen systems, but also greatly improves the stability and responsiveness of the system, enabling users to experience a smoother and more accurate multi-screen collaborative display and operation continuity, and significantly improving the human-computer interaction experience of the in-vehicle multimedia system. Attached Figure Description

[0015] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a flowchart of the system in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the method in Embodiment 2 of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1: Figure 1 The present invention provides an in-vehicle multimedia system that supports multi-screen interaction and seamless content flow, including a data acquisition module, a drift and deviation calculation module, a coupling evaluation module, a self-correction startup module, a context recovery module, and a control command broadcasting module; The data acquisition module is used to collect frame clock signals, rendering latency, interactive events and content status features from the main screen, secondary screen and rear display screen, and to build a temporal and semantic joint monitoring chain to form a unified feature stream in both time and semantic domains. The drift and deviation calculation module is used to apply drift operators and semantic deviation operators to calculate cross-screen temporal drift and contextual ambiguity on a unified feature flow, and form a synergistic index through a weighted formula; The coupling evaluation module is used to define a temporal-semantic coupling evolution equation based on the synergy index and evaluate whether the system exhibits a temporal-semantic misalignment coupling instability trend. The self-correction startup module is used to trigger the time drift self-correction mechanism based on the instability trend of the time-semantic misalignment coupling, and dynamically adjust the screen refresh phase and frame synchronization ratio. The context recovery module is used to recover the context inheritance semantic path by encoding historical behavior trajectories and current states after time drift self-correction. The control command broadcasting module is used to combine the corrected screen refresh phase and context inheritance semantic path to generate comprehensive control commands. These comprehensive control commands are broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

[0018] Collect frame clock signals, rendering latency, interactive events, and content status features from the main screen, secondary screen, and rear display screens; construct a temporal and semantic joint monitoring chain; and form a unified feature stream with both temporal and semantic domains. In this invention, frame clock signals from the main screen, secondary screen, and rear display screen are synchronously acquired, and the frame update time of each screen is recorded to form an independent clock signal sequence. The clock signal represents the frame update cycle of each screen. The rendering engine interface of the display screen is used to obtain the rendering latency of each screen in real time, that is, the time difference from receiving the rendering task to the completion of display. The rendering latency of the main screen, secondary screen, and rear display screen is recorded separately. User interaction events (such as touch, click, swipe, etc.) from the screen and the state characteristics of the content displayed on the screen (such as focus area, changes in displayed content, etc.) are collected through sensors and interaction modules. Interactive events and content status feature data are recorded in real time as a time-series data stream. The interactive events include the timestamps and types of user operation events, and the content status features include the images, text, focus, and other features currently displayed on the screen. The clock signals and rendering latency of the main screen, secondary screen, and rear display screen are synchronized using time alignment algorithms (such as linear interpolation, Kalman filtering, Bretschneider algorithm, etc.), so that the impact of frame update time and rendering latency of each screen is uniformly quantified. The synchronized clock signal sequence, interactive event data, and content status feature data are fused to form a multi-dimensional time-series semantic joint feature stream. This feature stream contains information in the time dimension (frame update and rendering latency) and the semantic dimension (interactive events and content status), and can reflect the status changes of each screen in the system and the real-time feedback of user interaction. The generated time-series semantic joint feature stream is input into the monitoring system to form a time-series semantic joint monitoring chain, which continuously tracks the time synchronization status and semantic consistency of each screen.

[0019] The drift operator and semantic deviation operator are applied to the unified feature flow to calculate cross-screen time drift and contextual ambiguity, and a synergistic index is formed by weighting formula; In this embodiment of the invention, the time drift between the main screen and the secondary screen, and the rear display screen, is calculated based on the collected clock signal data. The drift operator is used to calculate the clock signal difference between each pair of screens. ,in Cross-screen time drift represents the synchronization difference between the main screen and other screens (secondary screens, rear displays) at the time of frame update. and These are the time alignment signals for the source screen and the target screen, respectively. The semantic deviation operator is used to calculate contextual ambiguity across screens; contextual ambiguity refers to whether the user interaction and displayed content are consistent across screens. Contextual ambiguity is calculated based on extracted interaction event data and content contextual features. ,in Contextual ambiguity reflects the degree of ambiguity between cross-screen interaction and displayed content. The larger the value, the more ambiguous the contextual inheritance between screens, and the smaller the value, the higher the contextual consistency. Interaction event difference is used to calculate the difference between interaction events between the source screen and the target screen. It evaluates whether the user's actions on the main screen are correctly inherited to other screens. Content Difference is used to calculate the content difference between the source screen and the target screen. It evaluates the visual consistency between the content of the main screen and the secondary screen and the rear display screen. The purpose of calculating the difference in interaction events is to assess whether user actions on the main screen are correctly inherited by the secondary or rear display screens. This calculation is based on the collected interaction event data stream. (Source screen interaction event data stream) and (Target screen interaction event data stream), the specific steps are as follows: Obtaining interaction event data includes the following key information: Timestamp The time when the user action occurred.

[0020] Event Type For example, clicking, swiping, long pressing, etc.

[0021] Touch coordinates : Screen coordinates when the interactive event occurs.

[0022] To align and match the interaction events between the source and target screens in terms of time, the timestamps must first be ensured. The difference between the source screen and the target screen is less than the set tolerance range. ,in For the source screen timestamp, The timestamp is the target screen timestamp; if the timestamp difference exceeds the tolerance range, the event matching is considered to have failed and cannot be inherited correctly.

[0023] For successfully matched interaction events, calculate the difference between the event type and the touch coordinates. The difference in touch coordinates can be calculated using methods such as edit distance or Hamming distance. It can be calculated using Euclidean distance: ,in For the source screen touch coordinates, The target screen touch coordinates; Interaction event differences It can be weighted by a combination of event type difference and touch coordinate difference: ,in and These are the weighting coefficients for event type difference and touch coordinate difference, respectively. and All are greater than 0; It should be noted that, and The settings should be tailored to the specific circumstances. For example, an expert-empowered approach could be adopted, where experts in relevant fields are invited to determine the pre-defined proportions for each indicator through professional opinion surveys and comprehensive evaluations. and The initial value can be 0.5, 0.5; The purpose of the content difference calculation is to assess the visual consistency between the main screen, secondary screen, and rear display screens, especially whether the content displayed on the screens is consistent. This calculation is based on the collected content behavior characteristics. (Source screen content characteristics) and (Target screen content situation characteristics), the specific steps are as follows: Content context features of the source and target screens are extracted using image processing techniques. These features may include, but are not limited to: Image features: High-level feature vectors of screen content are extracted using a convolutional neural network (CNN).

[0024] Text information: If there is text on the screen, extract the text content and its layout style.

[0025] UI elements: the distribution and layout of graphical user interface elements such as buttons and icons.

[0026] Visual consistency is assessed by calculating the difference between the content posture feature vectors of the source and target screens. The calculation of content difference can be based on the cosine similarity of the feature vectors. ,in This is the cosine similarity calculation function; cosine similarity is used to measure the similarity of two feature vectors in terms of angle, and the closer the value is to 1, the more similar they are. To better integrate the impact of time synchronization and semantic consistency on multi-screen collaboration, a weighted formula is introduced to account for cross-screen time drift. With contextual ambiguity Weighted merging to form a synergy index ,in The collaboration index quantifies the degree of temporal and semantic mismatch in multi-screen collaboration. The weighting coefficient for cross-screen time drift. is the weighting coefficient for contextual ambiguity, and and All are greater than 0; It should be noted that, and The settings should be tailored to the specific circumstances. For example, an expert-empowered approach could be adopted, where experts in relevant fields are invited to determine the pre-defined proportions for each indicator through professional opinion surveys and comprehensive evaluations. and The initial value can be 0.5, 0.5; The coordination index reflects the overall synchronization and semantic consistency among multiple screens in the system. A higher coordination index indicates a larger discrepancy in time and semantics among multiple screens, while a lower coordination index indicates better synchronization and semantic consistency.

[0027] It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here.

[0028] Based on the aforementioned synergy index, a temporal-semantic coupling evolution equation is defined to assess whether the system exhibits a temporal-semantic misalignment coupling instability trend. In this embodiment of the invention, the synergy index is tracked. The changes over multiple time windows yielded the evolution sequence of the synergy index over time: ,in These are discrete time points (such as the update time of each frame or the trigger time of each interactive event). This is the synergy index calculated at discrete time points; Further analysis of the evolution trend of the synergy index, and calculation of the rate of change of the synergy index over time, i.e., the derivative of the synergy index: The derivative reflects the rate of change of the synergy index; the larger the value, the more the temporal and semantic mismatch of the system is intensifying during that time period. The temporal rate of change of the synergy index and its historical dependency are incorporated into the temporal-semantic coupled evolution equation, which is modeled as follows: ,in This represents the temporal-semantic coupling instability risk index, used to assess whether the system exhibits an instability trend of temporal and semantic misalignment. The time rate of change of the coordination index represents the current degree of mismatch in the system; It is the historical dependency coefficient, which represents the degree of influence of the system's historical cooperative states on its current state; It is a time delay term, representing the historical cooperative exponential state of the system ( It is the perceived latency, used to account for the lag effect of time and semantic feedback. It should be noted that, It is used to measure the impact of a system's historical states on its current state. The historical dependency coefficient is set based on the system's dynamic behavior characteristics and experimental data. Generally speaking, It is a positive value when the historical state of the system has a significant impact on the current state. The value will be relatively high. This value is usually adjusted experimentally to ensure the model's stability and sensitivity. How to set it: It can be set based on data analysis and past trends in the synergistic index. The initial value can usually be obtained from experimental data, and then fine-tuned based on actual performance. This represents the time delay from user action to system response, and can also be understood as the delay in signal propagation within the system. In multi-screen collaborative systems, this delay can be caused by various factors such as sensors, network, or screen refresh rates, and can be determined by measuring the system response time. Based on factors such as screen refresh rate, sensor sampling rate, and communication latency, a reasonable delay value can be derived, which can usually be adjusted through testing.

[0029] It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. Based on the temporal-semantic coupling evolution equation, the system is assessed for potential temporal-semantic misalignment coupling instability. Specifically, this is assessed by observing the temporal-semantic coupling instability risk index. Changes in [the system] can be used to determine whether there is a risk of system instability. Time-Semantic Coupling Instability Risk Index Greater than the preset instability threshold When the system is in a state of intensification, it indicates that the temporal and semantic misalignment is worsening and there is a trend of coupling instability. The system is then judged to be entering a risk zone of instability.

[0030] Time-Semantic Coupling Instability Risk Index Less than or equal to the preset instability threshold When the time interval is 1, it indicates that the system remains stable and the temporal and semantic mismatch has not increased significantly.

[0031] The specific threshold setting can be adjusted based on experimental data or historical records.

[0032] Once the risk index of temporal-semantic coupling instability is reached... Greater than the preset instability threshold When this happens, the following operations are triggered: Alarm Trigger: The system generates an instability alarm to notify the control system of potential synchronization and semantic misalignment issues.

[0033] Self-calibration trigger: Activates the time drift self-calibration mechanism to adjust the screen refresh phase and frame synchronization ratio.

[0034] Based on the aforementioned temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio. In this embodiment of the invention, when the time drift self-correction mechanism is triggered, the current cross-screen time drift needs to be recalculated. With contextual ambiguity This allows for dynamic adjustments based on the degree of mismatch; Based on the current cross-screen time drift With contextual ambiguity The synchronization adjustment coefficient used to adjust the screen refresh phase and frame synchronization weight is calculated. The first adjustment coefficient is obtained by weighted synthesis of cross-screen time drift and context ambiguity. Second adjustment coefficient ,in and The weighting coefficients for current cross-screen time drift and contextual ambiguity. and The maximum cross-screen time drift and contextual ambiguity allowed by the system; first adjustment factor. Second adjustment coefficient Used to determine the dynamic adjustment range of screen refresh phase and frame synchronization weight; According to the first adjustment coefficient Second adjustment coefficient The screen refresh phase and frame synchronization ratio are dynamically adjusted to correct timing and semantic misalignments. The specific adjustments are as follows: Refresh Phase Adjustment: Based on cross-screen time drift and the first adjustment factor Correct the screen refresh phase: ,in For the screen The current refresh phase, This is the adjusted refresh phase; Frame synchronization weight adjustment: based on contextual ambiguity Second adjustment coefficient Adjust the frame synchronization ratio of the screen: ,in For the screen Current frame synchronization ratio This is the adjusted frame synchronization weight.

[0035] It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. After time drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and current state. In this embodiment of the invention, after time drift self-correction, the historical behavior trajectory is obtained by collecting the timestamps, event types, triggered screen nodes, and corresponding content changes of all interactive events. ,in For historical behavioral trajectory, The interactive event data at the m-th time point t records data such as timestamp, event type, triggered screen node, and corresponding content changes; Based on historical behavioral patterns Extract historical behavioral semantic mappings This mapping describes the relationship between user actions and content on each screen; By encoding the current screen state in real time, the current screen content features are generated. and current interaction events This can be accomplished using time-series coding methods (such as RNN and LSTM) to ensure that the system can capture the current state of multiple screens; semantic mapping through historical behavior and current screen content features The core of this process is to restore the semantic path between the current state and historical behavior, which involves associating the current screen state with the historical state to ensure semantic inheritance between screens. The process of restoring the semantic path between the current state and historical behavior involves calculating the mapping relationship between the current screen state and historical behavior, and then restoring the context inheritance semantic path based on this relationship. The specific mapping formula is as follows: ,in To recover the semantic path, For at a certain point in time At any given moment, the screen's content features (such as images, text, focus, etc.) are mapped, reflecting the impact of user interaction on the screen state and helping to restore the semantic inheritance between screens. It represents the similarity between the current state encoding and the historical state content, calculated using cosine similarity. This similarity maximization process helps to recover the correct contextual semantic path.

[0036] It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. By combining the corrected screen refresh phase and context inheritance semantic path, a comprehensive control command is generated. The comprehensive control command is broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

[0037] In this embodiment of the invention, a comprehensive control instruction for screen node control is generated based on the corrected screen refresh phase and the context inheritance semantic path. The main contents of this instruction include: screen refresh phase adjustment: specifying a new refresh phase for each screen node. ;Context inheritance content update: restore the semantic path The corresponding content update information is transmitted to each screen. The format of the integrated control command is: ,in For the screen The phase is refreshed after the adjustment. For the screen Contextual inheritance semantic path; For each screen node, generate the corresponding control instructions. All control commands are packaged into a message packet (or broadcast packet) for broadcast to all screen nodes; The packaged control commands are broadcast to all screen nodes via the central control bus to ensure that every screen node in the system can receive the corresponding control commands. The broadcast mechanism uses an efficient broadcast protocol, such as UDP-based broadcast or message queues within a local area network, to ensure that instructions can be quickly transmitted to all screen nodes and avoid delays affecting multi-screen collaboration. The message packet is broadcast to each screen via the central control bus. The content of the instruction will be parsed on each screen node and adjusted synchronously according to the corrected refresh phase and semantic path.

[0038] It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. This invention solves the time synchronization and semantic inheritance problems existing in multi-screen collaborative systems by constructing a vehicle multimedia control method with multi-screen interaction and seamless content flow, significantly improving the human-computer interaction experience of the in-vehicle intelligent cockpit. By accurately collecting frame clock signals, rendering latency, interaction events, and content status characteristics of the main screen, secondary screen, and rear-seat display, it achieves real-time monitoring and analysis of the system's timing and semantic state. By applying drift operators and semantic deviation operators, it calculates cross-screen time drift and contextual ambiguity, and generates a collaborative index using a weighted formula, thereby dynamically evaluating and predicting the trend of timing-semantic misalignment coupling instability. Furthermore, based on the instability trend, it initiates a self-correction mechanism to dynamically adjust the screen... The screen refresh phase and frame synchronization ratio are weighted to ensure time synchronization and semantic consistency of content display between screens. Based on this, historical behavior trajectories and current state codes are restored to effectively reconstruct the context inheritance semantic path and ensure semantic coherence during interaction. Finally, the generated integrated control commands are broadcast to all screen nodes through the central control bus to ensure seamless collaboration of content and interactive operations between multiple screens. This method not only avoids abnormal phenomena such as visual flickering and content jumping in traditional multi-screen systems, but also greatly improves the stability and responsiveness of the system, enabling users to experience a smoother and more accurate multi-screen collaborative display and operation continuity, and significantly improving the human-computer interaction experience of the in-vehicle multimedia system.

[0039] Example 2: This example introduces a vehicle multimedia control method that supports multi-screen interaction and seamless content flow, such as... Figure 2 As shown, it includes the following steps: Collect frame clock signals, rendering latency, interactive events, and content status features from the main screen, secondary screen, and rear display screens; construct a temporal and semantic joint monitoring chain; and form a unified feature stream with both temporal and semantic domains. The drift operator and semantic deviation operator are applied to the unified feature flow to calculate cross-screen time drift and contextual ambiguity, and a synergistic index is formed by weighting formula; Based on the aforementioned synergy index, a temporal-semantic coupling evolution equation is defined to assess whether the system exhibits a temporal-semantic misalignment coupling instability trend. Based on the aforementioned temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio. After time drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and current state. By combining the corrected screen refresh phase and context inheritance semantic path, a comprehensive control command is generated. The comprehensive control command is broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

[0040] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0041] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0042] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0043] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and method described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0044] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways.

[0045] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An in-vehicle multimedia system that supports multi-screen interaction and seamless content transfer, characterized in that: It includes a data acquisition module, a drift and deviation calculation module, a coupling evaluation module, a self-calibration startup module, a context recovery module, and a control command broadcasting module; The data acquisition module is used to collect frame clock signals, rendering latency, interactive events and content status features from the main screen, secondary screen and rear display screen, and to build a temporal and semantic joint monitoring chain to form a unified feature stream in both time and semantic domains. The drift and deviation calculation module is used to apply drift operators and semantic deviation operators to calculate cross-screen temporal drift and contextual ambiguity on a unified feature flow, and form a synergistic index through a weighted formula; The coupling evaluation module is used to define a temporal-semantic coupling evolution equation based on the synergy index and evaluate whether the system exhibits a temporal-semantic misalignment coupling instability trend. The self-correction startup module is used to trigger the time drift self-correction mechanism based on the temporal-semantic misalignment coupling instability trend, and dynamically adjust the screen refresh phase and frame synchronization ratio. The context recovery module is used to recover the context inheritance semantic path by encoding historical behavior trajectories and current states after time drift self-correction. The control command broadcasting module is used to combine the corrected screen refresh phase and context inheritance semantic path to generate comprehensive control commands. These comprehensive control commands are broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

2. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 1, characterized in that: Cross-screen temporal drift and contextual ambiguity are calculated by applying drift and semantic bias operators to a unified feature flow, and a synergy index is formed through a weighted formula, as follows: The drift operator is used to calculate the clock signal difference between each pair of screens: ,in For cross-screen time drift, and These are the time alignment signals for the source screen and the target screen, respectively. The semantic deviation operator is used to calculate the contextual ambiguity across screens, and the contextual ambiguity is calculated based on the extracted interaction event data and content status features: ,in For contextual ambiguity, For the degree of difference in interactive events; The content difference metric is used to calculate the content difference between the source screen and the target screen. Cross-screen time drift With contextual ambiguity Weighted merging to form a synergy index ,in As a synergy index, The weighting coefficient for cross-screen time drift. is the weighting coefficient for contextual ambiguity, and and All are greater than 0.

3. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 2, characterized in that: Based on the aforementioned synergy index, a temporal-semantic coupling evolution equation is defined to evaluate whether the system exhibits a temporal-semantic misalignment coupling instability trend, as detailed below: By tracking the synergy index The changes over multiple time windows yielded the evolution sequence of the synergy index over time: ,in For discrete time points, This is the synergy index calculated at discrete time points; Analyze the evolution trend of the synergy index and calculate its rate of change over time: ; The temporal rate of change of the synergy index and its historical dependency are incorporated into the temporal-semantic coupled evolution equation, which is modeled as follows: ,in This represents the risk index of temporal-semantic coupling instability. The rate of change of the synergistic index over time; It is the historical dependence coefficient; It is a time delay term, representing the historical cooperative exponential state of the system. It is the perceived latency.

4. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 3, characterized in that: Based on the temporal-semantic coupling evolution equation, the system is assessed for potential temporal-semantic misalignment coupling instability. Specifically, this is assessed by observing the temporal-semantic coupling instability risk index. Changes in [the system] can be used to determine whether there is a risk of system instability. Time-Semantic Coupling Instability Risk Index Greater than the preset instability threshold If the value is 0, it indicates a tendency for coupling instability, and the current system is judged to have entered the instability risk zone; Time-Semantic Coupling Instability Risk Index Less than or equal to the preset instability threshold When this time, it indicates that the system remains stable.

5. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 4, characterized in that: Based on the aforementioned temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio, as follows: When the time drift self-correction mechanism is triggered, the current cross-screen time drift needs to be recalculated. With contextual ambiguity ; Based on the current cross-screen time drift With contextual ambiguity The synchronization adjustment coefficient used to adjust the screen refresh phase and frame synchronization weight is calculated. The first adjustment coefficient is obtained by weighted synthesis of cross-screen time drift and context ambiguity. Second adjustment coefficient ,in and These are the weighting coefficients for the current cross-screen time drift and contextual ambiguity. and The maximum cross-screen time drift and contextual ambiguity allowed by the system; first adjustment factor. Second adjustment coefficient The dynamic adjustment range used to determine the screen refresh phase and frame synchronization weight.

6. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 5, characterized in that: According to the first adjustment coefficient Second adjustment coefficient The screen refresh phase and frame synchronization ratio are dynamically adjusted to correct timing and semantic misalignments. The specific adjustments are as follows: Refresh Phase Adjustment: Based on cross-screen time drift and the first adjustment factor Correct the screen refresh phase: ,in For the screen The current refresh phase, This is the adjusted refresh phase; Frame synchronization weight adjustment: based on contextual ambiguity Second adjustment coefficient Adjust the frame synchronization ratio of the screen: ,in For the screen Current frame synchronization ratio This is the adjusted frame synchronization weight.

7. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 6, characterized in that: After time-drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and the current state, as follows: After time drift self-correction, the historical behavior trajectory is obtained by collecting the timestamps, event types, triggering screen nodes, and corresponding content changes of all interaction events: ,in For historical behavioral trajectory, This refers to the interaction event data at the m-th time point t; Based on historical behavioral patterns Extract historical behavioral semantic mappings ; By encoding the current screen state in real time, the current screen content features are generated. and current interaction events ; semantic mapping through historical behavior and current screen content features This restores the semantic path between the current state and historical behavior.

8. The in-vehicle multimedia system supporting multi-screen interaction and seamless content flow according to claim 7, characterized in that: The process of restoring the semantic path between the current state and historical behavior involves calculating the mapping relationship between the current screen state and historical behavior, and then restoring the context inheritance semantic path based on this relationship. The specific mapping formula is as follows: ,in To recover the semantic path, For at a certain point in time At any given moment, the characteristics of the content on the screen, This represents the similarity between the current state code and the historical state content, calculated using cosine similarity.

9. A vehicle multimedia control method supporting multi-screen interaction and seamless content flow, used to implement the vehicle multimedia system supporting multi-screen interaction and seamless content flow as described in any one of claims 1-8, characterized in that: Includes the following steps: Collect frame clock signals, rendering latency, interactive events, and content status features from the main screen, secondary screen, and rear display screens; construct a temporal and semantic joint monitoring chain; and form a unified feature stream with both temporal and semantic domains. The drift operator and semantic deviation operator are applied to the unified feature flow to calculate cross-screen time drift and contextual ambiguity, and a synergistic index is formed by weighting formula; Based on the aforementioned synergy index, a temporal-semantic coupling evolution equation is defined to assess whether the system exhibits a temporal-semantic misalignment coupling instability trend. Based on the aforementioned temporal-semantic misalignment coupling instability trend, a time drift self-correction mechanism is triggered to dynamically adjust the screen refresh phase and frame synchronization ratio. After time drift self-correction, the context inheritance semantic path is recovered by encoding historical behavior trajectories and current state. By combining the corrected screen refresh phase and context inheritance semantic path, a comprehensive control command is generated. The comprehensive control command is broadcast to all screen nodes via the central control bus to drive multi-screen collaborative display and interaction.

Images