Vehicle-mounted motion compensation system and vehicle-mounted motion compensation method

Abstract

A vehicle motion compensation system and method, the system comprising: a hardware repeater connected in series in a video display link, the hardware repeater being arranged on a physical link between a vehicle host and a display panel, the hardware repeater comprising: a bus message hardware listening unit, directly interfacing a vehicle communication bus through hardware logic, real-time analyzing bus signals, extracting key fields of vehicle dynamics data and writing into a register; a hardware mapping and timing synchronization engine, processing the vehicle dynamics data and outputting a target position of a compensation dot array; and a video stream real-time interception and aliasing unit, real-time monitoring a scanning position of a video stream from the vehicle host, determining a pixel region covered by the compensation dot array according to the target position, and when the scanning position falls within the pixel region, mixing pixel data of the compensation dot array pattern with display data of corresponding pixel points in the video stream to modify the display data of the corresponding pixel points in the video stream.

Description

Technical Field

[0001] This invention relates to the field of vehicle display technology, specifically to a vehicle motion compensation system and method, and more particularly to a system and method for achieving vehicle motion visual compensation by hardware interception of the display link. Background Technology

[0002] With the rapid development of autonomous driving technology and in-vehicle infotainment systems, the number of displays in vehicle cabins is increasing daily, including central control screens, passenger entertainment screens, rear entertainment screens, and instrument panels. It has become commonplace for passengers to watch videos, browse the web, or perform other visual tasks on these displays while the vehicle is in motion. However, vehicles inevitably experience acceleration, deceleration, turning, and bumps during operation. When a passenger's visual system perceives a static or relatively static image on the display, while the vestibular system perceives the vehicle's real-time movement, a conflict arises between the two sensory information, potentially triggering motion sickness, manifesting as dizziness, nausea, and other discomfort.

[0003] To alleviate the aforementioned problems, various visual compensation schemes have been proposed in existing technologies. For example, one scheme generates a visually compensated image in real time that reflects the vehicle's real-time motion relative to the road, and then overlays this image onto the display screen of the in-vehicle infotainment system. This allows the user to see their own motion relative to the ground on the screen, thereby reducing the conflict between the visual and vestibular nerves. Bosch has also proposed a related motion sickness mitigation method by determining the vehicle's lateral and / or longitudinal acceleration and adjusting the displayed image accordingly. Apple has proposed an immersive virtual display solution for in-vehicle VR / AR scenarios, which maps the actual movement of the vehicle to visual movement in the virtual environment to maintain consistency in the user's sensory experience. Summary of the Invention

[0004] The problem the invention aims to solve

[0005] However, most of the existing visual compensation solutions mentioned above are implemented based on the application layer or kernel layer of the operating system (OS). The generation and display of their compensation signals depend on the software processing flow of the onboard host—vehicle dynamics data is first transmitted from sensors to the main processor (SoC) via a bus, and then goes through a series of software steps such as operating system scheduling, driver processing, and GPU rendering before finally being displayed on the screen as a compensated image. These software solutions have the following significant drawbacks:

[0006] First, there is the uncertainty of latency and the asynchrony with perception. Due to the complexity of CPU scheduling, GPU rendering pipeline, and operating system interrupt handling, there is a significant and uncontrollable delay between the displacement of the visual compensation point and the actual movement of the vehicle. Because the software processing path involves multiple interrupts, task switching, and memory read / write operations, the latency often reaches tens or even hundreds of milliseconds, and the latency time is not fixed and exhibits jitter. Even slight latency jitter can cause the displacement of the visual compensation point to be out of sync with the inertial motion perceived by the passenger's body, resulting in a significant phase difference between the visual compensation signal and the physical motion. Untimely compensation may actually exacerbate motion sickness symptoms.

[0007] Second, the system suffers from insufficient dependence and reliability. The normal operation of the visual compensation function in the existing solution is highly dependent on the stability of the on-board host system (such as Android / Linux). When the host system experiences excessive load, lag, frame rate fluctuations, black screen, or even system crash, the visual compensation function will fail and will be unable to provide passengers with continuous motion compensation prompts.

[0008] Third, the system consumes a lot of resources. In a high-resolution, multi-screen cockpit environment, real-time rendering of dynamic compensation patterns requires additional CPU / GPU computing power and memory bandwidth. This creates resource competition with the in-vehicle infotainment system, which is simultaneously running multiple tasks such as navigation, voice interaction, and audio-visual entertainment, potentially affecting the overall system performance and user experience.

[0009] Technical means to solve the problem

[0010] To address the issues of latency uncertainty, strong system dependence, and high resource consumption in the existing technologies, this invention provides an in-vehicle motion compensation system based on display link hardware interception. This system utilizes an ASIC hardware repeater connected in series on the video display link between the in-vehicle host and the display panel. This pure hardware circuitry enables bus monitoring of vehicle dynamics data, coordinate mapping, and real-time interception and aliasing of video stream pixels. Thus, without the need for in-vehicle host CPU / GPU involvement or operating system scheduling, a compensation dot matrix synchronized with the vehicle's real-time motion state is superimposed on the display panel.

[0011] The first aspect of the present invention is an in-vehicle motion compensation system based on hardware interception of a display link, comprising: a hardware repeater connected in series in a video display link, the hardware repeater being disposed on the physical link between an in-vehicle host and a display panel, the hardware repeater comprising: a bus message hardware listening unit configured to directly interface with the in-vehicle communication bus through hardware logic, perform real-time parsing of bus signals, and automatically extract and write key fields of the vehicle dynamics data into a register when a message containing vehicle dynamics data appears on the bus; a hardware mapping and timing synchronization engine configured to process the vehicle dynamics data in the register and output the target position of the compensation dot matrix in the screen coordinate system; and a video stream real-time interception and aliasing unit configured to monitor the scanning position of the video stream from the in-vehicle host in real time, determine the pixel area covered by the compensation dot matrix according to the target position, and when the scanning position falls into the pixel area, mix the pixel data of the compensation dot matrix pattern with the display data of the corresponding pixel in the video stream to modify the display data of the corresponding pixel in the video stream.

[0012] In one possible implementation of the first aspect described above, the hardware mapping and timing synchronization engine further includes a digital filtering unit configured to digitally filter the vehicle dynamics data in the register to remove high-frequency noise.

[0013] In one possible implementation of the first aspect described above, the hardware mapping and timing synchronization engine further outputs a transparency parameter, and the video stream real-time interception and aliasing unit performs the mixing using the transparency parameter as a mixing weight.

[0014] In one possible implementation of the first aspect above, the blending is an alpha blend, and the transparency parameter is an alpha value.

[0015] In one possible implementation of the first aspect above, the hardware mapping and timing synchronization engine further includes a coordinate mapping unit. The coordinate mapping unit has a built-in lookup table. The lookup table pre-stores the mapping relationship between different vehicle dynamics data and the target position and transparency parameters of the compensation dot matrix in the screen coordinate system. The coordinate mapping unit is configured to query the lookup table using the processed vehicle dynamics data as an index and output the target position and transparency parameters of the compensation dot matrix.

[0016] In one possible implementation of the first aspect described above, the mapping relationship in the lookup table is obtained through pre-calibration for specific vehicle models and screen sizes.

[0017] In one possible implementation of the first aspect described above, the real-time video stream interception and aliasing unit performs the alpha mixing during the blanking period or line transmission of the video stream.

[0018] In one possible implementation of the first aspect described above, the hardware repeater has a power supply line independent of the vehicle host, and when the vehicle host or the operating system of the vehicle host fails, the hardware repeater independently maintains the display of the compensation dot matrix.

[0019] In one possible implementation of the first aspect above, when the video stream real-time interception and aliasing unit detects that the video stream from the vehicle host or the operating system has been lost or is abnormal due to a malfunction, a background image is automatically generated and the compensation dot matrix pattern is superimposed on the background image.

[0020] In one possible implementation of the first aspect described above, the hardware repeater further includes a configuration interface for receiving configurations from an external device for the lookup table, the message identifier of the vehicle dynamics data, or the filtering parameters of the digital filtering unit.

[0021] A second aspect of the present invention is an in-vehicle motion compensation method, comprising: a bus message hardware listening unit connected in series with a hardware repeater in the video display link, which directly interfaces with the in-vehicle communication bus in hardware logic to perform real-time parsing of bus signals; when a message containing vehicle dynamics data appears on the bus, automatically extracting the key fields of the vehicle dynamics data and writing them into a register; processing the vehicle dynamics data in the register through the hardware mapping and timing synchronization engine of the hardware repeater, and outputting the target position of the compensation dot matrix in the screen coordinate system; and monitoring the scanning position of the video stream from the in-vehicle host in real time through the video stream real-time interception and aliasing unit of the hardware repeater, determining the pixel area covered by the compensation dot matrix according to the target position, and when the scanning position falls into the pixel area, mixing the pixel data of the compensation dot matrix pattern with the display data of the corresponding pixel in the video stream to modify the display data of the corresponding pixel in the video stream.

[0022] In one possible implementation of the second aspect above, the processing of the vehicle dynamics data in the register includes: digitally filtering the vehicle dynamics data in the register through the digital filtering unit of the hardware mapping and timing synchronization engine to remove high-frequency noise.

[0023] In one possible implementation of the second aspect above, the hardware mapping and timing synchronization engine also outputs a transparency parameter, and the video stream real-time interception and aliasing unit performs the mixing using the transparency parameter as a mixing weight.

[0024] In one possible implementation of the second aspect above, the blending is an alpha blending, and the transparency parameter is an alpha value.

[0025] In one possible implementation of the second aspect above, the coordinate mapping unit included in the hardware mapping and timing synchronization engine uses the processed vehicle dynamics data as an index to query a lookup table built into the coordinate mapping unit, and outputs the target position and transparency parameters of the compensation dot matrix in the screen coordinate system corresponding to the vehicle dynamics data. The lookup table pre-stores the mapping relationship between different vehicle dynamics data and the target position and transparency parameters.

[0026] In one possible implementation of the second aspect above, the mapping relationship in the lookup table is obtained through pre-calibration for specific vehicle models and screen sizes.

[0027] In one possible implementation of the second aspect above, the real-time video stream interception and aliasing unit performs the alpha mixing during the blanking period or line transmission of the video stream.

[0028] In one possible implementation of the second aspect described above, when the vehicle-mounted host or the operating system of the vehicle-mounted host fails, the hardware repeater having a power supply line independent of the vehicle-mounted host maintains the display of the compensation dot matrix.

[0029] In one possible implementation of the second aspect above, when the video stream real-time interception and aliasing unit detects that the video stream from the vehicle host or the operating system has been lost or is abnormal due to a malfunction, a background image is automatically generated and the compensation dot matrix pattern is superimposed on the background image.

[0030] In one possible implementation of the second aspect described above, the configuration interface included in the hardware repeater is used to receive configurations from external devices for the lookup table, the message identifier of the vehicle dynamics data, or the filtering parameters of the digital filtering unit.

[0031] The effects of the invention

[0032] This invention utilizes a hardware repeater connected in series in the display link to achieve bus listening, coordinate mapping, and real-time interception and aliasing of video pixels using pure hardware circuitry. It completely bypasses the operating system and CPU / GPU software processing, ensuring that the delay from bus signal to pixel update is determined by hardware gate circuitry. This achieves constant deterministic synchronization of less than 1ms, eliminating phase differences caused by delay jitter in traditional solutions. This ensures precise synchronization between the compensation dot matrix and the vehicle's actual movement, effectively alleviating motion sickness. Simultaneously, the entire process does not consume CPU / GPU computing power or memory bandwidth, incurring no video memory overhead and avoiding resource competition with in-vehicle applications. The hardware repeater is independent of the in-vehicle host, maintaining compensation display independently even in the event of host failure, ensuring compensation continuity and providing functional safety-level redundancy for in-vehicle displays. Furthermore, all functions can be implemented with a single chip connected in series in the display link, eliminating the need for large-scale modifications to existing in-vehicle systems, resulting in low deployment costs and strong adaptability. Attached Figure Description

[0033] Figure 1 This is a structural block diagram of the vehicle motion compensation system of the present invention.

[0034] Figure 2 This is a schematic diagram illustrating the implementation of the vehicle motion compensation system of the present invention on an infotainment screen.

[0035] Figure 3 This is a flowchart illustrating the vehicle-mounted motion compensation method of the present invention. Detailed Implementation

[0036] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. Those skilled in the art should understand that the following embodiments are only for explaining the present invention and should not be construed as limiting the scope of protection of the present invention. All equivalent substitutions or modifications made based on the technical concept of the present invention fall within the scope of protection of the present invention.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0038] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] Implementation Method 1

[0040] like Figure 1As shown, the in-vehicle motion compensation system based on display link hardware interception provided by this invention mainly includes an ASIC hardware repeater connected in series in the video display link. This ASIC hardware repeater is located on the physical link between the in-vehicle host (i.e., the in-vehicle SoC) and the display panel. The in-vehicle host is responsible for generating the original video stream of the in-vehicle infotainment system and outputting the video stream to the display panel through the video display link (such as a MIPI DSI interface or LVDS interface). The ASIC hardware repeater consists of three functional units: a bus message hardware listening unit, a hardware mapping and timing synchronization engine, and a real-time video stream interception and aliasing unit.

[0041] The bus message hardware listening unit directly interfaces with the physical layer of the vehicle communication bus through hardware logic circuits, and performs real-time parsing of bus signals based on hardware protocol parsing circuits. Various vehicle status information is transmitted on the vehicle communication bus (such as CAN bus, CAN-FD bus, or vehicle Ethernet), including but not limited to vehicle dynamics data such as longitudinal acceleration, yaw rate, and steering column angle. When a message containing vehicle dynamics data appears on the bus, this unit automatically extracts the key fields of the vehicle dynamics data and writes them into a dedicated register. The entire process requires no software driver intervention and does not consume the CPU resources of the vehicle's main unit.

[0042] The hardware mapping and timing synchronization engine reads vehicle dynamics data from a dedicated register, performs digital filtering on it, and then uses the processed data as an index to query the built-in lookup table, outputting the target position and transparency parameters of the compensation dot matrix in the screen coordinate system.

[0043] A real-time video stream interception and aliasing unit is connected in series on the video display link between the vehicle-mounted head unit and the display panel. This unit monitors the scanning position of the video stream from the vehicle-mounted head unit in real time and determines the pixel area covered by the compensation dot matrix in the current video frame based on the target position output by the hardware mapping and timing synchronization engine. When the scanning position falls within this pixel area, the unit acquires the pixel data of the corresponding pixel in the pre-stored compensation dot matrix pattern, uses the transparency parameter as the alpha value to perform alpha mixing of this pixel data with the display data of the corresponding pixel in the video stream, thereby modifying the display data of the corresponding pixel in the video stream. The modified video stream is finally output to the display panel for display.

[0044] Figure 2 The effect of implementing the present invention on an actual in-vehicle infotainment screen is shown. For example... Figure 2As shown, the in-vehicle infotainment touchscreen (central control screen) displays navigation or advanced driver assistance systems (ADAS) images. The arrows and path lines in the center of the screen indicate the vehicle's current trajectory or lane-keeping guidance lines. The two knobs below the screen and the central area contain the physical buttons of the center console. On the far left and far right of the screen are arranged rows of dynamic icons composed of diamond / dashed squares. These icons are the specific implementation of the compensation dot matrix. When the vehicle accelerates, decelerates, or turns, these icons, based on calculations from the hardware mapping and timing synchronization engine, produce corresponding displacement, flashing, or edge-moving dynamic effects at the screen edges, thus providing intuitive feedback to the driver or passengers on the vehicle's real-time motion status, achieving the technical goal of alleviating motion sickness.

[0045] The core objective of the system of this invention is to achieve the perception, analysis and visual mapping of the real-time motion state of the vehicle by embedding an independent intermediate processing layer in hardware in the above-mentioned video display link, without the participation of the vehicle host CPU / GPU and without relying on the operation of the operating system. The compensation dot matrix reflecting the motion state is superimposed on the original video stream with extremely low and constant latency, and finally the dynamic compensation prompts that are precisely synchronized with the actual motion of the vehicle are presented on the LCD panel.

[0046] The following details the specific structure and workflow of each part of the system.

[0047] The bus message hardware listening unit is used to implement hardware-level acquisition of vehicle dynamics data. For example... Figure 1 As shown, this unit directly interfaces with the vehicle's underlying bus (such as CAN bus, CAN-FD bus, or vehicle Ethernet) through hardware logic circuits, forming a physical connection with the vehicle communication bus. Unlike existing technologies that rely on operating system drivers or MCU software polling to read bus data, the bus message hardware listening unit of this invention uses a purely hardware approach to implement bus protocol parsing and data extraction, without any software program involvement throughout the entire process.

[0048] Specifically, the bus message hardware listening unit integrates a hard-core protocol parsing circuit (i.e., a hardware protocol parsing state machine). This circuit directly interfaces with the physical layer of the vehicle's underlying bus via hardware logic. Taking the CAN bus as an example, this protocol parsing circuit performs real-time sampling, bit timing synchronization, bit stream decoding, and message frame parsing of the differential signals on the bus according to the CAN protocol specification. When a message arrives on the bus, the protocol parsing circuit directly compares the identifier field of the message frame at the physical layer—this identifier is the message ID, used to distinguish different types of messages on the bus. The hardware logic pre-sets one or more target message IDs to be monitored (such as specific messages containing vehicle dynamics data such as longitudinal acceleration, yaw rate, or steering column angle). These target IDs can be pre-written into the hardware registers through the chip's configuration interface. When a message matching a preset target ID is detected on the bus, the protocol parsing circuit immediately parses the data field of that message. Based on preset data mapping rules (i.e., the start bit, data length, and scaling factor of each physical quantity in the data field), it automatically extracts the original values ​​of key fields such as acceleration and angular velocity. In this process, the identification, filtering, parsing, and field extraction of bus messages are all completed in real-time by hardware logic circuits at the bitstream level of bus signal transmission, without any processing by the software protocol stack.

[0049] The extracted key fields of vehicle dynamics data are immediately written into a dedicated register within the bus message hardware listening unit for subsequent hardware mapping and timing synchronization engine use. This register update operation is directly driven by hardware logic, without triggering any operating system interrupts or requiring CPU involvement in data transfer. From the arrival of the bus message at this unit to the completion of the register update, the entire processing path is hardwired logic, with latency in the microsecond range. Furthermore, because this unit operates completely independently of the onboard host's operating system and driver layer, its operation is unaffected by the onboard host's load, system lag, or crash. Even if the onboard host fails, this unit can still continuously and stably acquire vehicle dynamics data from the bus.

[0050] Furthermore, the bus message hardware listening unit can also include multiple bus interfaces to adapt to the bus configurations of different vehicle models. For example, for vehicles equipped with both CAN bus and automotive Ethernet, this unit can simultaneously listen to relevant messages on both buses and select the data source according to a preset priority strategy. The protocol parsing circuit of this unit can also be configured and adapted for different bus protocols (such as CAN, CAN-FD, automotive Ethernet, etc.).

[0051] The hardware mapping and timing synchronization engine is used to process the transformation from vehicle dynamics data to screen pixel coordinates. For example... Figure 1As shown, the engine's input is connected to a dedicated register inside the bus message hardware listening unit, and its output is connected to the video stream real-time interception and aliasing unit. The engine reads vehicle dynamics data (such as longitudinal acceleration, yaw rate, steering column angle, etc.) extracted by the hardware listening unit from the dedicated register, performs digital filtering and coordinate mapping processing on it, and finally outputs the target position (X, Y) and transparency parameter (Alpha) of the compensation dot matrix in the screen coordinate system, which is then used by the video stream real-time interception and aliasing unit to perform pixel overlay operation.

[0052] Since the raw dynamics data transmitted on the vehicle's underlying bus inevitably contains high-frequency noise components introduced by factors such as road bumps, engine vibration, and electromagnetic interference, directly using this raw data for coordinate mapping would cause high-frequency jitter in the compensation matrix, potentially exacerbating passenger discomfort. Therefore, the hardware mapping and timing synchronization engine of this invention integrates a digital filtering unit. This digital filtering unit, implemented with hardware logic circuitry, performs real-time digital filtering on the raw vehicle dynamics data read from a dedicated register, removing high-frequency noise components and retaining effective signals that accurately reflect the overall motion trend of the vehicle. In a preferred embodiment, the digital filtering unit employs a low-pass filter structure, with its cutoff frequency preset according to the noise characteristics of the vehicle environment and adjustable via the chip's configuration interface to adapt to the vibration characteristics of different vehicle models. In another embodiment, the digital filtering unit can also be implemented using a hardware-based moving average filter or Kalman filter. The digitally filtered vehicle dynamics data is then fed into the coordinate mapping unit for further processing. The entire digital filtering process described above is completed in real time by hardware logic circuits on the data path, without involving the execution of any software algorithms, and therefore does not introduce any additional processing delay.

[0053] The vehicle dynamics data, after digital filtering, is fed into the coordinate mapping unit. The core of this unit is a built-in hardware look-up table (LUT). The look-up table pre-stores the mapping relationships between different vehicle dynamics data and the target positions and transparency parameters of the compensation dot matrix in the screen coordinate system. Specifically, the look-up table uses the processed vehicle dynamics data (such as filtered longitudinal acceleration values, yaw rate values, etc.) as the input index, and the target positions (X and Y coordinates) and transparency parameters (Alpha values) of the compensation dot matrix in the screen coordinate system as output entries. The look-up table is constructed as follows: for a specific vehicle model and screen size, the optimal displacement and transparency of the compensation dot matrix on the screen under different motion states (such as different accelerations, different turning radii, etc.) are obtained through real-vehicle calibration. This calibration data is then stored in the look-up table in tabular form. Since the screen size, resolution, and installation position vary for different vehicle models, the mapping data in the look-up table needs to be individually calibrated for specific vehicle models to ensure optimal compensation results.

[0054] In the actual operation of the coordinate mapping unit, the hardware logic uses the filtered vehicle dynamics data as the address to directly address the lookup table, reading the corresponding target position (X, Y) and transparency parameter (Alpha) from the lookup table. The entire lookup process is completed by the hardware logic within one or several clock cycles, without involving any branch judgments or loop operations, thus exhibiting extremely low determinism and predictable latency. For multi-dimensional input cases (e.g., when longitudinal acceleration and yaw rate are present simultaneously), the lookup table can adopt a multi-dimensional addressing method—encoding multiple dynamic parameters as the input address of the lookup table, directly outputting the corresponding two-dimensional target position and transparency parameter. Furthermore, the mapping relationship stored in the lookup table can be updated online according to actual test results. New calibration data is written to the lookup table storage area through the chip's configuration interface (such as I²C or SPI interface) to adapt to the configuration requirements of different vehicle models or different displays.

[0055] The hardware mapping and timing synchronization engine outputs the target position (X, Y) and transparency parameter (Alpha) obtained from the lookup table to the real-time video stream interception and aliasing unit. The target position (X, Y) defines the anchor point position of the compensation dot matrix in the screen coordinate system (e.g., the top-left vertex or geometric center of the compensation dot matrix pattern), and the transparency parameter (Alpha) defines the weighting coefficient when the compensation dot matrix and the original video stream are alpha-blended. The engine's output is directly connected to the input of the real-time video stream interception and aliasing unit via hardwired logic, without any intermediate buffers or software forwarding. This ensures that the entire processing path from vehicle dynamics data input to compensation parameter output is a deterministic hardwired path, eliminating latency and jitter introduced by software scheduling or bus arbitration.

[0056] As described above, the hardware mapping and timing synchronization engine of this invention eliminates high-frequency noise through a digital filtering unit and achieves rapid mapping from vehicle dynamics data to screen coordinates through a hardware lookup table. The entire process is completed independently by hardware logic circuits without relying on any software program or operating system. From the vehicle dynamics data written to the dedicated register by the bus message hardware listening unit to the engine outputting the target position and transparency parameters, the latency of the entire processing path is determined by the transmission latency of the hardware gate circuits, exhibiting high determinism and predictability, and ensuring a constant low-latency synchronization path from "bus signal state change" to "real-time update of display pixels".

[0057] The real-time video stream interception and aliasing unit is used to implement the direct overlay of compensation pixels onto the original video stream at the physical layer. For example... Figure 1 As shown, this unit is connected in series in the video display link between the vehicle's main unit (i.e., the vehicle's SoC) and the display panel. Its input is connected to the video stream output of the vehicle's main unit, and its output is connected to the driving circuit of the display panel. Another input of this unit is connected to the output of the hardware mapping and timing synchronization engine, receiving the target position (X, Y) and transparency parameter (Alpha) of the compensation dot matrix in the screen coordinate system output by the engine. This unit pre-stores pixel data of the compensation dot matrix pattern (such as icons or halo patterns of specific shapes, colors, and sizes) and monitors the scanning position of the video stream from the vehicle's main unit in real time. When the scanning position enters the pixel area covered by the compensation dot matrix, it performs real-time interception of pixel data and Alpha mixing operation.

[0058] In automotive display systems, the vehicle-mounted host continuously transmits video stream data to the display panel via a video display link (such as a MIPI DSI interface or LVDS interface). The video stream is transmitted in a progressive scan manner, that is, the display data of each pixel in each frame is sent sequentially from left to right and from top to bottom. The video stream typically contains a horizontal sync signal (HSync), a vertical sync signal (VSync), and a data enable signal (DE) to identify which row and column are currently being transmitted and whether the data is valid. The video stream real-time interception and aliasing unit of this invention integrates a scan position monitoring circuit. This circuit directly parses the horizontal sync signal, vertical sync signal, and data enable signal in the video stream through hardware logic, and tracks the screen coordinates (row number and column number) corresponding to the currently transmitted pixel in real time. This monitoring process is entirely completed by hardware logic and does not rely on any software counters or operating system timers, thus not introducing additional delays or jitter.

[0059] As mentioned earlier, the hardware mapping and timing synchronization engine outputs the target position (X, Y) of the compensation dot matrix in the screen coordinate system. This target position is typically the anchor point coordinates of the compensation dot matrix pattern (e.g., the top-left vertex or geometric center of the pattern). Since the compensation dot matrix pattern has a predetermined size (e.g., occupying M rows × N columns of pixels), the real-time video stream interception and aliasing unit needs to determine the complete pixel area covered by the compensation dot matrix in the current video frame based on the target position and the size information of the pre-stored pattern. Specifically, during the field blanking period (VBlank) of each frame of the video stream, the hardware logic inside the unit calculates the boundary of the pixel area covered by the compensation dot matrix based on the currently received target position (X, Y) and the width (W) and height (H) of the pre-stored pattern: the horizontal range is from X to X+W-1, and the vertical range is from Y to Y+H-1. This calculation result is stored in the unit's internal region register for subsequent pixel-by-pixel comparison. When the compensation dot matrix is ​​a multi-dot matrix or multiple independent patterns, the unit can maintain multiple pixel region registers simultaneously to support the simultaneous overlay of multiple compensation dot matrices.

[0060] During the line-scan transmission of the video stream, the scan position monitoring circuit continuously compares the current scan position (current row and column number) with the pixel region boundaries stored in the region register in real time. This comparison operation is completed by a hardware comparator within each pixel clock cycle, without consuming any additional processing time. When it is detected that the current scan position falls within the pixel region covered by the compensation dot matrix (i.e., the current row number is between Y and Y+H-1 and the current column number is between X and X+W-1), the comparison circuit immediately generates a trigger signal to initiate the pixel data interception and mixing operation. It is worth noting that since the video stream is transmitted line by line, each row of pixels covered by the compensation dot matrix is ​​triggered sequentially during the transmission of that row, thereby ensuring that each row of the pattern can be correctly superimposed onto the original video stream.

[0061] When a trigger signal is generated, the real-time video stream interception and aliasing unit performs the following operations: First, based on the relative offset of the current scan position in the compensation dot matrix pattern (i.e., the offset of the current column number relative to X and the offset of the current row number relative to Y), it reads the pixel data (including the values ​​of the R, G, and B color channels) of the corresponding pixel from the pre-stored compensation dot matrix pattern; then, using the transparency parameter (Alpha) output by the hardware mapping and timing synchronization engine as the blending weight, it performs Alpha blending between the original pixel data of the compensation dot matrix pattern and the original display data of the corresponding pixel in the video stream. The mathematical expression for Alpha blending is as follows:

[0062] R_final = R_overlay × Alpha + R_background × (1 - Alpha)

[0063] G_final = G_overlay × Alpha + G_background × (1 - Alpha)

[0064] B_final = B_overlay × Alpha + B_background × (1 - Alpha)

[0065] Where R_overlay / G_overlay / B_overlay are the RGB values ​​of the corresponding pixels in the compensation dot matrix pattern, R_background / G_background / B_background are the RGB values ​​of the corresponding pixels in the original video stream, and Alpha is the transparency parameter (ranging from 0 to 1, or an equivalent integer range of 0 to 255). The mixed RGB values ​​are directly written to the position of the corresponding pixel in the video stream through the unit's output, thereby achieving real-time modification of the display data of that pixel in the original video stream. The above Alpha mixing operation is completed by the unit's internal hardware multiplier, adder, and shift register within one or several pixel clock cycles, without involving any software computation or memory access.

[0066] Since alpha blending operations need to be completed in real time during video stream transmission, and pixel clock frequencies are typically high (e.g., for a 1080p@60Hz display specification, the pixel clock is approximately 148.5MHz), the blending operation must be completed within a very short time to avoid affecting the normal transmission timing of the video stream. To this end, the real-time video stream interception and aliasing unit of this invention fully utilizes the blank interval in the video stream for necessary preparation work, while only performing the most critical blending operations during the line transmission period (Active Period). Specifically, during the field blank interval (VBlank) of each frame or the line blank interval (HBlank) of each line, the unit completes the calculation of pixel region boundaries and updates the region registers; during line transmission, the unit only performs real-time comparison of scan positions and alpha blending operations. This design schedules computationally intensive operations to be performed during the blank interval and time-sensitive operations to be performed during line transmission, minimizing the impact on the video stream transmission timing and ensuring that the superimposed image does not exhibit display anomalies such as tearing, flickering, or delay. Furthermore, since the entire interception and mixing process is completed in real time along the transmission path of the video stream from the vehicle host to the display panel, without involving any frame buffer read / write operations, it does not generate any video memory overhead, nor does it require additional GPU rendering or memory bandwidth.

[0067] Because the real-time video stream interception and aliasing unit is connected in series on the physical link between the vehicle-mounted host and the display panel, and operates independently of the vehicle-mounted host's operating system and power supply system, when the vehicle-mounted host or operating system fails, this unit can still continuously display the compensation dot matrix on the display panel based on the latest target position and transparency parameters output by the hardware mapping and timing synchronization engine—regardless of whether the original video stream is normal at this time. In a preferred embodiment, when the unit detects that a failure of the vehicle-mounted host or operating system has caused the video stream signal from the vehicle-mounted host to be lost or abnormal (e.g., by monitoring the validity of the data enable signal DE or the horizontal synchronization signal HSync), the signal generation circuit inside the unit can automatically generate a frame of solid color background (such as black or gray) and superimpose the compensation dot matrix pattern on the background, thereby ensuring that even if the original video stream is completely interrupted, the compensation prompt signal can still be presented to the passenger. This failure protection mechanism does not rely on the normal operation of any software or operating system, and is completely completed autonomously by the hardware logic, providing functional safety level redundancy for the vehicle display system.

[0068] In a specific application example of this invention, when the vehicle makes a left turn, a yaw rate message reflecting the turning motion appears on the vehicle communication bus. The bus message hardware listening unit captures the message and extracts the yaw rate data. The hardware mapping and timing synchronization engine looks up a table and outputs the target position and transparency parameters of the compensation dot matrix at the right edge of the screen. During the transmission of the next frame of the video stream, the video stream real-time interception and aliasing unit performs alpha blending based on the corresponding pixel area at the right edge of the screen according to the target position, and superimposes the pre-stored arrow-shaped compensation dot matrix pattern onto the original navigation screen in a semi-transparent manner. The dot matrix moves to the right in real time with the change of the turning angular velocity. Throughout the process, the end-to-end latency from the arrival of the bus message to the completion of pixel modification is constant and less than 1 millisecond. The displacement of the compensation dot matrix seen by the passenger on the screen is precisely synchronized with the vehicle movement felt by the body, effectively alleviating motion sickness symptoms.

[0069] In summary, this invention utilizes an ASIC hardware repeater connected in series on the video display link between the vehicle signal source and the display panel to achieve a complete processing link from vehicle dynamics data acquisition and coordinate mapping to real-time video stream pixel interception and aliasing using pure hardware circuitry. The bus message hardware listening unit directly interfaces with the physical layer of the vehicle's underlying bus through hardware logic circuitry. Based on the hardware protocol parsing circuitry, it performs real-time parsing of the bus signals, automatically extracts key fields of the vehicle dynamics data, and writes them to registers. The entire process bypasses the operating system and driver layer of the vehicle's host computer. The hardware mapping and timing synchronization engine performs digital filtering on the vehicle dynamics data in the registers, uses the processed data as an index to query the built-in lookup table, and outputs the target position and transparency parameters of the compensation dot matrix in the screen coordinate system. The real-time video stream interception and aliasing unit is connected in series on the video display link. It monitors the scanning position of the video stream in real time, determines the pixel area covered by the compensation dot matrix based on the target position, and when the scanning position falls within this area, it performs alpha mixing of the pixel data of the pre-stored compensation dot matrix pattern with the display data of the corresponding pixel in the original video stream, using the transparency parameter as the alpha value. This enables real-time modification of the display data of the corresponding pixel in the video stream. The above three functional units are directly connected by hard-wired logic, forming a deterministic synchronous path from changes in bus signal state to updates in display pixels. The entire process does not rely on any software program or operating system.

[0070] The present invention has the following advantages over the prior art:

[0071] First, deterministic low latency enables precise synchronization of visual and motion perception. Existing visual compensation schemes based on the operating system application layer or kernel layer are limited by the complexity of CPU scheduling, GPU rendering pipeline, and operating system interrupt handling, often resulting in latency of tens or even hundreds of milliseconds, and exhibiting uncontrollable jitter. In contrast, the bus message hardware listening unit of this invention directly interfaces with the physical layer of the vehicle's underlying bus through hardware logic circuits. Based on hardware protocol parsing circuits, it performs real-time parsing and field extraction of bus signals. This extraction and writing process completely bypasses the operating system and driver layer of the onboard host, avoiding software steps such as CPU scheduling, interrupt handling, and GPU rendering pipeline. The latency between changes in bus signal state and updates in displayed pixels is determined by the hardware gate circuit transmission delay, achieving a constant deterministic synchronization path of less than 1ms. This deterministic low latency eliminates the phase lag between the visual compensation signal and the actual vehicle movement caused by latency jitter in traditional software solutions, ensuring precise synchronization between the displacement of the compensation dot matrix and the inertial motion perceived by the passenger, effectively alleviating motion sickness symptoms.

[0072] Secondly, it consumes zero CPU / GPU resources, reducing system overhead. Existing software solutions require additional CPU / GPU computing power and memory bandwidth to render dynamic compensation patterns in real time in high-resolution, multi-screen cockpit environments. The compensation pattern generation and overlay logic of this invention is entirely driven by the hardware circuitry of an ASIC hardware repeater, physically isolated from the vehicle's operating system, memory space, and graphics processing unit (GPU). During the real-time scanning process of the video stream transmission, the video stream interception and aliasing unit directly modifies the display data of specific pixels at the physical layer, overlaying the compensation dot matrix onto the original image in real time. The entire process incurs no video memory overhead and requires no additional video rendering. Therefore, this invention does not consume the vehicle's CPU computing power, GPU computing power, or memory bandwidth, and will not compete for resources with in-vehicle applications such as navigation, voice interaction, and audio-visual entertainment in high-resolution, multi-screen cockpit environments.

[0073] Third, high availability and display redundancy protection. Existing software solutions' visual compensation functions heavily rely on the stability of the in-vehicle host system (such as Android / Linux). When the host system experiences excessive load, stuttering, frame rate fluctuations, black screens, or even system crashes, the compensation function will fail. Because the ASIC hardware repeater of this invention operates independently of the in-vehicle host system, has its own power supply line, and does not depend on the stability of the operating system, when a failure of the in-vehicle host causes an anomaly in the original video stream, the ASIC hardware repeater can still independently maintain the display of the compensation prompt signal based on the underlying bus signals. Even if the original video stream is interrupted due to a host failure, the motion compensation dot matrix on the screen can still maintain a constant refresh rate of 60 FPS or higher, ensuring the continuity of the compensation effect and providing functional safety-level redundancy protection for the in-vehicle display system.

[0074] Fourth, the system architecture is simple, deployment cost is low, and adaptability is strong. This invention only requires connecting an ASIC hardware repeater chip in series in the video display link between the vehicle signal source and the display panel to achieve all functions, without requiring large-scale modifications to the existing vehicle host's hardware architecture and software system. This ASIC hardware repeater can reuse similar chips already used in some vehicle models for fault alarm light mapping, further reducing deployment costs. This invention can be widely applied to various passenger vehicles equipped with in-vehicle displays, including central control screens, passenger entertainment screens, rear entertainment screens, instrument panels, and other display terminals, exhibiting good platform versatility.

[0075] Implementation Method 2

[0076] The present invention also provides a vehicle motion compensation method based on display link hardware interception. This method can be implemented using the hardware repeater described in the above system embodiments, but is not limited thereto.

[0077] like Figure 3 As shown, the method mainly includes the following steps:

[0078] Step S1: Hardware listening for bus messages

[0079] The bus message hardware listening unit, located on the physical link between the vehicle's main unit and the display panel, directly interfaces with the vehicle's communication bus via hardware logic to perform real-time parsing of bus signals. When a message containing vehicle dynamics data appears on the bus, the key fields of the vehicle dynamics data are automatically extracted and written to a register. The specific implementation of this step is the same as the operation of the aforementioned bus message hardware listening unit, and will not be repeated here.

[0080] In a preferred embodiment, the vehicle communication bus includes at least one of CAN bus, CAN-FD bus, or vehicle Ethernet; the vehicle dynamics data includes at least one of longitudinal acceleration, yaw rate, and steering column angle.

[0081] Step S2: Data Processing and Coordinate Mapping

[0082] The vehicle dynamics data in the register is processed by the hardware mapping and timing synchronization engine of the hardware repeater, and the target position of the compensation point matrix in the screen coordinate system is output.

[0083] In a preferred embodiment, before querying the lookup table, the digital filtering unit of the hardware mapping and timing synchronization engine digitally filters the vehicle dynamics data in the register to remove high-frequency noise. This digital filtering can be implemented in hardware using low-pass filtering, moving average filtering, or Kalman filtering.

[0084] In a preferred embodiment, the hardware mapping and timing synchronization engine includes a coordinate mapping unit. The processed vehicle dynamics data is used as an index to query a lookup table built into the coordinate mapping unit, outputting the target position and transparency parameters of the compensation dot matrix in the screen coordinate system corresponding to the vehicle dynamics data. The lookup table pre-stores mapping relationships between different vehicle dynamics data and target position and transparency parameters, obtained through pre-calibration for specific vehicle models and screen sizes.

[0085] Step S3: Video stream interception and pixel aliasing

[0086] The video stream interception and aliasing unit of the hardware repeater monitors the scanning position of the video stream from the vehicle host in real time. Based on the target position, it determines the pixel area covered by the compensation dot matrix. When the scanning position falls into the pixel area, it mixes the pixel data of the compensation dot matrix pattern with the display data of the corresponding pixel in the video stream to modify the display data of the corresponding pixel in the video stream.

[0087] In a preferred embodiment, the real-time video stream interception and aliasing unit pre-stores pixel data of a compensation dot matrix pattern. The hardware mapping and timing synchronization engine also outputs a transparency parameter, which the real-time video stream interception and aliasing unit uses as the blending weight to perform alpha blending, with the transparency parameter serving as the alpha value. The real-time video stream interception and aliasing unit performs alpha blending during the blanking period or line transmission of the video stream.

[0088] In a preferred embodiment, when the vehicle-mounted host or operating system malfunctions, a hardware repeater with a power supply line independent of the vehicle-mounted host independently maintains the display of the compensation dot matrix. When the video stream real-time interception and aliasing unit detects that a malfunction in the vehicle-mounted host or its operating system has caused the video stream from the vehicle-mounted host to be lost or abnormal, it automatically generates a background image and overlays the compensation dot matrix pattern on the background image.

[0089] In a preferred embodiment, the configuration interface included in the hardware repeater receives configurations from external devices for lookup tables, message identifiers of vehicle dynamics data, or filtering parameters of digital filtering units.

[0090] It is understood that the embodiments of the present invention described above are merely illustrative and not restrictive. Those skilled in the art, inspired by the inventive concept, can make various modifications and variations to the above embodiments without departing from the spirit of the present invention.

[0091] For example, although the above embodiments describe the implementation of the hardware repeater as an ASIC (Application-Specific Integrated Circuit), the present invention is not limited thereto. The hardware repeater of the present invention can also be implemented using an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or other programmable hardware logic circuits, as long as it can implement bus message listening, coordinate mapping, and video stream pixel interception and aliasing functions in a purely hardware manner. All of the above-described implementations fall within the protection scope of the present invention.

[0092] For example, although the hardware repeater is described as a separate chip in the above embodiments, the present invention is not limited thereto. The functional units of the hardware repeater can also be integrated into the driver chip of the display panel, or into the video output interface chip of the vehicle host, as long as it is connected in series on the video display link between the vehicle signal source and the display panel and the above functions are implemented in hardware.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vehicle motion compensation system based on display link hardware interception, characterized in that, include: A hardware repeater connected in series in the video display link, wherein the hardware repeater is located on the physical link between the vehicle-mounted host and the display panel. The hardware repeater includes: The bus message hardware listening unit is configured to directly interface with the vehicle communication bus through hardware logic, perform real-time parsing of bus signals, and automatically extract the key fields of the vehicle dynamics data and write them into the register when a message containing vehicle dynamics data appears on the bus. A hardware mapping and timing synchronization engine is configured to process the vehicle dynamics data in the registers and output the target position of the compensation point matrix in the screen coordinate system; and The video stream real-time interception and aliasing unit is configured to monitor the scanning position of the video stream from the vehicle host in real time, determine the pixel area covered by the compensation dot matrix based on the target position, and when the scanning position falls within the pixel area, mix the pixel data of the compensation dot matrix pattern with the display data of the corresponding pixel in the video stream to modify the display data of the corresponding pixel in the video stream.

2. The system according to claim 1, characterized in that, The hardware mapping and timing synchronization engine also includes a digital filtering unit configured to digitally filter the vehicle dynamics data in the register to remove high-frequency noise.

3. The system according to claim 1, characterized in that, The hardware mapping and timing synchronization engine also outputs a transparency parameter, and the real-time video stream interception and aliasing unit uses the transparency parameter as the aliasing weight to perform the aliasing.

4. The system according to claim 3, characterized in that, The blending is an alpha blend, and the transparency parameter is used as an alpha value.

5. The system according to claim 3 or 4, characterized in that, The hardware mapping and timing synchronization engine also includes a coordinate mapping unit. The coordinate mapping unit has a built-in lookup table. The lookup table pre-stores the mapping relationship between different vehicle dynamics data and the target position and transparency parameters of the compensation dot matrix in the screen coordinate system. The coordinate mapping unit is configured to use the processed vehicle dynamics data as an index to query the lookup table and output the target position and transparency parameters of the compensation dot matrix.

6. The system according to claim 5, characterized in that, The mapping relationship in the lookup table is obtained through pre-calibration for specific vehicle models and screen sizes.

7. The system according to claim 4, characterized in that, The real-time video stream interception and aliasing unit performs the alpha mixing during the blanking period or line transmission of the video stream.

8. The system according to claim 1, characterized in that, The hardware repeater has a power supply line independent of the vehicle host. When the vehicle host or its operating system fails, the hardware repeater independently maintains the display of the compensation dot matrix.

9. The system according to claim 8, characterized in that, When the video stream real-time interception and aliasing unit detects that the video stream from the vehicle host or the operating system has been lost or is abnormal due to a malfunction, it automatically generates a background image and overlays the compensation dot matrix pattern on the background image.

10. The system according to claim 6, characterized in that, The hardware repeater also includes a configuration interface for receiving configurations from external devices for the lookup table, the message identifier of the vehicle dynamics data, or the filtering parameters of the digital filtering unit.

11. A vehicle motion compensation method based on display link hardware interception, characterized in that, include: The bus message hardware listening unit of the hardware repeater set on the physical link between the vehicle host and the display panel directly connects to the vehicle communication bus with hardware logic, performs real-time analysis of the bus signal, and automatically extracts and writes the key fields of the vehicle dynamics data into the register when a message containing vehicle dynamics data appears on the bus. The vehicle dynamics data in the register is processed by the hardware mapping and timing synchronization engine of the hardware repeater, and the target position of the compensation dot matrix in the screen coordinate system is output. The video stream real-time interception and aliasing unit of the hardware repeater monitors the scanning position of the video stream from the vehicle host in real time, determines the pixel area covered by the compensation dot matrix based on the target position, and when the scanning position falls into the pixel area, mixes the pixel data of the compensation dot matrix pattern with the display data of the corresponding pixel in the video stream to modify the display data of the corresponding pixel in the video stream.

Images