Vehicle-mounted motion sickness visual compensation method, device and system

Abstract

This application provides a method, device, and system for in-vehicle motion sickness visual compensation. The method includes: acquiring vehicle motion data and passenger biometric data; generating a dynamic motion sickness risk index based on the vehicle motion data and passenger biometric data; generating a visual compensation signal based on the dynamic motion sickness risk index, vehicle motion data, and passenger biometric data; and performing multi-screen differentiated visual compensation based on the visual compensation signal. By integrating vehicle motion data and passenger biometric data and designing an adaptive dynamic visual compensation algorithm, this application can effectively solve the shortcomings of existing technologies in cross-domain perception, real-time response, and individual adaptation. It comprehensively and accurately assesses the motion sickness level based on the motion sickness generation mechanism and dynamically adjusts the compensation strategy, achieving precise, dynamic, and personalized intervention for motion sickness in in-vehicle scenarios, and significantly improving passenger comfort.

Description

Technical Field

[0001] This application relates to the field of vehicle intelligent cockpit technology, and in particular to a vehicle-mounted motion sickness visual compensation method, device and system. Background Technology

[0002] In the field of intelligent vehicle cockpit technology, existing motion sickness compensation technologies mainly focus on mitigating passenger motion sickness, such as reducing perceived vibrations by adjusting vehicle suspension or seat damping strategies. These solutions only passively compensate for the vehicle's physical motion and rely on hardware modifications, making them difficult to flexibly adapt to different vehicle models or user needs. Another approach involves detecting passenger motion sickness symptoms through biometrics, such as using seat sensors to collect physiological indicators like heart rate and posture for assessment. However, this approach only reaches the detection level and has a single monitoring dimension, failing to comprehensively reflect the complex physiological mechanisms of motion sickness. Current motion sickness compensation solutions cannot comprehensively and accurately assess the level of motion sickness based on its underlying mechanisms and dynamically adjust compensation strategies accordingly. Summary of the Invention

[0003] The purpose of this application is to provide a method, device and system for visual compensation of motion sickness in vehicles. By integrating vehicle motion data and passenger biometric data and designing an adaptive dynamic visual compensation algorithm, it can effectively solve the shortcomings of existing technologies in cross-domain perception, real-time response and individual adaptation. It can comprehensively and accurately assess the level of motion sickness according to the mechanism of motion sickness and dynamically adjust the compensation strategy, so as to achieve accurate, dynamic and personalized intervention for motion sickness in vehicle scenarios and significantly improve passenger comfort.

[0004] In a first aspect, this application provides a method for in-vehicle motion sickness visual compensation, the method comprising: acquiring vehicle motion data and passenger biometric data; generating a dynamic motion sickness risk index based on the vehicle motion data and passenger biometric data; generating a visual compensation signal based on the dynamic motion sickness risk index, vehicle motion data and passenger biometric data; and performing multi-screen differentiated visual compensation based on the visual compensation signal.

[0005] Furthermore, the steps for generating a dynamic motion sickness risk index based on vehicle motion data and passenger biometric data include: weighted fusion of vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index; or, using vehicle motion parameters and passenger biometric data as observations, estimating the dynamic motion sickness risk index through Kalman filtering.

[0006] Furthermore, the aforementioned vehicle motion data includes: longitudinal acceleration, lateral acceleration, and yaw rate; passenger biometric data includes: head posture angle; the step of weightedly fusing the vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index includes: calculating the square root of the sum of the squares of the longitudinal acceleration and the lateral acceleration to obtain a first motion sickness risk index; calculating the absolute value of the product of the yaw rate and the rate of change of the head posture angle over time to obtain a second motion sickness risk index; and weightedly summing the first motion sickness risk index and the second motion sickness risk index to obtain the dynamic motion sickness risk index.

[0007] Furthermore, the above method also includes: acquiring road condition prediction information; and, based on the road condition prediction information, determining the vehicle movement trend to indicate that a specified road condition will occur within a specified time in the future, generating a visual compensation signal.

[0008] Furthermore, the steps for generating visual compensation signals based on the dynamic motion sickness risk index, vehicle motion data, and passenger biometric data include: determining a compensation mode based on the dynamic motion sickness risk index or passenger biometric data; generating visual compensation signals based on chaos theory and asymmetric phase control based on vehicle motion data and the compensation mode; or generating visual compensation signals based on a deep learning-based dynamic compensation model based on vehicle motion data and passenger biometric data.

[0009] Furthermore, the aforementioned passenger biometric data also includes: target feature parameters; the target feature parameters include: pupil diameter change rate, heart rate change rate, and / or skin conductance intensity; the step of determining the compensation mode based on the dynamic motion sickness risk index or passenger biometric data includes: if the dynamic motion sickness risk index exceeds a first threshold, triggering a basic compensation mode, using two fixed values ​​as two adaptive coefficients; if the dynamic motion sickness risk index exceeds a second threshold, or the target feature parameters exceed a specified threshold, triggering a compensation intensity multiplication mode, increasing the two adaptive coefficients by a specified percentage respectively; the second threshold is greater than the first threshold; the increase percentage of the adaptive coefficient corresponding to the vehicle's comprehensive acceleration is higher than the increase percentage of the adaptive coefficient corresponding to the chaotic function value.

[0010] Furthermore, the aforementioned vehicle motion data also includes: vertical acceleration; the step of generating a visual compensation signal based on chaos theory and asymmetric phase control includes: determining the vehicle's comprehensive acceleration based on longitudinal acceleration, lateral acceleration, and vertical acceleration; determining the chaotic function value based on the change value of yaw rate over time; weighting and summing the vehicle's comprehensive acceleration and the chaotic function value with two adaptive coefficients to obtain a visual compensation signal containing screen compensation displacement; the step of generating a visual compensation signal based on a deep learning-based dynamic compensation model includes: inputting vehicle motion data and passenger biometric data into a hybrid model of convolutional neural network and long short-term memory network so that the model outputs a visual compensation signal containing motion parameters of the compensation graphic.

[0011] Furthermore, the above-mentioned steps for multi-screen differentiated visual compensation based on visual compensation signals include: displaying different compensation images on the central control display screen, head-up display screen, and rear entertainment screen respectively, and controlling the motion amplitude of the compensation images according to the visual compensation signals; the compensation images include: dynamic dots, dynamic stripes, dynamic background light, three-dimensional particle field, or dynamic virtual scene.

[0012] Secondly, this application also provides an in-vehicle motion sickness visual compensation device, the device comprising: a data acquisition module for acquiring vehicle motion data and passenger biometric data; a data fusion module for weighted fusion of vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index; a compensation signal generation module for generating a visual compensation signal based on the dynamic motion sickness risk index or passenger biometric data; and a visual compensation module for performing multi-screen differentiated visual compensation based on the visual compensation signal.

[0013] Thirdly, this application also provides an in-vehicle motion sickness visual compensation system, the system comprising: a sensor layer, a data processing and control layer, and an execution and feedback layer; the sensor layer comprising: an inertial measurement unit, a camera mounted on the roof of the vehicle, or further comprising: millimeter-wave radar, an in-vehicle navigation system; the data processing and control layer comprising: an in-vehicle infotainment system, or further comprising: an edge computing device; the data processing and control layer is used to execute the method described in the first aspect; the execution and feedback layer comprises: a central control display screen, a head-up display, and a rear-seat entertainment system; preferably, the system further comprises an extended collaboration layer, comprising: a cloud server, for storing historical data, training and updating advanced algorithm models, and providing support for multi-passenger personalized compensation strategies.

[0014] The in-vehicle motion sickness visual compensation method, device, and system provided in this application first acquire vehicle motion data and passenger biometric data; then, based on the vehicle motion data and passenger biometric data, a dynamic motion sickness risk index is generated; further, based on the dynamic motion sickness risk index, vehicle motion data, and passenger biometric data, a visual compensation signal is generated; finally, multi-screen differentiated visual compensation is performed based on the visual compensation signal. This application, by integrating vehicle motion data and passenger biometric data, determines a dynamic motion sickness risk index that can accurately assess the level of motion sickness. Further, based on this dynamic motion sickness risk index, and combined with vehicle motion data and passenger biometric data, a visual compensation signal is generated. Multi-screen differentiated visual compensation is then performed based on this visual compensation signal, effectively addressing the shortcomings of existing technologies in cross-domain perception, real-time response, and individual adaptation. It comprehensively and accurately assesses the level of motion sickness based on the mechanism of motion sickness and dynamically adjusts the compensation strategy, achieving precise, dynamic, and personalized intervention for motion sickness in in-vehicle scenarios, significantly improving passenger comfort. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 A flowchart of a vehicle motion sickness visual compensation method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of the overall architecture of an in-vehicle motion sickness visual compensation system. Figure 3 A structural block diagram of a vehicle-mounted motion sickness visual compensation device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a vehicle-mounted motion sickness visual compensation system provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Existing technologies have the following key drawbacks in the context of alleviating motion sickness in vehicles: 1. Limitations of single motion perception: Vehicle motion states usually involve complex parameters such as multi-axis vibration (longitudinal, lateral, and vertical acceleration) and yaw rate. However, existing technologies only passively dampen the vibration through the suspension system without combining vehicle sensor data for dynamic visual compensation, which makes it impossible to specifically counteract the core contradiction of vestibular-visual perception conflict.

[0019] 2. Insufficient biometric monitoring dimensions: Triggers of motion sickness in passengers include the mismatch between visual input and the vestibular system. However, existing biometric monitoring technologies rely on only a single indicator (such as heart rate) and do not integrate more direct physiological signals such as head posture and eye movements, making it difficult to accurately assess the level of motion sickness and dynamically adjust compensation strategies.

[0020] 3. Lack of cross-domain data fusion: Although the visual compensation solution for mobile devices has achieved certain results in the consumer electronics field, it is based only on the motion data of the device itself (such as gyroscope), without considering the global nature of vehicle motion (such as navigation traffic prediction, V2X communication data), and without combining passenger biometrics (such as changes in pupil diameter). This results in the compensation logic being out of touch with actual needs and making it difficult to adapt to the complexity and diversity of in-vehicle scenarios.

[0021] 4. Insufficient adaptability of the control algorithm: In existing solutions, the motion patterns of visual compensation graphics are mostly linear or fixed (such as uniform speed movement), and are not adaptively adjusted according to the nonlinear characteristics of vehicle motion (such as sudden changes in angular velocity in continuous curves) or individual differences of passengers (such as motion sickness sensitivity), resulting in the compensation effect being limited to specific working conditions or groups of people.

[0022] The core reason for the above defects is: Technological cognitive bias: Traditional approaches treat motion sickness as a single physical movement problem without delving into the multimodal interaction mechanisms of vestibular-visual-biofeedback; Data source limitations: Data fusion technology between vehicle sensors (such as IMU, CAN bus) and cabin biological monitoring equipment (such as cameras, millimeter-wave radar) has not been fully applied in the field of visual compensation; Algorithm complexity requirements: The dynamic compensation model needs to process multi-source heterogeneous data (such as vehicle motion parameters and passenger physiological signals) in real time and generate reverse visual feedback, which places higher demands on computational efficiency and algorithm robustness. Existing technologies have not yet broken through this bottleneck.

[0023] To address the aforementioned problems, this application provides an in-vehicle motion sickness visual compensation method, device, and system. By integrating vehicle motion data and passenger biometric data, and designing an adaptive dynamic visual compensation algorithm, it effectively solves the shortcomings of existing technologies in cross-domain perception, real-time response, and individual adaptation. To facilitate understanding of this embodiment, a detailed description of the in-vehicle motion sickness visual compensation method disclosed in this application is provided first.

[0024] Figure 1 A flowchart of a vehicle-mounted motion sickness visual compensation method provided in this application embodiment is shown. The method specifically includes the following steps: Step S102: Obtain vehicle motion data and passenger biometric data; The vehicle motion data mentioned above includes: longitudinal acceleration, lateral acceleration, vertical acceleration, and yaw rate; passenger biometric data includes: head posture angle; the passenger biometric data also includes: target feature parameters; target feature parameters include: pupil diameter change rate, heart rate change rate, and / or skin conductance response intensity.

[0025] In practice, longitudinal acceleration can be obtained through the vehicle's CAN bus. ), lateral acceleration ( ), vertical acceleration ( ) and yaw rate ( Motion parameters such as head posture angle are monitored in real time via cockpit cameras. ), Pupil diameter change rate ( Passenger biometric data, such as passenger biometrics.

[0026] Step S104: Generate a dynamic motion sickness risk index based on vehicle motion data and passenger biometric data; In this step, vehicle motion data and passenger biometric data can be weighted and fused to generate a dynamic motion sickness risk index (MSI); alternatively, vehicle motion parameters and passenger biometric data can be used as observations, and the MSI can be estimated using Kalman filtering. The specific implementation process will be detailed later.

[0027] Step S106: Generate a visual compensation signal based on the dynamic motion sickness risk index, vehicle motion data, and passenger biometric data; The compensation mode can be determined based on the dynamic motion sickness risk index or passenger biometric data; then, based on vehicle motion data and the compensation mode, a visual compensation signal is generated using chaos theory and asymmetric phase control; alternatively, a visual compensation signal can be generated based on a deep learning-based dynamic compensation model using vehicle motion data and passenger biometric data. The specific signal generation process will be detailed later.

[0028] Step S108: Perform multi-screen differentiated visual compensation based on the visual compensation signal.

[0029] For example, different compensations are applied to the central control screen, HUD, and rear entertainment screen.

[0030] The in-vehicle motion sickness visual compensation method provided in this application integrates vehicle motion data and passenger biometric data, and designs an adaptive dynamic visual compensation algorithm. This effectively solves the shortcomings of existing technologies in cross-domain perception, real-time response, and individual adaptation. It comprehensively and accurately assesses the motion sickness level based on the motion sickness generation mechanism and dynamically adjusts the compensation strategy, thereby achieving precise, dynamic, and personalized intervention for motion sickness in in-vehicle scenarios and significantly improving passenger comfort.

[0031] This application embodiment also provides another vehicle motion sickness visual compensation method, which is implemented based on the previous embodiment. This embodiment focuses on describing the data acquisition process, the index calculation process, the signal generation process, and the compensation process.

[0032] See Figure 2 The diagram shows the architecture of the vehicle motion sickness visual compensation system. The dynamic visual compensation system in this embodiment is based on the existing smart cockpit hardware architecture of the vehicle and forms a closed-loop control link by integrating sensors, biological monitoring equipment and vehicle system.

[0033] I. The process for collecting vehicle motion data and passenger biometric data is as follows: (1) Vehicle motion data acquisition: The longitudinal acceleration of the vehicle is acquired through an inertial measurement unit (IMU). ), lateral acceleration ( ), vertical acceleration ( ) and yaw rate ( The data fusion processor (labeled 201) of the IMU and the vehicle system is connected via a CAN bus, with a data sampling frequency of 200Hz to ensure real-time performance.

[0034] In a preferred embodiment, an onboard millimeter-wave radar can be used to replace the IMU, extracting vehicle motion parameters through a multi-target tracking algorithm. Millimeter-wave radar has the advantages of strong anti-interference capability and no need for physical installation, making it particularly suitable for motion state perception under complex road conditions. It can solve the problem of IMU accuracy degradation in strong vibration environments, while reducing hardware installation costs (millimeter-wave radar can reuse existing ADAS systems).

[0035] Furthermore, by combining CAN bus data with road condition prediction information from the vehicle navigation system (such as curve radius and gradient changes), compensation signals can be generated in advance by predicting vehicle movement trends (such as angular velocity changes within the next 3 seconds), thereby improving the foresight of dynamic response. Specifically, the method in this embodiment further includes: acquiring road condition prediction information; and, based on the road condition prediction information, determining when a specified road condition will occur within a specified timeframe, generating a visual compensation signal. By using navigation prediction data, the compensation logic is activated before vehicle movement occurs (such as before entering a curve), further reducing passenger perception delay and improving comfort.

[0036] (2) Biometric data collection: The cockpit camera connects to the processor via a USB interface to capture passenger head posture angles. ) and the rate of change in pupil diameter ( The camera has a sampling rate of 60Hz and supports infrared illumination to ensure monitoring accuracy in low-light environments.

[0037] In this embodiment, the vehicle motion data sampling rate is 200Hz, the biometric data sampling rate is 60Hz, and the data fusion algorithm has a running cycle of 50ms to ensure real-time performance.

[0038] II. The calculation process for the dynamic motion sickness risk index is as follows: The first method involves weighted fusion of vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index. Specifically, this includes: calculating the square root of the sum of the squares of longitudinal and lateral accelerations to obtain a first motion sickness risk index; calculating the absolute value of the product of the yaw rate and the rate of change of the head posture angle over time to obtain a second motion sickness risk index; and then weighted summing the first and second motion sickness risk indices to obtain the dynamic motion sickness risk index. The corresponding formula is as follows: ; in, This represents longitudinal acceleration (unit: m / s²). Transverse acceleration (unit: m / s²). The vehicle's yaw rate (unit: rad / s). Passenger head posture angle (unit: rad). This represents the rate of change of head posture angle over time. The weighting coefficients (0.6 and 0.4) are optimized based on the ISO 15007-2 visual comfort standard and experimental data to ensure that the MSI accurately reflects the risk of motion sickness.

[0039] Before performing the calculation using the above formula, a data preprocessing process is also included: processing the vehicle motion data collected by the IMU ( Low-pass filtering (cutoff frequency 50Hz) is applied to eliminate high-frequency noise; biometric data collected by the camera ( Normalization is performed to ensure data dimension consistency.

[0040] The second method: can adjust vehicle motion parameters ) and passenger biometric data ( Using the observed values, a dynamic motion sickness risk index is estimated through Kalman filtering. This method optimizes the data fusion process through state estimation, improving computational efficiency and real-time performance. In vehicle infotainment systems with limited computing resources (such as low-end models), Kalman filtering reduces algorithm complexity while maintaining compensation accuracy and enhancing technical compatibility.

[0041] III. The signal generation process is as follows: The first method: (1) Determine the compensation mode based on the dynamic motion sickness risk index or passenger biometric data; specifically, if the dynamic motion sickness risk index exceeds the first threshold, trigger the basic compensation mode, using two fixed values ​​as two adaptive coefficients; if the dynamic motion sickness risk index exceeds the second threshold, or the target feature parameter exceeds the specified threshold, trigger the compensation intensity multiplication mode, increasing the two adaptive coefficients by a specified proportion respectively; the second threshold is greater than the first threshold; the increase proportion of the adaptive coefficient corresponding to the vehicle's comprehensive acceleration is higher than the increase proportion of the adaptive coefficient corresponding to the chaotic function value. (2) Based on the vehicle motion data and the compensation mode, generate a visual compensation signal based on chaos theory and asymmetric phase control. The corresponding formula is as follows: ; in, This is the composite vector of vehicle acceleration (unit: m / s²). The vehicle's yaw rate (unit: rad / s). For time variables, and The adaptive coefficient (unit: dimensionless) is dynamically optimized using historical motion sickness data of passengers, with an optimization period of 300 seconds. Chaotic functions Based on the Lorenz attractor model, this algorithm simulates nonlinear motion characteristics to match vehicle dynamics. It is suitable for scenarios with complex operating conditions.

[0042] This application embodiment also provides a signal calculation method under simple working conditions (such as uniform linear motion), namely, the above-mentioned vehicle motion data further includes: vertical acceleration; the step of generating a visual compensation signal based on chaos theory and asymmetric phase control includes: determining the vehicle's comprehensive acceleration based on longitudinal acceleration, lateral acceleration, and vertical acceleration; determining the chaotic function value based on the change value of yaw rate over time; and weighting and summing the vehicle's comprehensive acceleration and the chaotic function value with two adaptive coefficients to obtain a visual compensation signal including screen compensation displacement; the corresponding formula is as follows: s=k1 +k2 ( ); in, s represents the screen compensation displacement. To accelerate the vehicle, Let ω be the vehicle's angular velocity. The angle of the passenger's head posture is k1 / k2, which are adaptive coefficients that are dynamically adjusted according to the passenger's motion sickness sensitivity.

[0043] In the two formulas above, the parameters will be adaptively adjusted. The optimization logic for the adaptive coefficients k1 and k2 is as follows: This application embodiment can also use biometric data to correct compensation parameters in real time, forming a closed-loop mechanism of "monitoring-computation-intervention". The specific steps are as follows: Monitoring: The cockpit camera collects the rate of change of passenger pupil diameter (ΔDpupil / Δt) in real time.

[0044] Calculation and correction: When ΔDpupil / Δt > 15% / s, the compensation intensity multiplication mode is triggered, meaning the adaptive coefficients k1 and k2 increase by 20% and 15%, respectively. The corrected compensation parameters are transmitted to the compensation modules on each screen through the vehicle system, updating the motion trajectory and display characteristics of the compensation graphics. Specifically, if the passenger has a high historical sensitivity to motion sickness (e.g., ΔDpupil / Δt > 15% / s), then k1 and k2 increase by 20% and 15%, respectively; if the passenger has a low sensitivity, then k1 and k2 remain at their default values ​​(e.g., k1 = 0.8, k2 = 0.5), ensuring the compensation intensity adapts to individual needs.

[0045] The aforementioned compensation intensity is directly related to MSI (e.g., a multiplication mode is activated when MSI ≥ 0.7). In another implementation, multi-level thresholds can be introduced (e.g., basic compensation is activated when MSI ≥ 0.5, and strong compensation is activated when MSI ≥ 0.8), and the compensation mode can be dynamically switched based on the passenger's real-time physiological signals (e.g., ΔDpupil / Δt) (e.g., switching from low-frequency dots to high-frequency stripes). Through multi-level thresholds and mode switching, the compensation strategy can be made more refined, avoiding over-intervention (e.g., only basic compensation is activated when MSI changes slightly), while increasing the intervention intensity in high-risk scenarios.

[0046] In addition to the aforementioned methods, heart rate sensors and skin conductance sensors can be added. Multimodal data (such as heart rate variability ΔHR / Δt and skin conductance intensity EER) can be used to correct compensation parameters (such as k1 and k2), improving the comprehensiveness of motion sickness level assessment. Through multimodal biometric fusion, early signs of motion sickness (such as heart rate changes) can be detected earlier, enhancing the proactiveness of compensation strategies.

[0047] In another preferred embodiment, tactile feedback and / or auditory feedback can be introduced while controlling the motion amplitude of the compensated image; tactile feedback includes adjustment of seat vibration frequency; auditory feedback includes playing guidance messages. That is, in addition to visual compensation, tactile feedback (such as adjustment of seat vibration frequency) or auditory feedback (such as playing guidance messages) is introduced to reduce passenger motion sickness through multi-channel intervention. For example, when MSI ≥ 0.8, seat vibration (frequency...) is simultaneously activated. ) and visual compensation ( This enables multimodal perceptual alignment. Tactile or verbal feedback provides auxiliary intervention for visual compensation, which is particularly suitable for visually distracted passengers (such as children in the back seat) and enhances the overall intervention effect.

[0048] The second method involves generating visual compensation signals based on a deep learning-based dynamic compensation model, using vehicle motion data and passenger biometric data. For example, vehicle motion data and passenger biometric data can be input into a hybrid model combining a convolutional neural network and a long short-term memory network, so that the model outputs a visual compensation signal containing motion parameters of the compensation graphic.

[0049] In practical implementation, a hybrid model combining Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM) can be used. Inputting multi-source data (ax, ay, az, ωz, θhead, ΔDpupil / Δt), the output is the motion parameters of the compensation graphic (such as displacement ΔS, velocity v, and color temperature L). Model training data includes motion sickness experimental data from different vehicle models and passengers, and transfer learning is used to adapt to different scenarios. Deep learning models can capture the complex correlation between nonlinear motion and biometrics, further enhancing the personalized adaptability of the compensation strategy.

[0050] IV. The compensation process is as follows: Different compensation images are displayed on the central control display, head-up display, and rear entertainment screen respectively, and the motion amplitude of the compensation images is controlled according to the visual compensation signal; the compensation images include: dynamic dots, dynamic stripes, dynamic background light, three-dimensional particle field or dynamic virtual scene.

[0051] The following are some different compensation methods: The first approach: Design differentiated compensation logic for multi-screen systems (central control screen, HUD, rear entertainment screen) in in-vehicle scenarios. The central control screen uses low-frequency, large-amplitude circular dots (e.g., diameter 4.7±0.3mm, speed 0.8·αvehicle). The HUD projects high-brightness dynamic stripes (brightness adaptive 500-1200 nits), synchronized with the vehicle's yaw rate. The rear entertainment screen generates a three-dimensional particle field, which guides visual stability through color temperature changes (such as turning left → warm tone).

[0052] The second type: Central control screen compensation: Displays dynamic dots with low-frequency, large amplitude vibrations, the diameter of which is... (Based on ISO 15007-2 visual comfort standard), the movement speed is... (Unit: mm / s) Ensure that the movement of the dot is synchronized in the opposite direction to the vehicle's acceleration, thereby reducing visual-vestibular conflict.

[0053] Compensation logic: When MSI exceeds the threshold (e.g., MSI≥0.7), the dot starts to move; when MSI≤0.3, the dot stops displaying to avoid excessive intervention.

[0054] HUD compensation: Projects bright dynamic stripes, its brightness (Unit: nit), relative to the absolute value of the vehicle's yaw rate. Positive correlation; the color temperature adjustment range is 3000K-6500K. When turning left, the color temperature shifts towards a warmer tone (e.g., increasing to 5500K), and when turning right, it shifts towards a cooler tone (e.g., decreasing to 4500K), guiding passengers' visual stability through color temperature changes.

[0055] Rear entertainment screen compensation: Generates a three-dimensional particle field, and the particle trajectory simulates the vehicle's vertical acceleration. By adjusting particle density and velocity, the system guides passengers to focus their attention on a relatively stable area. Compensation logic: When the passenger is in the back seat and MSI≥0.5, the particle field starts to be dynamically generated; if the passenger switches to the child safety seat mode, the particle field is automatically turned off.

[0056] The third option is to configure it as follows: Central control screen: Replace dynamic dots with dynamic stripes (such as sine waves), the amplitude of which is related to vehicle acceleration. Related, frequency and yaw rate synchronous; HUD: Replace the bright dynamic stripes with dynamic background light (such as gradient color blocks), and adjust the color temperature range to 2800K-6000K to further reduce visual interference; Rear entertainment screen: Replace the 3D particle field with a dynamic virtual scene (such as a simulated view outside the window), and enhance the visual guidance effect through the combination of particle field and virtual scene.

[0057] By adjusting the type of graphic, it is possible to cater to the visual preferences of different passengers (for example, some passengers are less sensitive to dots, while stripes are more effective). The compensation graphic is displayed in the center of the screen and can also be adjusted to the edge of the passenger's field of vision (such as the lower left corner of the screen). It leverages the peripheral visual sensitivity to dynamic information to reduce interference with main functions (such as navigation and entertainment). By adjusting the display position, the compensation effect can be maintained while minimizing interference with main functions, making it suitable for users sensitive to visual distractions.

[0058] In addition, passengers are allowed to manually adjust the compensation intensity (e.g., select "strong," "medium," or "weak" modes) via voice or touchscreen, or disable the compensation function for specific screens (e.g., only retaining the central control screen compensation). This solution enhances user experience adaptability through user self-control. Interactive control allows passengers to flexibly adjust the intervention intensity according to their own needs, improving user satisfaction.

[0059] The vehicle-mounted motion sickness visual compensation method provided in this application embodiment has the following significant improvements in technical effect compared with the prior art: 1. Solve the problem of the disconnect between multimodal perception and visual compensation. This application embodiment achieves a technological upgrade from physical shock reduction to perception alignment by fusing cross-domain data from vehicle sensors (IMU, CAN bus) and cabin biological monitoring equipment (camera). It directly targets the root cause of vestibular-visual conflict by designing a dynamic compensation model.

[0060] 2. To address the shortcomings of insufficient dimensions in biometric monitoring. This application embodiment integrates head posture angle (θhead) and pupil diameter change rate (… Dpupil By incorporating multidimensional biological characteristics such as t), a more comprehensive assessment system for motion sickness can be constructed, providing a basis for the precise formulation of compensation strategies.

[0061] 3. Adapts to the complexity of vehicle motion, surpassing the limitations of mobile device solutions. This application's embodiments solve the adaptability defects of mobile device solutions in in-vehicle scenarios by designing compensation logic that matches the vehicle's dynamics through global perception of vehicle motion data (such as multi-axis acceleration of continuous curves and undulating road sections).

[0062] 4. Improve the flexibility of compensation strategies through adaptive algorithms. Based on chaos theory and asymmetric phase control, this application presents an embodiment that designs a dynamic compensation model. The compensation intensity is adjusted in real time through adaptive coefficients (k1 / k2) to make the intervention measures more in line with passenger needs.

[0063] 5. Reduce hardware modification costs and enhance technology compatibility. This application's embodiments are centered on the vehicle's infotainment system, requiring no additional hardware modifications. They utilize software algorithms to call upon existing vehicle sensors (such as IMU and cameras) and display devices (central control screen and HUD), significantly lowering the implementation threshold and offering strong compatibility.

[0064] In summary, the embodiments of this application effectively address the shortcomings of existing technologies in terms of perception dimension, compensation logic adaptability, and hardware dependence through multi-source data fusion, biofeedback closed loop, and adaptive algorithm innovation, providing a more accurate, flexible, and low-cost solution for alleviating motion sickness in vehicles.

[0065] Based on the above method embodiments, this application also provides a vehicle-mounted motion sickness visual compensation device, see [link to relevant documentation]. Figure 3 As shown, the device includes: a data acquisition module 32 for acquiring vehicle motion data and passenger biometric data; a data fusion module 34 for weighted fusion of vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index; a compensation signal generation module 36 for generating a visual compensation signal based on the dynamic motion sickness risk index or passenger biometric data; and a visual compensation module 38 for performing multi-screen differentiated visual compensation based on the visual compensation signal.

[0066] The vehicle-mounted motion sickness visual compensation device provided in this application embodiment has the following beneficial effects: 1. Improved compensation accuracy: By using multi-source data fusion (MSI calculation formula) and asymmetric phase control (ΔS model), the motion trajectory of the compensation graphic is aligned with the dynamic characteristics of the vehicle. Experiments show that the incidence of motion sickness can be reduced by more than 40%.

[0067] 2. Enhanced individual adaptability: The dynamic adjustment logic of adaptive coefficients k1 and k2 (optimized based on historical data) enables the compensation intensity to adapt to the motion sickness sensitivity of different passengers. Experimental data shows that the motion sickness relief efficiency is improved by 25% for people with high sensitivity.

[0068] 3. Hardware compatibility optimization: No additional modifications to the suspension or seats are required; the implementation cost is reduced by more than 60% simply by using software algorithms to call upon existing onboard sensors and display devices.

[0069] Furthermore, the aforementioned data fusion module 34 is used to perform weighted fusion of vehicle motion data and passenger biometric data to generate a dynamic motion sickness risk index; or, using vehicle motion parameters and passenger biometric data as observations, the dynamic motion sickness risk index is estimated through Kalman filtering.

[0070] Furthermore, the aforementioned vehicle motion data includes: longitudinal acceleration, lateral acceleration, and yaw rate; passenger biometric data includes: head posture angle; the data fusion module 44 is used to calculate the square root of the sum of the squares of the longitudinal acceleration and the lateral acceleration to obtain the first motion sickness risk index; to calculate the absolute value of the product of the yaw rate and the rate of change of the head posture angle over time to obtain the second motion sickness risk index; and to perform a weighted summation of the first motion sickness risk index and the second motion sickness risk index to obtain the dynamic motion sickness risk index.

[0071] Furthermore, the aforementioned compensation signal generation module 36 is used to acquire road condition prediction information; based on the road condition prediction information, when the vehicle movement trend is determined to be such that a specified road condition occurs within a specified time in the future, a visual compensation signal is generated.

[0072] Furthermore, the aforementioned compensation signal generation module 36 is used to determine the compensation mode based on the dynamic motion sickness risk index or passenger biometric data; generate a visual compensation signal based on chaos theory and asymmetric phase control based on vehicle motion data and the compensation mode; or generate a visual compensation signal based on a deep learning dynamic compensation model based on vehicle motion data and passenger biometric data.

[0073] Furthermore, the aforementioned passenger biometric data also includes: target feature parameters; the target feature parameters include: pupil diameter change rate, heart rate change rate, and / or skin conductance intensity; compensation signal generation module 36, used to trigger a basic compensation mode if the dynamic motion sickness risk index exceeds a first threshold, using two fixed values ​​as two adaptive coefficients; if the dynamic motion sickness risk index exceeds a second threshold, or the target feature parameters exceed a specified threshold, trigger a compensation intensity multiplication mode to increase the two adaptive coefficients by a specified percentage respectively; the second threshold is greater than the first threshold; the increase percentage of the adaptive coefficient corresponding to the vehicle's comprehensive acceleration is higher than the increase percentage of the adaptive coefficient corresponding to the chaotic function value.

[0074] Furthermore, the aforementioned vehicle motion data also includes: vertical acceleration; a compensation signal generation module 46, used to determine the vehicle's comprehensive acceleration based on longitudinal acceleration, lateral acceleration, and vertical acceleration; to determine the chaotic function value based on the change in yaw rate over time; to weight and sum the vehicle's comprehensive acceleration and the chaotic function value using two adaptive coefficients to obtain a visual compensation signal containing screen compensation displacement; and a compensation signal generation module 36, used to input vehicle motion data and passenger biometric data into a hybrid model of a convolutional neural network and a long short-term memory network, so that the model outputs a visual compensation signal containing motion parameters of the compensation graphic.

[0075] Furthermore, the aforementioned basic visual compensation module 38 is used to display different compensation images on the screens of the central control display, head-up display, and rear entertainment screen, and to control the motion amplitude of the compensation images according to the visual compensation signal; the compensation images include: dynamic dots, dynamic stripes, dynamic background light, three-dimensional particle field, or dynamic virtual scene.

[0076] The device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts of the device embodiment not mentioned can be referred to the corresponding content in the aforementioned method embodiment.

[0077] Based on the above method embodiments, this application also provides an in-vehicle motion sickness visual compensation system, see [link to relevant documentation]. Figure 4 As shown, the system includes a sensor layer 41, a data processing and control layer 42, and an execution and feedback layer 43. The sensor layer 41 includes an inertial measurement unit, a camera mounted on the roof of the vehicle, or may also include millimeter-wave radar and an in-vehicle navigation system. The data processing and control layer 42 includes an in-vehicle infotainment system, or may also include an edge computing device. The data processing and control layer 42 is used to execute the methods described in the preceding method embodiments. The execution and feedback layer 43 includes a central control display screen, a head-up display, and a rear-seat entertainment system. Preferably, the system also includes an extended collaboration layer, including a cloud server for storing historical data, training and updating advanced algorithm models, and supporting personalized compensation strategies for multiple passengers. The layers are interconnected via a high-speed in-vehicle network (such as Ethernet or CAN / LIN bus).

[0078] (1) Sensor layer: Function: Responsible for collecting raw data such as vehicle movement status and passenger biometrics.

[0079] Core components include: IMU (Inertial Measurement Unit): Installed inside the vehicle to measure the vehicle's acceleration (ax, ay, az) and angular velocity (ωz) in real time. Camera: Installed on the roof of the vehicle (e.g., where the roof reading light is located) to capture passenger head posture (θhead) and pupil diameter change rate (ΔDpupil / Δt). Millimeter-wave radar: Can be used in conjunction with the camera as an alternative or supplementary solution for more stable tracking of passenger head position and movement. In-vehicle navigation system: Connected via CAN bus, providing road condition prediction information (such as curve radius and gradient).

[0080] (2) Data processing and control layer: Function: Receive and fuse data from sensors, and run the core algorithm to generate compensation instructions.

[0081] The core components include: Vehicle infotainment system: As the brain of the system, it integrates all data, runs motion sickness risk assessment models and dynamic visual compensation algorithms, and calculates the final compensation signal (ΔS). Edge computing device (optional): As an auxiliary computing unit for the vehicle infotainment system, it can take over high-computing tasks (such as deep learning model inference) to reduce the load on the main system.

[0082] (3) Execution and Feedback Layer Function: Generate and present dynamic compensation graphics on a designated display device according to the instructions issued by the vehicle system.

[0083] Core components include: A central control display screen: located on the center console, showing compensated graphics (such as dynamic dots) to the driver and front passenger. A head-up display (HUD): projects compensated graphics (such as bright dynamic stripes) onto the windshield for non-intrusive guidance. A rear-seat entertainment system: provides personalized compensated content (such as 3D particle fields or dynamic virtual scenes) on individual screens for rear passengers.

[0084] (4) Extended Collaboration Layer (Optional) Function: Enhance system performance by utilizing external resources, especially when complex calculations or big data support are required.

[0085] The core components include: a cloud server: connected via a 5G / 4G network, used to store historical data, train and update advanced algorithm models, and support personalized compensation strategies for multiple passengers.

[0086] The data flow and control logic are described below: Data Acquisition: All devices in the sensor layer continuously acquire raw data and send it to the data processing and control layer via the vehicle network.

[0087] Data fusion and decision-making: The vehicle system receives and merges all data, and runs the core algorithm to assess the current motion sickness risk index (MSI). Once the MSI exceeds a preset threshold, the system will activate a dynamic visual compensation mechanism.

[0088] Command issuance and execution: The vehicle system sends commands containing compensation parameters to one or more devices in the execution and feedback layer.

[0089] Results Feedback and Optimization: The compensation graphic is displayed on the corresponding screen. Simultaneously, the system can dynamically adjust the compensation strategy based on the passenger's real-time biofeedback (such as pupil changes and heart rate), forming a closed-loop control system. For algorithms requiring high computing power, edge computing devices can assist in the process, or collaborative computing can be performed through cloud servers.

[0090] Display device parameters: The central control screen has a refresh rate of 120Hz, the HUD brightness adjustment range is 500-1200 nits, and the rear entertainment screen supports the HDR10 display standard to ensure the clarity and response speed of the compensated graphics.

[0091] Algorithm operating environment: The vehicle system must have at least a 4-core CPU or NPU and ≥4GB of memory to ensure multi-task parallel processing capability.

[0092] In summary, the embodiments of this application, through multi-source data fusion, dynamic compensation model design, multi-screen collaboration strategy and biofeedback closed loop, achieve precise, dynamic and personalized intervention for in-vehicle motion sickness, effectively solving the shortcomings of existing technologies in terms of perception dimension, compensation logic adaptability and hardware dependence, and providing passengers with a more comfortable travel experience.

[0093] Dedicated edge computing devices are deployed in vehicles and connected to the vehicle's infotainment system via Ethernet. These devices are responsible for real-time processing of multi-source data (such as MSI calculation and ΔS generation), reducing the computational load on the infotainment system while improving algorithm efficiency. Edge computing devices can be adapted to infotainment systems with lower computing power (such as in lower-end models) while still meeting real-time requirements. Through cloud collaboration, personalized compensation for multiple passengers (such as generating independent compensation strategies for passengers in different seats) can be supported, further expanding the technology's coverage.

[0094] Some data (such as historical motion sickness features and road condition predictions) is uploaded to a cloud server. A compensation strategy is then generated using a cloud model (such as a deep learning-based compensation algorithm) and sent to the vehicle's infotainment system for execution. This solution is suitable for scenarios with high computing power requirements (such as personalized compensation for multiple passengers), but data transmission latency must be ensured to be ≤100ms to maintain real-time performance.

[0095] The system provided in this application embodiment has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0096] The computer program products of the methods, apparatus, and electronic devices provided in the embodiments of this application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementations, please refer to the method embodiments, which will not be repeated here.

[0097] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0100] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method of motion sickness visual compensation for a vehicle, characterized by, The method includes: Acquire vehicle motion data and passenger biometric data; A dynamic motion sickness risk index is generated based on the vehicle motion data and the passenger biometric data; A visual compensation signal is generated based on the dynamic motion sickness risk index, the vehicle motion data, and the passenger biometric data. Multi-screen differentiated visual compensation is performed based on the aforementioned visual compensation signal.

2. The method of claim 1, wherein, The step of generating a dynamic motion sickness risk index based on the vehicle motion data and the passenger biometric data includes: The vehicle motion data and the passenger biometric data are weighted and fused to generate a dynamic motion sickness risk index; Alternatively, the vehicle motion parameters and passenger biometric data can be used as observations, and the dynamic motion sickness risk index can be estimated using Kalman filtering.

3. The method of claim 2, wherein, The vehicle motion data includes: longitudinal acceleration, lateral acceleration, and yaw rate; the passenger biometric data includes: head posture angle; the step of weightedly fusing the vehicle motion data and the passenger biometric data to generate a dynamic motion sickness risk index includes: The first motion sickness risk index is obtained by taking the square root of the sum of the squares of the longitudinal acceleration and the lateral acceleration. The absolute value of the product of the yaw rate and the rate of change of the head posture angle over time is obtained to obtain the second motion sickness risk index. The dynamic motion sickness risk index is obtained by weighted summation of the first motion sickness risk index and the second motion sickness risk index.

4. The method of claim 1, wherein, The method further includes: Obtain traffic forecast information; Based on the road condition prediction information, when the vehicle movement trend is determined to be such that a specified road condition occurs within a specified time in the future, a visual compensation signal is generated.

5. The method of claim 2, wherein, The step of generating a visual compensation signal based on the dynamic motion sickness risk index, the vehicle motion data, and the passenger biometric data includes: Based on the dynamic motion sickness risk index or passenger biometric data, a compensation mode is determined; based on the vehicle motion data and the compensation mode, a visual compensation signal is generated using chaos theory and asymmetric phase control. Alternatively, a visual compensation signal can be generated based on a deep learning-based dynamic compensation model, using the vehicle motion data and the passenger biometric data.

6. The method of claim 5, wherein, The passenger biometric data further includes: target feature parameters; the target feature parameters include: pupil diameter change rate, heart rate change rate, and / or skin conductance intensity; the step of determining the compensation mode based on the dynamic motion sickness risk index or passenger biometric data includes: If the dynamic motion sickness risk index exceeds the first threshold, the basic compensation mode is triggered, using two fixed values ​​as two adaptive coefficients. If the dynamic motion sickness risk index exceeds the second threshold, or if the target feature parameter exceeds the specified threshold, the compensation intensity multiplication mode is triggered to increase the two adaptive coefficients by a specified ratio respectively. The second threshold is greater than the first threshold; the increase rate of the adaptive coefficient corresponding to the vehicle's comprehensive acceleration is higher than the increase rate of the adaptive coefficient corresponding to the chaotic function value.

7. The method of claim 6, wherein, The vehicle motion data also includes: vertical acceleration; the step of generating a visual compensation signal based on chaos theory and asymmetric phase control includes: The vehicle's overall acceleration is determined based on the longitudinal acceleration, the lateral acceleration, and the vertical acceleration; the chaotic function value is determined based on the change of the yaw rate over time; the vehicle's overall acceleration and the chaotic function value are weighted and summed using the two adaptive coefficients to obtain a visual compensation signal that includes screen compensation displacement. The steps for generating visual compensation signals using a deep learning-based dynamic compensation model include: The vehicle motion data and passenger biometric data are input into a hybrid model of convolutional neural network and long short-term memory network, so that the model outputs a visual compensation signal containing motion parameters of the compensation graphics.

8. The method of claim 1, wherein, The steps for performing multi-screen differentiated visual compensation based on the visual compensation signal include: Different compensation images are displayed on the central control display screen, head-up display screen, and rear entertainment screen respectively, and the movement amplitude of the compensation images is controlled according to the visual compensation signal; the compensation images include: dynamic dots, dynamic stripes, dynamic background light, three-dimensional particle field, or dynamic virtual scene.

9. An on-vehicle motion sickness visual compensation device characterized by comprising: The device includes: The data acquisition module is used to acquire vehicle motion data and passenger biometric data; The data fusion module is used to weightedly fuse the vehicle motion data and the passenger biometric data to generate a dynamic motion sickness risk index; The compensation signal generation module is used to generate a visual compensation signal based on the dynamic motion sickness risk index or passenger biometric data. The visual compensation module is used to perform multi-screen differentiated visual compensation based on the visual compensation signal.

10. An on-vehicle motion sickness visual compensation system characterized by comprising: The system includes: a sensor layer, a data processing and control layer, and an execution and feedback layer; The sensor layer includes: an inertial measurement unit, a camera mounted on the roof of the vehicle, or may also include: millimeter-wave radar, an in-vehicle navigation system; The data processing and control layer includes: a vehicle infotainment system, or further includes: an edge computing device; the data processing and control layer is used to perform the method as described in any one of claims 1-8; The execution and feedback layer includes: a central control display screen, a head-up display, and a rear entertainment system; Preferably, the system also includes an extended collaboration layer, comprising: a cloud server for storing historical data, training and updating advanced algorithm models, and supporting personalized compensation strategies for multiple passengers.

Images