Carsickness relieving method and system based on display screen dynamic background image enhancement

Abstract

The invention provides a carsickness relieving method and system based on display screen dynamic background image enhancement, and the method comprises the steps: collecting the acceleration and posture data of a vehicle through a sensor, and carrying out the real-time fusion processing of the data through a data fusion algorithm; the input display picture is cut and zoomed, a black and white stripe area is set, black and white stripes are controlled to move, the rotation angle of an effective display area is adjusted in real time through an algorithm by means of sensor data, black spots are set in the effective display area to move up and down for visual compensation and other processing, and video signals are transcoded and processed; and the video content is output to an adaptive display interface, so that the display of the video content is realized, carsickness is relieved by introducing visual prompts to passengers in the vehicle, and the comfort and the sense of safety of the passengers are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-machine interface (HMI), vehicle-mounted display system optimization, anti-motion sickness and image processing, and in particular relates to a method and system for alleviating motion sickness based on dynamic background image enhancement of a display screen. Background Art

[0002] With the development of smart cars, the application of in-vehicle entertainment systems and information display devices has become more and more widespread. However, the use of in-vehicle display devices, especially during driving, may cause passengers to feel motion sickness due to the irregular movement of the vehicle. The root cause of motion sickness is the inconsistency between visual perception and the body's vestibular system's perception of vehicle movement. Especially when passengers focus on the display device and cannot obtain external motion clues, the symptoms of motion sickness will be more obvious. Traditional anti-motion sickness methods mainly rely on drugs, special seat designs or transparent displays on car windows, but drugs have side effects, special seats and display devices are expensive to manufacture and have limited applicability. Some existing solutions try to use pleasant sounds and human-machine interfaces (HMIs) to inform passengers in advance of upcoming movements that may cause motion sickness, but existing technologies rely on motion sickness warnings, such as sounds and simple visual cues. These frequent warnings can cause passengers to feel annoyed or distracted, thereby reducing the practicality of these systems. Considering that in actual scenarios, if the vehicle is traveling at high speed or the influence of the interior of the vehicle, the ambient noise may be too loud to mask the sound cues, and frequent changes of direction or acceleration and deceleration in complex road conditions, excessive sound cues can cause passengers to feel irritable or anxious, which in turn aggravates the discomfort of motion sickness. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present application provides a motion sickness relief method and system based on dynamic background image enhancement of a display screen, which solves the motion sickness problem caused by the inconsistency between the visual and vestibular system information of passengers during vehicle driving, and reduces the dependence on sound by introducing visual prompts, thereby effectively alleviating the motion sickness reaction of passengers. The present invention adopts the following technical solutions:

[0004] According to a first aspect of the present invention, a method for alleviating motion sickness based on display screen dynamic background image enhancement is proposed, comprising the following steps:

[0005] Data fusion: using a data fusion algorithm to process the acquired acceleration and angular velocity data, obtaining attitude angle data after processing, and obtaining a rotation matrix based on the attitude angle data;

[0006] Image processing: Calculate boundary coordinates according to the obtained video signal, determine the screen boundary by the boundary coordinates, control the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, determine the black and white stripe area according to the effective display area and the screen boundary, control the left and right movement of the black and white stripes according to the yaw angle in the attitude angle data, control the rotation of the effective display area according to the rotation matrix, and set a plurality of black dots around the effective display area, and the black dots move up and down with the acceleration value for visual compensation; crop and scale the video data to adapt it to the size of the output display device; utilize the design of the dynamic black and white stripe background and its synchronous change with the vehicle's steering state to optimize the visual information processing of passengers and improve their adaptability and comfort.

[0007] Video output: The video signal is transcoded and processed into a format suitable for a display device and transmitted to the display device, thereby alleviating motion sickness by introducing visual cues to passengers in the vehicle.

[0008] Preferably, before data fusion, the following steps are also included:

[0009] Video decoding: The processing unit receives the video signal from the video source through the interface and decodes the video signal into image data and synchronization signal;

[0010] Data acquisition: The processing unit reads acceleration and angular velocity data from the sensor through an interface, and stores the acquired acceleration and angular velocity data in a memory.

[0011] The data fusion further includes: the processing unit acquiring the acceleration and the angular velocity data from the memory;

[0012] The image processing further includes: processing the decoded image data to adapt to the size of the display device through cropping and scaling.

[0013] Preferably, the accelerometer data includes: accelerometer components of X-axis, Y-axis and Z-axis, and the gyroscope data includes: gyroscope components of X-axis, Y-axis and Z-axis, each of which occupies two bytes and is represented as a 12-bit signed integer.

[0014] Preferably, the data fusion specifically includes: initializing the quaternion to q0=1, q1=0, q2=0, q3=0, and the expression of the quaternion is as follows:

[0015] q=[q0 q1 q2 q3]

[0016] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis, the proportional gain coefficient Kp and the integral gain coefficient Ki in the data fusion algorithm are initialized to appropriate values, and whenever new sensor data is received, the quaternion is updated according to the sensor data and error feedback;

[0017] The yaw angle θ is obtained by the following formula:

[0018] θ=atan2(2(q0q3+q1q2),1-2(q2 2 +q3 2 ))

[0019] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis,

[0020] Each frame of the image is rotated slightly by constructing a rotation matrix, wherein the rotation matrix R z The formula is as follows:

[0021]

[0022] Among them, θ represents the yaw angle, and the values ​​of cosθ and sinθ are obtained by looking up the table.

[0023] Preferably, the image processing specifically includes: the calculation formulas of the left boundary coordinates, the right boundary coordinates, the upper boundary coordinates and the lower boundary coordinates of the effective display area are as follows:

[0024]

[0025]

[0026] Among them, h pixel is the horizontal resolution after scaling, h disp is the horizontal resolution of the display device, v pixel is the vertical resolution after scaling, v disp is the vertical resolution of the display device, the border distance parameter controls the distance between the effective display area and the screen border, the left border coordinate and the right border coordinate are the horizontal coordinates, and the upper border coordinate and the lower border coordinate are the vertical coordinates.

[0027] Preferably, the image processing specifically includes: generating black and white stripes with fixed-interval pixels by performing a modulo operation on the sum of a display device row counter and a deflection factor; when a time parameter accumulates to a threshold, updating the deflection factor using the yaw angle collected by the sensor; an increase or decrease in the deflection factor directly affects the movement of the black and white stripes; and the range of the black and white stripe area is controlled by a distance parameter between the effective display area and the boundary.

[0028] Preferably, the image processing specifically includes: processing the rotation part by perspective transformation, performing small-amplitude rotation processing on each frame of the image by constructing the rotation matrix, setting threshold parameters, and when the rotation angle is between -5° and 5°, it is an invalid rotation, using a bilinear interpolation method to calculate the pixel values ​​of non-integer coordinates after rotation, and accelerating the overall calculation process by pipeline and parallel processing. The formula of the perspective transformation is as follows:

[0029]

[0030] Among them, (x, y) is the input pixel coordinates, (x′, y′) is the unnormalized pixel component after rotation, these projection points obtained by perspective transformation need to be divided by the normalization factor w in order to map back to the two-dimensional plane, the matrix element a is the coefficient that controls the scaling in the x direction, b is the coefficient that controls the beveling, c is the coefficient that controls the translation in the x direction, d is the coefficient that controls the rotation, e is the coefficient that controls the scaling in the y direction, f is the coefficient that controls the translation in the y direction, w is the normalization factor in homogeneous coordinates, g controls the deformation degree of perspective transformation in the horizontal direction, and h controls the deformation degree of perspective transformation in the vertical direction. In order to produce a certain displacement effect, the values ​​of the parameters c and f of the perspective transformation matrix are dynamically adjusted according to the size of the rotation angle. In order to make it more in line with the visual habits of the human eye during rotation, the parameters g and h are dynamically adjusted to flexibly change the scaling effect of the object in the video in terms of viewing angle, where the relationship between a, b, d and e and θ is expressed as:

[0031]

[0032] Among them, θ represents the yaw angle. By adopting the perspective transformation method, each frame of the image can be rotated slightly by constructing a rotation matrix. In order to take into account the impact of the vehicle's slight tilt to the left or right in real life, which causes the image to shake, a threshold parameter is set. Only when the rotation angle is large enough is it considered a valid rotation. The rotation matrix is ​​composed of pre-calculated cosθ and sinθ values, and the multiplication and addition operations of the processing unit are used to realize the transformation of pixel coordinates. The bilinear interpolation method is used to calculate the pixel values ​​of non-integer coordinates after rotation, which can improve the accuracy; the overall calculation process is accelerated by pipeline and parallel processing, which can ensure that the image can complete the rotation effect in real time and smoothly.

[0033] Preferably, the image processing specifically includes: adding three black dots on both sides of the effective display area, the black dots make corresponding visual compensation according to the acceleration value of the vehicle in the forward or backward direction, and when the acceleration value of the vehicle accelerating forward or backward is greater than the set threshold, the black dots move with a corresponding amplitude according to the magnitude of the acceleration; processing the value of the acceleration Y-axis data stored in the memory, setting the initial position of the black dots to the left and right sides of the display area, and the moving distance of the black dots is related to the Y-axis value in the acceleration data. The addition of black dots improves the display effect and ensures the consistency of visual information without affecting the viewing effect, so that passengers can get a better anti-motion sickness experience.

[0034] According to a second aspect of the present invention, a system based on display screen dynamic background image enhancement is provided, comprising:

[0035] Data fusion module: Data fusion module: Use data fusion algorithm to process the acquired acceleration and angular velocity data, obtain attitude angle data after processing, and obtain rotation matrix according to the attitude angle data; to obtain more accurate motion state information;

[0036] Image processing module: calculate boundary coordinates according to the obtained video signal, determine the screen boundary according to the boundary coordinates, control the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, and determine the black and white stripe area according to the effective display area and the screen boundary. Specifically, set the boundary distance parameter to control the interval between the effective display area and the screen boundary; control the left and right movement of the black and white stripes according to the yaw angle in the attitude angle data, and control the rotation of the effective display area according to the rotation matrix. A plurality of black dots are arranged around the effective display area, and the black dots move up and down with the value of acceleration for visual compensation;

[0037] Video output module: transcodes and processes the video signal into a format suitable for the display device, transmits it to the display device, and helps passengers in the car relieve motion sickness by adding visual cues to the displayed content.

[0038] Preferably, it also includes:

[0039] Video decoding module: receiving a video signal from a video source through an interface, and decoding the video signal into image data and a synchronization signal;

[0040] Data acquisition module: reads acceleration and angular velocity data from the sensor through a data interface, and stores the acquired acceleration and angular velocity data in a memory.

[0041] The data fusion module further comprises: acquiring the acceleration and the angular velocity data from the memory;

[0042] The image processing module further includes: processing the decoded image data through cropping and scaling to adapt the size of the display device.

[0043] The system collects the acceleration and posture data of the vehicle through sensors, combines the data fusion algorithm to perform real-time fusion processing on the data, and then processes the input display image to output the dynamic stripe background and visual signals to the display device to realize the visual prompt function, which can effectively alleviate the passengers' motion sickness reaction and improve the passengers' comfort and sense of security.

[0044] Preferably, the video decoding module comprises:

[0045] Serial-to-parallel conversion unit: converts the input serial differential signal into serial single-ended data, and then converts it into fixed-width parallel data for subsequent processing;

[0046] A word alignment calibration unit: comparing the parallel data with a control character, and when the control character is detected continuously in a plurality of delay values, selecting an intermediate value as a final calibration delay;

[0047] Data synchronization unit: performs synchronization processing on the image data of multiple channels to ensure the timing consistency of the color channels;

[0048] The encoding and decoding unit decodes the parallel data from the high-order encoding format into low-order data, and generates a line field synchronization signal and a data valid enable signal at the same time.

[0049] Preferably, it also includes a sensor, a display device, a memory controller and a memory storage module arranged in the processing unit, and the memory storage module includes a read cache unit and a write cache unit. The data acquisition module stores the acquired data in the temporary storage area of ​​the memory storage module, and the data fusion module performs fusion processing on the data and then outputs it to the image processing module through the temporary storage area.

[0050] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0051] Compared with the prior art, the beneficial results of the present invention are:

[0052] The present invention provides a motion sickness relief method and system based on dynamic background image enhancement of a display screen. Different from the traditional motion sickness solution, the human-machine interface (HMI) can intuitively display the motion state of the vehicle through a graphical interface, animation or other visual elements, and the HMI can be customized according to the preferences of the passengers to provide a more personalized experience. The proposal collects the acceleration and posture data of the vehicle through sensors, and combines the data fusion algorithm to perform real-time fusion processing on these data. The processing of the video image includes display size adjustment and black and white stripe generation, anti-shake and high-precision real-time adjustment of rotation processing, and black dot assistance of acceleration and deceleration images. Finally, by processing the input display image, the dynamic stripe background and visual signals are output to the display screen, and the visual perception of the passenger is synchronized with the perception of the inner ear balance system in the form of visual feedback, thereby reducing sensory conflicts, significantly alleviating the passenger's motion sickness, and making the passenger more comfortable during driving; by early warning of steering changes, the passenger can better adapt to the vehicle movement, thereby reducing the probability of motion sickness; in addition, the dynamically adjusted display effect can improve the consistency of visual information and enhance the passenger's riding experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0054] Figure 1 is a flow chart of a method for alleviating motion sickness based on display screen dynamic background image enhancement according to a specific embodiment of the present invention;

[0055] Figure 2 is a demonstration diagram of the division of the effective display area and the black and white stripe area according to a specific embodiment of the present invention;

[0056] Figure 3 is a rotation processing demonstration diagram according to a specific embodiment of the present invention;

[0057] Figure 4 It is a demonstration of black spot acceleration and deceleration processing according to a specific embodiment of the present invention;

[0058] Figure 5 is an overall block diagram of a motion sickness relief system based on display screen dynamic background image enhancement according to a specific embodiment of the present invention;

[0059] Figure 6The figure is a flow chart of dynamic background stripe video processing according to a specific embodiment of the present invention.

[0060] The meaning of the numbers in the figure are: 1. Effective display area; 2. Black and white stripe area; 3. Video decoding module; 4. Data acquisition module; 5. Data fusion module; 6. Image processing module; 7. Video output module; 8. Memory storage module. DETAILED DESCRIPTION

[0061] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0062] In the description of this proposal, it should be noted that the terms "include", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The terms "installed", "provided with", "set / connected", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection, a mechanical connection, an electrical connection, a direct connection, an indirect connection through an intermediate medium, or a communication between the two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0063] To facilitate understanding by those skilled in the art, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0064] Figure 1 FIG. 4 is a flow chart showing a method for alleviating motion sickness based on display screen dynamic background image enhancement according to a specific embodiment of the present invention. Figure 1 The specific workflow is as follows:

[0065] Video decoding: The processing unit receives the video signal from the video source through the interface and decodes the video signal into image data and synchronization signal;

[0066] Data acquisition: the processing unit reads the acceleration and angular velocity data from the sensor through an interface, and stores the acquired acceleration and angular velocity data in a memory;

[0067] Data fusion: using a data fusion algorithm to process the acquired acceleration and angular velocity data, obtaining attitude angle data after processing, and obtaining a rotation matrix based on the attitude angle data;

[0068] Image processing: Calculate boundary coordinates according to the obtained video signal, determine the screen boundary by the boundary coordinates, control the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, determine the black and white stripe area according to the effective display area and the screen boundary, control the left and right movement of the black and white stripes according to the yaw angle in the attitude angle data, control the rotation of the effective display area according to the rotation matrix, and set a plurality of black dots around the effective display area, and the black dots move up and down with the acceleration value for visual compensation; crop and scale the video data to adapt it to the size of the output display device; utilize the design of the dynamic black and white stripe background and its synchronous change with the vehicle's steering state to optimize the visual information processing of passengers and improve their adaptability and comfort.

[0069] Video output: The video signal is transcoded and processed into a format suitable for a display device and transmitted to the display device, thereby alleviating motion sickness by introducing visual cues to passengers in the vehicle.

[0070] In a specific embodiment, the processing unit can be implemented by using FPGA (Field Programmable Gate Array). Specifically, the video decoding includes outputting the video source to the FPGA via HDMI (High Definition Multimedia Interface), and the FPGA receives the TMDS (Transition Minimized DifferentialSignaling) signal is decoded into RGB data and synchronization signal; the data acquisition includes: FPGA reads the data of accelerometer and gyroscope from sensor through I2C interface and stores it in memory; the data fusion includes: FPGA reads the data of accelerometer and gyroscope from memory; the data fusion algorithm uses Mahony complementary filtering algorithm to fuse the data; the image processing includes: cutting and scaling the decoded RGB data, calculating the boundary coordinates of the scaled video signal, setting the boundary distance parameter border_width to control the interval between the effective display area 1 and the screen boundary and determining the black and white stripe area 2; controlling the left and right movement of the black and white stripes according to the size of the yaw angle, controlling the rotation of the effective display area 1 according to the size of the rotation matrix, and black dots are set around the effective display area 1, and the black dots move up and down with the Y-axis value of the accelerometer for visual compensation; the video output includes: transcoding and processing the video into LVDS (Low Voltage Differential Signaling) signal, outputting it to the LVDS display, and introducing visual prompts to passengers in the car to relieve motion sickness.

[0071] In a specific embodiment, the RGB data adopts RGB888.

[0072] Through the above-mentioned motion sickness relief method based on dynamic background image enhancement of the display screen, sensors are used to collect the acceleration and posture data of the vehicle, and the data are fused and processed in real time in combination with the Mahony complementary filtering algorithm. The visual prompt information is adapted to the size of the LVDS screen by cropping and scaling the video data; black and white stripes, rotation processing and black dots are added to assist, so that passengers in the car can be reminded in time regardless of acceleration or deceleration, and when the vehicle is turning, and the display effect is dynamically adjusted to improve the consistency of visual information, and finally output to the LVDS display, so as to synchronize the passenger's visual perception with the perception of the inner ear balance system in the form of visual feedback, thereby reducing sensory conflicts, significantly alleviating the passenger's motion sickness, and making the passenger more comfortable during driving.

[0073] In a specific embodiment, the sensor uses an MPU6050 sensor, which integrates a three-axis accelerometer and a three-axis gyroscope, and can measure the acceleration and angular velocity of an object in real time. The accelerometer data includes: accelerometer components of the X-axis, Y-axis and Z-axis, and the gyroscope data includes: gyroscope components of the X-axis, Y-axis and Z-axis. Each of the components occupies two bytes and is represented as a 12-bit signed integer. In other embodiments, other inertial measurement unit (IMU) sensors can also be used.

[0074] The Mahony (complementary filtering) algorithm calculates the attitude error by using a PI (proportional-integral) controller in combination with the gravity direction provided by the accelerometer and the angular velocity of the gyroscope. The error is applied to the gyroscope data through a feedback mechanism to correct the drift of the attitude solution to achieve an accurate attitude angle. To implement the Mahony algorithm on an FPGA, it is necessary to consider the problem of floating-point operations. Since the hardware description language cannot directly process floating-point numbers or complex mathematical functions, the Mahony algorithm is implemented using fixed-point numbers and pipeline design methods. The specific implementation method is to first read the accelerometer and gyroscope data. Since the algorithm uses quaternions to represent the rotation state of an object, the quaternion is first initialized to a unit quaternion, that is:

[0075] In a specific embodiment, the data fusion includes: initializing the quaternion to q0=1, q1=0, q2=0, q3=0, and the expression of the quaternion is as follows:

[0076] q=[q0 q1 q2 q3]

[0077] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis, the proportional gain coefficient Kp and the integral gain coefficient Ki in the data fusion algorithm are initialized to appropriate values, and the quaternion is updated according to the sensor data and error feedback at each rising edge of the clock; when the module is reset, the attitude error is first multiplied by K p The error term is calculated and the integral term is updated by ki. The attitude error refers to the difference between the accelerometer and the quaternion. Then, each component of the quaternion is updated according to the updated integral term and gyroscope data.

[0078] After obtaining the quaternion, the inverse tangent value is obtained by using a lookup table, and then the yaw angle θ is obtained by the following formula:

[0079] θ=atan2(2(q0q3+q1q2),1-2(q2 2 +q3 2 ))

[0080] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis,

[0081] In a specific embodiment, for the rotation matrix, since the present application uses the attitude angle of left and right movement, that is, the yaw angle, it is assumed that the yaw angle is θ, and a rotation matrix is ​​constructed to perform a small rotation process on each frame of the image. The rotation matrix R z The formula is as follows:

[0082]

[0083] Among them, θ represents the yaw angle, and the values ​​of cosθ and sinθ are obtained by looking up the table.

[0084] In a specific embodiment, the image processing specifically includes: the calculation formulas of the left boundary coordinates, the right boundary coordinates, the upper boundary coordinates and the lower boundary coordinates of the effective display area are as follows:

[0085]

[0086] Among them, h pixel is the horizontal resolution after scaling, h disp is the horizontal resolution of the display device, v pixel is the vertical resolution after scaling, v disp is the vertical resolution of the display device, the border distance parameter controls the distance between the effective display area and the screen border, the left border coordinate and the right border coordinate are the horizontal coordinates, and the upper border coordinate and the lower border coordinate are the vertical coordinates.

[0087] In a specific embodiment, the image processing also includes: generating black and white stripes with a pixel interval of 20 by performing a modulo operation on the sum of the display row counter h_cnt and the deflection factor stripe_pos; when the time parameter cnt_hz accumulates to a threshold, updating the deflection factor by the yaw angle collected on the rising edge; the increase or decrease of the deflection factor directly affects the movement of the black and white stripes; and the range of the black and white stripes is controlled by the boundary distance parameter border_width.

[0088] In a specific embodiment, the image processing specifically includes: processing the rotation part by perspective transformation, performing a small rotation process on each frame of the image by constructing the rotation matrix, setting a threshold parameter, and when the rotation angle is between -5° and 5°, it is an invalid rotation, and using a bilinear interpolation method to calculate the pixel value of the non-integer coordinate after rotation, and accelerating the overall calculation process by pipeline and parallel processing. The formula of the perspective transformation is as follows:

[0089]

[0090] Among them, (x, y) is the input pixel coordinates, (x′, y′) is the unnormalized pixel component after rotation, these projection points obtained by perspective transformation need to be divided by the normalization factor w in order to map back to the two-dimensional plane, the matrix element a is the coefficient that controls the scaling in the x direction, b is the coefficient that controls the beveling, c is the coefficient that controls the translation in the x direction, d is the coefficient that controls the rotation, e is the coefficient that controls the scaling in the y direction, f is the coefficient that controls the translation in the y direction, w is the normalization factor in homogeneous coordinates, g controls the deformation degree of perspective transformation in the horizontal direction, and h controls the deformation degree of perspective transformation in the vertical direction. In order to produce a certain displacement effect, the values ​​of the parameters c and f of the perspective transformation matrix are dynamically adjusted according to the size of the rotation angle. In order to make it more in line with the visual habits of the human eye during rotation, the parameters g and h are dynamically adjusted to flexibly change the scaling effect of the object in the video in terms of viewing angle, where the relationship between a, b, d and e and θ is expressed as:

[0091]

[0092] Among them, θ represents the yaw angle. By adopting the perspective transformation method, each frame of the image can be rotated slightly by constructing a rotation matrix. In order to take into account the impact of the vehicle tilting slightly left or right in real situations, which causes the image to shake, a threshold parameter is set. Only when the rotation angle is large enough is it considered a valid rotation. The rotation matrix is ​​composed of pre-calculated cosθ and sinθ values, and the pixel coordinates are transformed with the multiplication and addition operations of the FPGA. The bilinear interpolation method is used to calculate the pixel values ​​of non-integer coordinates after rotation, which can improve the accuracy; the overall calculation process is accelerated by pipeline and parallel processing, which can ensure that the image can complete the rotation effect in real time and smoothly.

[0093] In a specific embodiment, the image processing further includes: adding three black dots on both sides of the effective display area 1, respectively, and the black dots make corresponding visual compensation according to the acceleration value of the vehicle in the forward or backward direction, that is, the Y-axis acceleration value. When the acceleration value of the vehicle accelerating forward or backward is greater than the set threshold, the black dots move with a corresponding amplitude according to the magnitude of the acceleration; processing the value of the accelerometer Y-axis data stored in the memory, setting the initial position of the black dots to the left and right sides of the display area, and the moving distance of the black dots is related to the value of the accelerometer Y-axis. The addition of black dots improves the display effect and ensures the consistency of visual information without affecting the viewing effect, so that passengers can get a better anti-motion sickness experience.

[0094] Figure 2 1 is a diagram showing the division of the effective display area 1 and the black and white stripe area 2 according to a specific embodiment of the present invention. Figure 2As shown in the figure, the dark area is the black and white stripe display area, the white area is the effective display area 1, the effective display area 1 is smaller than the screen size, and the black and white stripe display area is located between the screen and the effective display area 1. The interval between the effective display area 1 and the screen boundary is controlled by setting the border distance parameter border_width, and the range of the black and white stripe area 2 is determined thereby.

[0095] In a specific embodiment, the area of ​​the effective display area 1 is taken as an example of a resolution of 380*700. In other embodiments, the area of ​​the effective display area 1 can be adjusted according to the actual screen size of the LVDS display to ensure a better anti-motion sickness effect.

[0096] Figure 3 is a rotation processing demonstration diagram according to a specific embodiment of the present invention, such as Figure 3 As shown in the figure, when the vehicle turns left or right, the system will sense and drive the black and white stripes background to move smoothly in the corresponding direction in real time, intuitively prompting the passengers to make the upcoming steering operation. In addition, the system dynamically adjusts the rotation angle of the display screen based on the fused yaw angle information to ensure that it is highly consistent with the actual motion state of the vehicle. The rotation angle is related to the yaw angle θ, and the unit is deg. The moving speed of the black and white stripes is v, and the unit is deg / s. The Mahony (complementary filtering) algorithm uses the PI (proportional-integral) controller to combine the gravity direction provided by the accelerometer and the angular velocity of the gyroscope to calculate the attitude error. The error is applied to the gyroscope data through the feedback mechanism to correct the drift of the attitude solution to achieve an accurate attitude angle. To implement the Mahony algorithm on FPGA, it is necessary to consider the problem of floating-point operations. Since the hardware description language cannot directly process floating-point numbers or complex mathematical functions, the fixed-point number and pipeline design method are used to implement the Mahony algorithm. The specific implementation method is to first read the accelerometer and gyroscope data. Since the algorithm uses quaternions to represent the rotation state of the object, the quaternion is first initialized to a unit quaternion, that is:

[0097] In a specific embodiment, the quaternion is initialized to q0=1, q1=0, q2=0, q3=0, and the expression of the quaternion is as follows:

[0098] q=[q0 q1 q2 q3]

[0099] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis, and the proportional gain coefficient K p Initialized to 2000, integral gain coefficient K i Initialized to 1, at each rising edge of the clock, the quaternion is updated according to the sensor data and error feedback; when the module is reset, the attitude error is first multiplied by K pThe error term is calculated and the integral term is updated by ki. The attitude error refers to the difference between the accelerometer and the quaternion. Then, each component of the quaternion is updated according to the updated integral term and gyroscope data.

[0100] After obtaining the quaternion, the inverse tangent value is obtained by using a lookup table, and then the yaw angle θ is obtained by the following formula:

[0101] θ=atan2(2(q0q3+q1q2),1-2(q2 2 +q3 2 ))

[0102] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis,

[0103] In a specific embodiment, for the rotation matrix, since the present application uses the attitude angle of left and right movement, that is, the yaw angle, it is assumed that the yaw angle is θ, and a rotation matrix is ​​constructed to perform a small rotation process on each frame of the image. The rotation matrix R z The formula is as follows:

[0104]

[0105] Among them, θ represents the yaw angle, and the values ​​of cosθ and sinθ are obtained by looking up the table.

[0106] In a specific embodiment, the image processing specifically includes: the calculation formulas of the left boundary coordinates, the right boundary coordinates, the upper boundary coordinates and the lower boundary coordinates of the effective display area are as follows:

[0107]

[0108] Among them, h pixel is the horizontal resolution after scaling, h disp is the horizontal resolution of the display device, v pixel is the vertical resolution after scaling, v disp is the vertical resolution of the display device, the border distance parameter controls the distance between the effective display area and the screen border, the left border coordinate and the right border coordinate are the horizontal coordinates, and the upper border coordinate and the lower border coordinate are the vertical coordinates.

[0109] In a specific embodiment, a modulo operation is performed on the sum of a row counter h_cnt and a deflection factor stripe_pos to generate black and white stripes with a pixel interval of 20. When the time parameter cnt_hz accumulates to a threshold, the deflection factor is updated by the yaw angle collected on the rising edge. The increase or decrease of the deflection factor directly affects the movement of the black and white stripes. The range of the black and white stripes is controlled by the boundary distance parameter border_width.

[0110] In a specific embodiment, the rotation part is processed by perspective transformation. Each frame of the image is rotated slightly by constructing the rotation matrix. The threshold parameter is set. When the rotation angle is between -5° and 5°, it is invalid rotation, that is, when the absolute value of the rotation angle is greater than 5°, it is considered a valid rotation. The bilinear interpolation method is used to calculate the pixel values ​​of non-integer coordinates after rotation, and the overall calculation process is accelerated by pipeline and parallel processing. The formula of the perspective transformation is as follows:

[0111]

[0112] Among them, (x, y) is the input pixel coordinates, (x′, y′) is the unnormalized pixel component after rotation, these projection points obtained by perspective transformation need to be divided by the normalization factor w in order to map back to the two-dimensional plane, the matrix element a is the coefficient that controls the scaling in the x direction, b is the coefficient that controls the beveling, c is the coefficient that controls the translation in the x direction, d is the coefficient that controls the rotation, e is the coefficient that controls the scaling in the y direction, f is the coefficient that controls the translation in the y direction, w is the normalization factor in homogeneous coordinates, g controls the deformation degree of perspective transformation in the horizontal direction, and h controls the deformation degree of perspective transformation in the vertical direction. In order to produce a certain displacement effect, the values ​​of the parameters c and f of the perspective transformation matrix are dynamically adjusted according to the size of the rotation angle. In order to make it more in line with the visual habits of the human eye during rotation, the parameters g and h are dynamically adjusted to flexibly change the scaling effect of the object in the video in terms of viewing angle, where the relationship between a, b, d and e and θ is expressed as:

[0113]

[0114] Among them, θ represents the yaw angle. By adopting the perspective transformation method, each frame of the image can be rotated slightly by constructing a rotation matrix. In order to take into account the impact of the vehicle tilting slightly left or right in real situations, which causes the image to shake, a threshold parameter is set. Only when the rotation angle is large enough is it considered a valid rotation. The rotation matrix is ​​composed of pre-calculated cosθ and sinθ values, and the pixel coordinates are transformed with the multiplication and addition operations of the FPGA. The bilinear interpolation method is used to calculate the pixel values ​​of non-integer coordinates after rotation, which can improve the accuracy; the overall calculation process is accelerated by pipeline and parallel processing, which can ensure that the image can complete the rotation effect in real time and smoothly.

[0115] Figure 4 is a demonstration of black spot acceleration and deceleration processing according to a specific embodiment of the present invention, such as Figure 4 As shown, when the vehicle accelerates or decelerates, the black dots in the picture move forward and backward with the change of acceleration, providing a precise visual compensation effect. This design can not only effectively attract the attention of passengers, but also significantly improve the passengers' expectations and adaptability to vehicle dynamics through visual signals, thereby greatly reducing the risk of motion sickness. Specifically, three black dots are added on both sides of the effective display area 1. The black dots make corresponding visual compensation according to the acceleration value of the vehicle in the forward or backward direction, that is, the Y-axis acceleration value. The moving speed of the black dots is v, and the unit is deg / s. When the acceleration value of the vehicle accelerating forward or backward is greater than the set threshold, the black dots move with a corresponding amplitude according to the magnitude of the acceleration; the value of the accelerometer Y-axis data stored in the memory is processed, and the initial position of the black dots is set to the left and right sides of the display area. The moving distance of the black dots is related to the value of the accelerometer Y-axis. The addition of black dots improves the display effect without affecting the viewing effect and ensures the consistency of visual information, so that passengers can get a better anti-motion sickness experience.

[0116] Figure 5 is an overall block diagram of a motion sickness relief system based on display screen dynamic background image enhancement according to a specific embodiment of the present invention. Figure 5 As shown, according to a second aspect of the present invention, a system based on display screen dynamic background image enhancement is proposed, comprising:

[0117] Video decoding module 3: receiving a video signal from a video source through an interface, and decoding the video signal into image data and a synchronization signal;

[0118] Data acquisition module 4: reads acceleration and angular velocity data from the sensor through a data interface, and stores the acquired acceleration and angular velocity data in a memory.

[0119] Data fusion module 5: reads the acceleration and angular velocity data of the sensor from the memory, and processes the acceleration and angular velocity data using a data fusion algorithm to obtain more accurate motion state information;

[0120] Image processing module 6: cropping and scaling the decoded image data, calculating the boundary coordinates of the scaled video signal, controlling the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, and determining the black and white stripe area. Specifically, setting a boundary distance parameter to control the interval between the effective display area and the screen boundary; controlling the left and right movement of the black and white stripes according to the size of the yaw angle, and controlling the rotation of the effective display area according to the size of the rotation matrix. Black dots are set around the effective display area, and the black dots move up and down with the Y-axis value of the accelerometer for visual compensation;

[0121] Video output module 7: transcodes and processes the video signal into a format suitable for the display device, transmits it to the display device, and helps passengers in the car relieve motion sickness by adding visual cues to the displayed content.

[0122] In a specific embodiment, the processing unit is implemented by FPGA. Specifically, the video decoding module 3 includes: the video source is output to the FPGA through HDMI. Optionally, taking the video source with a resolution of 1920*1080 as an example, the FPGA decodes the received TMDS signal into RGB888 data and synchronization signal; the data acquisition module 4 includes: reading the data of the accelerometer and gyroscope from the MPU6050 sensor through the I2C interface, and storing the read data in the FIFO (First Input First Output) memory; the data fusion module 5 includes: reading the data of the accelerometer and gyroscope from the FIFO memory, and fusing the data using the Mahony algorithm; this module facilitates efficient acquisition and processing of sensor data; the image processing module 6: cutting and scaling the decoded RGB888 data, calculating the boundary coordinates of the scaled video signal, setting the parameter border_width to control the interval between the effective display area 1 and the screen boundary and determining the black and white stripe area 2; controlling the left and right movement of the black and white stripes according to the size of the yaw angle, and controlling the rotation of the effective display area 1 according to the size of the rotation matrix, and setting black dots around the effective display area 1, and the black dots move up and down with the Y-axis value of the accelerometer for visual compensation; by cutting and scaling the video data, it is adapted to the size of the LVDS screen; using the design of the dynamic stripe background and its synchronous change with the vehicle steering state, the visual information processing of the passengers is optimized at the same time, and the adaptability and comfort of the passengers are improved. The video output module 7 includes: transcoding and processing the video into an LVDS signal, outputting it to the LVDS display, and introducing visual prompts to the passengers in the car to relieve motion sickness.

[0123] The system collects the acceleration and posture data of the vehicle through sensors, combines the Mahony complementary filtering algorithm to perform real-time fusion processing on the data, and then processes the input display screen to output the dynamic stripe background and visual signals to the display screen to achieve a visual prompt function, which can effectively alleviate passengers' motion sickness reactions and improve passengers' comfort and sense of security.

[0124] Optionally, the video decoding module 3 further includes:

[0125] A serial-to-parallel conversion unit: converts the input serial differential signal into serial single-ended data, and then converts the serial single-ended data into parallel data of a fixed width. In a specific embodiment, the parallel data of the fixed width is 10 bits.

[0126] Word alignment calibration unit: compares the parallel data with the control character. In a specific embodiment, the number of the control characters is 4. If the control character is detected continuously at multiple delay values, it is considered that the range of the delay value has been found. At this time, the middle delay value is taken as the final delay value, indicating that the calibration has been completed. In a specific embodiment, if the control character is detected continuously at 16 delay values, it is considered that the range of the delay value has been found.

[0127] Data synchronization unit: synchronizes multi-channel image data. In a specific embodiment, the multi-channel image data is three groups of data: red, green and blue, to prevent color deviation of the image.

[0128] The encoding and decoding unit decodes the parallel data from the high-bit encoding format into the low-bit data. In a specific embodiment, the 10-bit parallel data is converted into 8-bit data, and a line field signal and a data valid enable are generated at the same time.

[0129] Finally, the decoded RGB888 signal is transmitted to the image processing module 6.

[0130] In a specific embodiment, an MPU6050 inertial measurement unit (IMU) sensor is used to integrate a three-axis accelerometer and a three-axis gyroscope to measure the acceleration and angular velocity of an object. In this proposal, a data acquisition module 4 is designed on the FPGA to efficiently acquire and process sensor data. The data acquisition module 4 reads the data of the accelerometer and gyroscope from the MPU6050 sensor each time through the I2C interface. These data include the acceleration components of the X, Y and Z axes, and the gyroscope components of the X, Y and Z axes. Each component occupies two bytes and is represented as a 12-bit signed integer. Each 12-byte data read is sequentially stored in the FIFO in the memory storage module 8 for subsequent processing.

[0131] In a specific embodiment, the data fusion module 5 includes:

[0132] Error calculation: The Mahony algorithm uses a PI (proportional-integral) controller to combine the gravity direction provided by the accelerometer and the angular velocity of the gyroscope to calculate the attitude error. The error is applied to the gyroscope data through a feedback mechanism to correct the drift of the attitude solution to achieve an accurate attitude angle. Since the implementation of the Mahony algorithm on the FPGA needs to consider the problem of floating-point operations, and the hardware description language cannot directly process floating-point numbers or complex mathematical functions, the Mahony algorithm is implemented using fixed-point numbers and pipeline design methods. The accelerometer and gyroscope data are read from the memory storage module 8. Since the Mahony algorithm uses quaternions to represent the rotation state of an object, the quaternion is first initialized to a unit quaternion, specifically q0=1, q1=0, q2=0, q3=0. The expression of the quaternion is as follows:

[0133] q=[q0 q1 q2 q3]

[0134] Where q0 is the real part representing the angle of rotation, and q1, q2, q3 are the imaginary parts representing the direction of the rotation axis.

[0135] Integral term update: proportional gain coefficient K p Initialized to 2000, integral gain coefficient K i Initialized to 1, the quaternion is updated according to the sensor data and error feedback at each rising edge of the clock; when the module is reset, the difference between the accelerometer and the quaternion, that is, the attitude error, is first multiplied by K p Calculate the error term and pass k i Update the integral term; then update each component of the quaternion based on the updated integral term and the gyroscope data.

[0136] Yaw angle conversion: After obtaining the quaternion, the arc tangent value is obtained by using a lookup table, and the yaw angle θ is obtained by the following formula:

[0137] θ=atan2(2(q0q3+q1q2),1-2(q2 2 +q3 2 ))

[0138] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis;

[0139] Rotation matrix conversion: Each frame of the image is rotated slightly by constructing a rotation matrix. The rotation matrix R z The formula is as follows:

[0140]

[0141] Wherein, θ represents the yaw angle, and the values ​​of cosθ and sinθ are obtained by looking up the table. Each set of data is sequentially stored in the FIFO, and then output from the FIFO to the image processing module 6.

[0142] In a specific embodiment, the image processing module 6 includes:

[0143] Scaling and cropping: Scale and crop the video data to fit the size of the LVDS screen;

[0144] Black and white stripe area 2 division: By setting the border distance parameter border_width, the interval between the effective display area 1 and the screen border is controlled and the black and white stripe area 2 is determined; the calculation formulas of the left border coordinates, the right border coordinates, the upper border coordinates and the lower border coordinates of the effective display area are as follows:

[0145]

[0146]

[0147] Among them, h pixel is the horizontal resolution after scaling, h disp is the horizontal resolution of the display device, v pixel is the vertical resolution after scaling, v disp is the vertical resolution of the display device, the border distance parameter controls the distance between the effective display area and the screen border, the left border coordinate and the right border coordinate are the horizontal coordinates, and the upper border coordinate and the lower border coordinate are the vertical coordinates.

[0148] In a specific embodiment, a modulo operation is performed on the sum of a row counter h_cnt and a deflection factor stripe_pos to generate black and white stripes with a pixel interval of 20. When the time parameter cnt_hz accumulates to a threshold, the deflection factor is updated by the yaw angle collected on the rising edge. The increase or decrease of the deflection factor directly affects the movement of the black and white stripes. The range of the black and white stripes is controlled by the boundary distance parameter border_width.

[0149] Video rotation: Control the rotation of the effective display area 1 according to the size of the rotation matrix; use perspective transformation to process the rotation part, construct the rotation matrix to perform a small rotation on each frame of the image, set the threshold parameter, and when the rotation angle is between -5° and 5°, it is an invalid rotation. Use the bilinear interpolation method to calculate the pixel values ​​of the non-integer coordinates after rotation, and accelerate the overall calculation process through pipeline and parallel processing. The formula of the perspective transformation is as follows:

[0150]

[0151] Among them, (x, y) is the input pixel coordinates, (x′, y′) is the unnormalized pixel component after rotation, these projection points obtained by perspective transformation need to be divided by the normalization factor w in order to map back to the two-dimensional plane, the matrix element a is the coefficient that controls the scaling in the x direction, b is the coefficient that controls the beveling, c is the coefficient that controls the translation in the x direction, d is the coefficient that controls the rotation, e is the coefficient that controls the scaling in the y direction, f is the coefficient that controls the translation in the y direction, w is the normalization factor in homogeneous coordinates, g controls the deformation degree of perspective transformation in the horizontal direction, and h controls the deformation degree of perspective transformation in the vertical direction. In order to produce a certain displacement effect, the values ​​of the parameters c and f of the perspective transformation matrix are dynamically adjusted according to the size of the rotation angle. In order to make it more in line with the visual habits of the human eye during rotation, the parameters g and h are dynamically adjusted to flexibly change the scaling effect of the object in the video in terms of viewing angle, where the relationship between a, b, d and e and θ is expressed as:

[0152]

[0153] Among them, θ represents the yaw angle. By adopting the perspective transformation method, each frame of the image can be rotated slightly by constructing a rotation matrix. In order to take into account the impact of the vehicle tilting slightly left or right in real situations, which causes the image to shake, a threshold parameter is set. Only when the rotation angle is large enough is it considered a valid rotation. The rotation matrix is ​​composed of pre-calculated cosθ and sinθ values, and the pixel coordinates are transformed with the multiplication and addition operations of the FPGA. The bilinear interpolation method is used to calculate the pixel values ​​of non-integer coordinates after rotation, which can improve the accuracy; the overall calculation process is accelerated by pipeline and parallel processing, which can ensure that the image can complete the rotation effect in real time and smoothly.

[0154] Black dot drawing: black dots are set around the effective display area 1, and the black dots move up and down with the Y-axis value of the accelerometer for visual compensation; three black dots are added on both sides of the effective display area 1, and the black dots make corresponding visual compensation according to the acceleration value of the vehicle in the forward or backward direction. When the acceleration value of the vehicle accelerating forward or backward is greater than the set threshold, the black dots move with a corresponding amplitude according to the magnitude of the acceleration; the value of the accelerometer Y-axis data stored in the memory is processed, and the initial position of the black dots is set to the left and right sides of the display area. The moving distance of the black dots is related to the value of the Y-axis of the accelerometer. The addition of black dots improves the display effect without affecting the viewing effect, ensuring the consistency of visual information, and passengers can get a better anti-motion sickness experience.

[0155] In a specific embodiment, the video output module 7 includes display timing generation and parallel-serial conversion into LVDS differential signals. The video after rotation and striped background and black dot processing is transcoded and processed into an LVDS signal, and finally output to the LVDS display, providing accurate visual compensation effects and reducing the risk of motion sickness in passengers.

[0156] Optionally, it also includes a sensor, a display device, a memory controller, and a memory storage module 8 and an EDID reading module arranged in the FPGA, the memory storage module 8 includes a read cache unit and a write cache unit, the data acquisition module 4 stores the read data into a temporary storage area in the memory storage module 8, and the data fusion module 5 stores the data into the temporary storage area and outputs it to the image processing module 6.

[0157] In a specific embodiment, the sensor adopts an MPU6050 sensor, the display device adopts an LVDS display, the memory controller adopts a DDR3 controller, the data acquisition module 4 stores the read data into the FIFO memory in the memory storage module 8, the data fusion module 5 stores the data into the FIFO memory, and then outputs the data from the FIFO memory to the image processing module 6.

[0158] Figure 6 is a flow chart of dynamic background stripe video processing according to a specific embodiment of the present invention, such as Figure 6 As shown, after the operation starts, the video on the PC is input to the FPGA, and the data acquisition module 4 collects data, and the yaw angle is obtained through Mahony complementary filtering, which is processed in the FPGA. Then, it is determined whether the yaw angle is within the range of -5° to 5°, that is, whether it is a valid rotation, and black and white stripes and black dots are added. At the same time, the video is rotated and finally output to the LVDS display to relieve the passengers' motion sickness reaction, ending a running process.

[0159] The present invention provides a method and system for alleviating motion sickness based on dynamic background image enhancement of a display screen. Different from the traditional solution to motion sickness, this proposal collects the acceleration and posture data of the vehicle through sensors, and combines the Mahony complementary filtering algorithm to perform real-time fusion processing on these data. The processing of video images includes display size adjustment and black and white stripe generation, anti-shake and high-precision real-time adjustment rotation processing, and black dot assistance for acceleration and deceleration images. Finally, by processing the input display image, the dynamic stripe background and visual signals are output to the display screen, and the visual perception of the passenger is synchronized with the perception of the inner ear balance system in the form of visual feedback, thereby reducing sensory conflicts, significantly alleviating the passenger's motion sickness, and making the passenger more comfortable during driving; by early warning of steering changes, passengers can better adapt to vehicle movement, thereby reducing the probability of motion sickness; in addition, the dynamically adjusted display effect can improve the consistency of visual information and enhance the passenger's riding experience.

[0160] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for alleviating motion sickness based on display screen dynamic background image enhancement, characterized in that: The following steps are involved: Data fusion: using a data fusion algorithm to process the acquired acceleration and angular velocity data, obtaining attitude angle data after processing, and obtaining a rotation matrix based on the attitude angle data; Image processing: Calculate boundary coordinates according to the obtained video signal, determine the screen boundary according to the boundary coordinates, control the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, determine the black and white stripe area according to the effective display area and the screen boundary, control the left and right movement of the black and white stripes according to the yaw angle in the attitude angle data, and control the rotation of the effective display area according to the rotation matrix. A plurality of black dots are arranged around the effective display area, and the black dots move up and down with the value of acceleration for visual compensation; Video output: The video signal is transcoded and processed into a format suitable for a display device and transmitted to the display device, thereby alleviating motion sickness by introducing visual cues to passengers in the vehicle.

2. The method for alleviating motion sickness based on display screen dynamic background image enhancement according to claim 1, characterized in that: The following steps are also included: Video decoding: The processing unit receives a video signal from a video source through an interface and decodes the video signal into image data and a synchronization signal; Data acquisition: the processing unit reads the acceleration and angular velocity data from the sensor through an interface, and stores the acquired acceleration and angular velocity data in a memory; The data fusion further includes: the processing unit acquiring the acceleration and the angular velocity data from the memory; The image processing further includes: processing the decoded image data to adapt to the size of the display device through cropping and scaling.

3. The method for alleviating motion sickness based on display screen dynamic background image enhancement according to claim 1, characterized in that: The data fusion specifically includes: initializing the quaternion to q0=1, q1=0, q2=0, q3=0, and the expression of the quaternion is as follows: q=[q0 q1 q2 q3] Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis, and the proportional gain coefficient K in the data fusion algorithm is p and integral gain coefficient K i Initialization, whenever new sensor data is received, the quaternion is updated according to the sensor data and error feedback; The yaw angle θ is obtained by the following formula: <h2 style=";text-align:left;direction:ltr">θ = atan2(2(q0q3+q1q2),1-2(q2<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +q3<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> )) Where q0 is the real part representing the angle of rotation, q1, q2, q3 are the imaginary parts representing the direction of the rotation axis, The rotation matrix R z The formula is as follows: Here, θ represents the yaw angle.

4. The method for alleviating motion sickness based on display screen dynamic background image enhancement according to claim 1, characterized in that: The image processing specifically includes: the calculation formulas of the left boundary coordinates, the right boundary coordinates, the upper boundary coordinates and the lower boundary coordinates of the effective display area are as follows: Among them, h pixel is the horizontal resolution after scaling, h disp is the horizontal resolution of the display device, v pixel is the vertical resolution after scaling, v disp is the vertical resolution of the display device, and the border distance parameter controls the distance between the effective display area and the screen border; By performing a modulo operation on the sum of the display device row counter and the deflection factor, black and white stripes with a fixed pixel interval are generated. When the time parameter accumulates to a threshold, the deflection factor is updated by the yaw angle collected by the sensor. The increase or decrease of the deflection factor affects the movement of the black and white stripes. The black and white stripe area is controlled by the parameter of the distance between the effective display area and the screen boundary.

5. The method for alleviating motion sickness based on display screen dynamic background image enhancement according to claim 1, characterized in that: The image processing specifically includes: processing the rotation part by perspective transformation, performing small-amplitude rotation processing on each frame of the image by constructing the rotation matrix, setting threshold parameters, and when the rotation angle is between -5° and 5°, it is invalid rotation, and using bilinear interpolation method to calculate the pixel value of non-integer coordinates after rotation, and accelerating the overall calculation process by pipeline and parallel processing. The formula of perspective transformation is as follows: Among them, (x, y) is the input pixel coordinates, (x′, y′) is the unnormalized pixel component after rotation, the matrix element a is the coefficient that controls the scaling in the x direction, b is the coefficient that controls the beveling, c is the coefficient that controls the translation in the x direction, d is the coefficient that controls the rotation, e is the coefficient that controls the scaling in the y direction, f is the coefficient that controls the translation in the y direction, w is the normalization factor in the homogeneous coordinates, g controls the deformation degree of the perspective transformation in the horizontal direction, and h controls the deformation degree of the perspective transformation in the vertical direction. The relationship between a, b, d and e and θ is expressed as: Here, θ represents the yaw angle.

6. The method for alleviating motion sickness based on display screen dynamic background image enhancement according to claim 1, characterized in that: The image processing specifically includes: adding three black dots on both sides of the effective display area, and the black dots make corresponding visual compensation according to the acceleration value of the vehicle in the forward or backward direction. When the acceleration value of the vehicle accelerating forward or backward is greater than the set threshold, the black dots move with a corresponding amplitude according to the magnitude of the acceleration.

7. A system based on display screen dynamic background image enhancement, characterized in that: The invention comprises: Data fusion module: using the data fusion algorithm to process the acquired acceleration and angular velocity data, obtaining attitude angle data after processing, and obtaining the rotation matrix according to the attitude angle data; Image processing module: calculate boundary coordinates according to the obtained video signal, determine the screen boundary according to the boundary coordinates, control the size of the effective display area by adjusting the interval between the effective display area and the screen boundary, determine the black and white stripe area according to the effective display area and the screen boundary, control the left and right movement of the black and white stripes according to the yaw angle in the attitude angle data, and control the rotation of the effective display area according to the rotation matrix. A plurality of black dots are arranged around the effective display area, and the black dots move up and down with the value of acceleration for visual compensation; Video output module: transcodes and processes the video signal into a format suitable for a display device, transmits it to the display device, and alleviates motion sickness by introducing visual cues to passengers in the car.

8. The system based on display screen dynamic background image enhancement according to claim 7 is characterized in that: Also included is a component disposed in the processing unit: Video decoding module: receiving a video signal from a video source through an interface, and decoding the video signal into image data and a synchronization signal; Data acquisition module: reads the acceleration and angular velocity data from the sensor through an interface, and stores the acquired acceleration and angular velocity data in a memory; The data fusion module further comprises: acquiring the acceleration and the angular velocity data from the memory; The image processing module further includes: processing the decoded image data through cropping and scaling to adapt the size of the display device.

9. The system based on display screen dynamic background image enhancement according to claim 8, characterized in that: The video decoding module comprises: Serial-to-parallel conversion unit: converts the input serial differential signal into serial single-ended data, and then converts it into fixed-width parallel data; A word alignment calibration unit: comparing the parallel data with a control character, and when the control character is detected continuously in a plurality of delay values, selecting an intermediate value as a final calibration delay; Data synchronization unit: performs synchronous processing on the image data of multiple channels; The encoding and decoding unit decodes the parallel data from the high-order encoding format into low-order data, and generates a line field synchronization signal and a data valid enable signal at the same time.

10. A computer program product, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images