Display control system and method

Abstract

The present application relates to a display control method and system for on-screen content on a vehicle screen. The display control method comprises accepting multi-dimensional motion state information of a vehicle, obtaining control information of the vehicle, obtaining eye gaze information; and determining a display position of the on-screen content on the vehicle screen based on the multi-dimensional motion state information, the control information and the eye gaze information.

Description

Technical Field

[0001] This invention relates to display control, and more particularly to content display control of in-vehicle displays. Background Technology

[0002] Currently, all vehicles are equipped with displays, such as the central control display, which plays an important role in vehicle operation and control. For example, it displays vehicle driving parameters and navigation data. Vehicle users frequently interact with the central control display to access this information or control the vehicle, such as setting navigation and operating entertainment applications. However, a common problem is that when the vehicle encounters bumps, the central control screen shakes with the vehicle, leading to adverse experiences for the driver or passengers when viewing or operating the screen, including visual fatigue, distraction, increased reading difficulty, motion sickness, and operational errors. To mitigate these effects, some vehicle manufacturers have designed central control screen brackets with better shock absorption to cushion vehicle bumps and reduce their amplitude. However, while installing shock-absorbing brackets can solve the screen shaking problem to some extent, it also has some potential drawbacks, such as increased cost, space limitations, and increased maintenance difficulty. Furthermore, the cushioning performance of these brackets is limited; although they can reduce shaking to some extent, they may not completely eliminate the adverse effects of screen shaking under extremely bumpy road conditions.

[0003] In addition, existing technologies have proposed electronic image stabilization solutions that can determine the amount of screen content movement compensation by detecting the vehicle's motion information. However, this solution only considers the vehicle's motion state when determining screen movement compensation, so it cannot effectively eliminate the impact of screen shaking caused by bumps or swaying, resulting in a poor user experience. Summary of the Invention

[0004] This paper proposes a scheme to compensate for the motion of the central control screen content by comprehensively considering vehicle motion information, the driver's control status of the vehicle, and the driver's line of sight. This scheme aims to reduce the shaking of the screen content relative to the driver and improve the user experience.

[0005] According to one aspect of the present invention, a display control system for a vehicle screen is provided, comprising: a plurality of sensors for detecting multidimensional motion state information of the vehicle; an image sensor for capturing eye gaze information of the vehicle driver on the vehicle screen; and a controller configured to adjust the display position of screen content on the vehicle screen based on the multidimensional motion state information, vehicle control information, and the gaze information.

[0006] According to another aspect of the present invention, a method for controlling the display of content on a vehicle screen is provided, comprising: acquiring multidimensional motion state information of the vehicle; acquiring control information of the vehicle; acquiring eye gaze information of the driver of the vehicle; and adjusting the display position of the content on the vehicle screen based on the multidimensional motion state information, control information, and eye gaze information. Attached Figure Description

[0007] Figure 1 A schematic diagram illustrating the relative positions of the driver and the central control screen;

[0008] Figure 2 A schematic diagram of a screen display control system according to such an embodiment is shown;

[0009] Figure 3 A coordinate system diagram of the central control area and the central control screen is shown; and

[0010] Figure 4 A method for compensating for screen display content shaking is shown. Detailed Implementation

[0011] Before explaining any embodiments of the invention in detail, it should be understood that the application of the invention is not limited to the construction details set forth in the following description or shown in the accompanying drawings. The invention can have other embodiments and can be practiced or implemented in various ways.

[0012] During vehicle operation, it was observed that the shaking of the content displayed on the central control screen was directly affected by the vehicle's own motion. For example, driving on uneven roads inevitably prevents the vehicle from maintaining a stable posture. Furthermore, the shaking was also influenced by the vehicle's control. For instance, when braking, the front of the car might tilt forward, causing the screen to shift downwards. Similarly, when turning, the driver's line of sight relative to the screen might shift horizontally or vertically. Additionally, it was found that even when the driver experiences bumps, they also shake along with the vehicle, but the two movements are not perfectly synchronized. Figure 1 A schematic diagram illustrating the relative positions of the driver and the central control screen. (For example...) Figure 1 As shown, when the vehicle experiences bumps, the amplitude of the screen's vertical movement H1 is not always the same as the amplitude of the car seat's vertical movement H2. Therefore, it still causes relative eye movement with respect to the screen, resulting in discomfort. This invention proposes to adjust the display position of the content on the vehicle screen by accurately detecting the vehicle's motion state and combining it with real-time vehicle control information, while also considering changes in eye line of sight. This reduces or eliminates the relative eye movement of the central control screen content during driving, thereby improving the user experience.

[0013] Therefore, according to one embodiment of the present invention, a scheme is proposed to adjust the screen display content by comprehensively considering vehicle motion information, control state information, and eye gaze information. Figure 2 A schematic diagram of a display control system according to such an embodiment is shown. As shown, the display control system 100 includes a plurality of sensors 101, an image sensor 102, and a controller 103.

[0014] Here, sensor 101 is used to detect the multi-dimensional motion state information of the vehicle. MOV The motion state information here includes, for example, acceleration related to vehicle body sway, and is expressed as multidimensional motion state parameters or signals P. MOV The output can be in the form of a wireless signal, for example, sent to controller 103. Considering that vehicle body sway may occur in any dimension of three-dimensional space, multiple acceleration sensors can be set up and installed at different key locations on the vehicle to measure acceleration in different directions at different parts of the vehicle body. As an example, for instance... Figure 2 As shown, five acceleration sensors 1011-1015 can be provided, respectively arranged at the four corners and the center point of the vehicle, to ensure comprehensive capture of the vehicle's dynamic response acceleration under various driving conditions. Using the acceleration sensors 1011-1015, the acceleration changes of the vehicle in three directions—lateral, longitudinal, and vertical (hereinafter represented by the x, y, and z axes)—can be monitored in real time, thereby enabling the detection of the vehicle's vertical bumps, lateral swaying, and front-rear impacts. For ease of explanation, the motion state parameter P is referred to here. MOV Represented as:

[0015]

[0016] In the above formula, the numbers 1, 2, 3, 4, and 5 in the subscript of acceleration a indicate the corresponding accelerations in the x, y, and z axes detected at the four corners and the center point of the vehicle, respectively.

[0017] Image sensor 102 can capture the passenger's eye movements, for example, it can have a high frame rate to accurately track rapid eye movements, thereby detecting the driver's gaze information on the central control screen, such as its focus or focal point SP on the screen, such as... Figure 1 As shown. In one example, an eye-tracking algorithm STA can be integrated into the image sensor 102 to perform eye tracking, head pose estimation, etc., to determine the point SP where the passenger's gaze falls on the in-vehicle screen. This can be achieved using a known eye-tracking algorithm STA from the prior art, which calculates the gaze point SP on the central control screen by processing the real-time video stream from the camera 102, and outputs the two-dimensional coordinates (X, Y, X) of this point in the central control console space. eye ,Y eye ).

[0018] Controller 103 is used to acquire vehicle motion state information P from acceleration sensors 1011-1015. MOV and the landing point coordinates SP(X) eye ,Y eye In one example, to remove noise and retain the signal of the vehicle's actual motion, the controller 103 processes the acquired motion state data P. MOV Perform data preprocessing operations, including using a low-pass filter on the raw data P. MOV Filtering is performed, and to facilitate subsequent standardization, the filtered motion parameters P are... MOV Perform data normalization to convert the acceleration data into a standard format. For example, the collected acceleration 'a' can be normalized using the following formula:

[0019]

[0020] in:

[0021] 'a' represents the raw acceleration data collected, including accelerations along the x, y, and z axes, such as those at the four corners and center of the vehicle. 1x ,a 1y ,a 1z a 2x …,a 5y ,a 5z ;

[0022] a norm These are normalized acceleration data.

[0023] a min and a max These are the minimum and maximum values ​​of acceleration, respectively. It should be noted that a... min and a max It can be determined from the mixed dataset of accelerations in the x, y, and z axes, that is:

[0024] a min =Min(a) 1x ,a 1y ,a 1z a 2x …,a 5y ,a 5z );

[0025] a max =Max(a 1x ,a 1y ,a 1z a 2x …,a 5y ,a 5z ).

[0026] Or in another implementation, a min and a max Alternatively, the data can be determined separately for each of the x, y, and z axes, and used for normalizing the acceleration data in each of those axes. For example, for normalizing the acceleration in the x-axis, the acceleration a in the x-axis can be determined as follows: min and a max :

[0027] a min =Min(a) 1x ,a 2x ,a 3x ,a 4x ,a 5x );

[0028] a max =Max(a 1x ,a 2x ,a 3x ,a 4x ,a 5x ).

[0029] For a in the Y and Z axis directions min and a max The method for determining the X direction can be similar to that described above.

[0030] Based on the above processing, the normalized motion parameter P can be obtained. MOV ′={a norm}

[0031] Furthermore, considering the landing point coordinates SP(X) eye ,Y eye SP(x) represents the coordinates on the plane of the entire central control area. Controller 103 needs to perform coordinate transformation to convert the landing point coordinates into coordinates in the central control display screen's spatial coordinate system, hereinafter represented as SP(x). eye ,y eye This represents the driver's focal point on the central control screen. Since the relative positions of the central control display screen within the entire central control area are fixed, such as... Figure 3 As shown, XY represents the coordinate system established based on the central control area CCPAnel, while xy represents the coordinate system established based on the central control screen. Therefore, a mapping relationship can be established between the central control area coordinate system XY and the central control screen coordinate system xy. For example, when the reference point O of the central control screen (e.g., the lower left position of the screen) is taken as the origin of the xy coordinate system and its coordinates in the XY coordinate system are (X0, Y0), the coordinates of the landing point SP in the xy coordinate system can be calculated as follows:

[0032] xeye =X eye -X0

[0033] y eye =Y eye -Y0

[0034] In addition to determining the focal point position SP(x) eye ,y eye and normalized motion parameters P MOV ′={a norm The controller 103 also determines the controls exerted on the vehicle by the driver, which can be achieved by reading multiple control information P from the vehicle's onboard bus, such as the CAN bus. CTL To achieve this, these control information include control parameters such as steering angle θ, throttle position A, and brake pressure B. For example, Figure 2 As shown, the controller can read this control information P from devices such as the vehicle's built-in electronic control unit (ECU). CTL As another example, controller 103 can also be configured to acquire control information P, such as steering angle θ, throttle position A, and brake pressure B, from control signals transmitted from onboard steering angle sensors, throttle sensors, and brake sensors. CTL .

[0035] Therefore, the controller 103 is based on the multi-dimensional motion state information P MOV Control information P CTL and eye gaze information SP(x) eye ,y eye The controller 103 determines the display position of the content on the vehicle screen. According to one example of the invention, the controller 103 uses a trained neural network model TNN1 to process this information to estimate the display position compensation amount Δx for the content on the screen. screen Δy screen , where Δx screen Δy represents the amount of screen movement offset or compensation in the x-direction, while Δy represents the amount of screen movement offset or compensation in the x-direction. screen This represents the amount of screen movement offset or compensation in the y-direction. According to an example of the invention, model TNN1 can be a multi-input neural network model whose inputs include:

[0036] ●Vehicle control information P CTL This includes the steering angle θ(t), throttle position A(t), and brake pressure B(t) at the current time t;

[0037] ●Gaze focus position data: SP(x) eye ,y eye ),

[0038] ●Vehicle motion parameter P MOV′:a x (t),a y (t),a z (t),

[0039] Among them, a x (t),a y (t),a z (t) represents all acceleration parameters of the vehicle in the x, y, and z axes measured at the current time t, including accelerations in different directions at different parts of the vehicle as mentioned earlier. In this example, a x (t),a y (t),a z (t) represents the normalized accelerations a. norm .

[0040] According to one embodiment of the present invention, a deep learning neural network model TNN1 can be trained to simultaneously learn the adjustment parameter Δx of the display screen. screen Δy screen That is, (Δx) screen ,Δy screen )=

[0041] F model (a x (t),a y (t),a z (t),θ(t),A(t),B(t),x eye ,y eye ).

[0042] Where F model This represents the display control model TNN1, trained using deep learning. The adjustment parameters (Δx) of the screen are then estimated using the deep learning model TNN1. screen Δy screen Furthermore, the display position of the content on the central control screen can be dynamically adjusted using the following formula:

[0043] x adjust (t)=x screen (t-1)+Δx screen

[0044] y adjust (t)=y screen (t-1)+Δy screen

[0045] Where x screen (t-1) and y screen (t-1) represents the x and y axis positions of the screen content before adjustment, while x adjust With y adjustThis represents the x and y axis positions of the adjusted screen content applied to compensate for vehicle sway, thereby effectively counteracting visual sway of the vehicle screen caused by various possible factors.

[0046] In another example of the present invention, the display position compensation amount of the display screen is the adjustment parameter (Δx). screen Δy screen Alternatively, different machine learning models can be used to learn the display position compensation amount Δx separately. screen Δy screen ,Right now

[0047] Δx screen =f model (a x (t),a y (t),a z (t),θ(t),A(t),B(t),x eye ,y eye )

[0048] Δy screen =g model (a x (t),a y (t),a z (t),θ(t),A(t),B(t),x eye ,y eye )

[0049] Where f model ,g model These represent different machine learning models that have been trained. Using these different models avoids the limitations of a single model, and model f... model ,g model Different model architectures can be used for implementation, such as model f model An MLP neural network is used, and the model g model Training with deep learning networks such as convolutional neural networks (CNNs) avoids the degradation of computational accuracy caused by a single model, thereby improving Δx. screen with Δy screen The accuracy of the calculation.

[0050] Furthermore, in the example above, all vehicle motion parameters P are included. MOV ′、Control parameter P CTL and the fixation focus position parameter SP(x) eye ,y eye(a) is used as the input to the machine learning model TNN; in other examples of this invention, the model TNN can also be implemented with fewer inputs. In this case, the model input can be reduced by feature fusion, for example, all acceleration parameters measured at time t, such as (a) 1x ,a 1y ,a 1z a 2x …,a 5y ,a 5z After feature vector transformation, vector fusion processing is performed to generate a combined vector, which serves as an input to the display control model TNN, and is then combined with other inputs P. CTL and SP(x) eye ,y eye To predict the offset.

[0051] In another example of the invention, to further enhance the user experience and avoid abrupt screen jumps caused by adjustments, a smoothing factor is used to smooth the adjustment process. For this purpose, as an example, a smoothing factor γ can be set to achieve a smoothing effect from the current screen display position (x) before adjustment. screen (t-1),y screen (t-1) to the final display position after adjustment (x) adjust (t),y adjust The smooth transition of (t) can be calculated using the following formula:

[0052] γ·x screen (t-1)+(1-γ)·x adjust (t)→x adjust (t)

[0053] γ·y screen (t-1)+(1-γ)·y adjust (t)→y adjust (t)

[0054] Here, the value of γ is typically between 0 and 1. Therefore, the screen position update process can be made smoother by using the aforementioned smoothing factor γ, avoiding frequent jumps.

[0055] It should be noted that this invention is not limited to the method for determining the display position compensation of the screen content exemplified above. For example, considering that the focal position of the driver's eyes is dynamically changing, the display position (x) of the screen content can also be determined based on multi-dimensional motion state information and control information. adjust (t),y adjust(t)) to keep the displayed content constant relative to the dynamically changing gaze focus position. For this purpose, in another example, another deep learning neural network model, TNN2, can be trained on the multidimensional motion state information P. MOV ′ and control information P CTL The system learns to determine the initial position compensation amount, for example, denoted as Δx1 and Δy1. The controller then analyzes the change in gaze focus position due to changes in the driver's line of sight, for example, ΔSx = x eye (t)-x eye (t-1),ΔSy=y eye (t)-y eye (t-1), and adjust the first position compensation amount Δx1 and Δy1 in the following manner, for example, to generate the display position compensation amount (Δx screen Δy screen ),in

[0056] Δx screen =Δx1 - ΔSx,

[0057] Δy screen =Δy1-ΔSy

[0058] Therefore, the aforementioned method can be used based on the (Δx) calculated above. screen Δy screen To generate and adjust the display position (x) adjust (t),y adjust (t)).

[0059] Therefore, according to the present invention, the controller 103 continuously monitors the vehicle's motion state, such as acceleration a, and vehicle control information, such as steering angle θ, throttle position A, and brake pressure B, and updates x when a change is detected. screen (t) and y screen (t). According to one example of the present invention, in order to reduce the computational load of models TNN1 and TNN2, the controller 103 can process only the changed information when calling model TNN1 or TNN2, while ignoring other unchanged information. For example, when only the vehicle's throttle parameter A and steering angle θ change, while other control information, motion information, and eye gaze information remain unchanged, the controller 103 can set the unchanged information to a predetermined value (e.g., zero) so that model TNN only predicts the possible impact (Δx) of throttle parameter A and steering angle θ on screen shaking. screen Δy screen Therefore, controller 103 calculates the updated result x. screen (t) and y screen (t).

[0060] According to another embodiment of the present invention, x can also be updated for controller 103. screen (t) and y screen (t) Set conditions to prevent controller 103 from calculating and updating x when parameter changes are slight. screen (t) and y screen (t), thus the display position update is only triggered when the controller confirms that the preset conditions are met. For example, detecting the vehicle acceleration a. x ,a y ,a z Does it exceed the acceleration threshold a? T Here, the same or different acceleration thresholds can be set for the x, y, and z directions. When the controller 103 determines that one or all accelerations 'a' in the x, y, and z directions exceed the corresponding acceleration threshold 'a', the system will automatically adjust the acceleration threshold accordingly. T When the controller 103 determines that the detected acceleration a exceeds a certain threshold, the method disclosed in this invention is executed to adjust the display position of the content on the vehicle screen. Alternatively, it determines whether the control information exceeds a control threshold, such as whether the steering angle θ exceeds a steering angle threshold, whether the throttle position A is too large and exceeds a throttle opening threshold, and whether the brake pressure B exceeds a pressure threshold. When it is determined that the control information exceeds the control threshold, the method disclosed in this invention is executed to adjust the display position of the content on the vehicle screen. Thus, according to different embodiments of this invention, only when the controller 103 determines that the detected acceleration a exceeds a certain threshold, the display position of the content on the vehicle screen is adjusted. x ,a y ,a z And the update parameters are only processed to generate the update position x when any one or more of the parameters θ, A, B, etc., trigger the corresponding threshold condition. screen With y screen .

[0061] According to the present invention, by using an onboard accelerometer to accurately sense the vehicle's movement, such as capturing the up-and-down vibration and left-and-right or front-and-back swaying signals of the vehicle as described above, and combining this with the control information generated when the vehicle performs dynamic operations such as sharp turns, sudden acceleration, or emergency braking, and further combining this with the identified level of the person's eyes and the screen, as well as the point where the eyes' gaze falls on the screen, the shaking of the content on the vehicle's central control screen during driving can be significantly reduced.

[0062] Figure 4 This paper illustrates a control method for controlling the display content on a vehicle-mounted display screen to compensate for possible shaking, according to an embodiment of the present invention. In step 401, the multi-dimensional motion state information P of the vehicle is detected. MOV For example, by installing acceleration sensors at different key locations on the vehicle, the lateral, longitudinal, and vertical acceleration of different parts of the vehicle body can be detected, thereby enabling the detection of the vehicle's vertical bumps, lateral swaying, and front-to-back impacts. For instance, in... Figure 1The figure shows the multidimensional acceleration data P that can be collected when five acceleration sensors (1011-1015) are installed. MOV as follows:

[0063]

[0064] In step 403, the driver's eye movements are captured by the image sensor 102 to detect the point SP where the driver's gaze falls on the central control screen, which serves as an eye fixation parameter. Here, a known gaze-tracking algorithm can be used to detect the point SP and output the two-dimensional coordinates (X, Y, X) of this point in the central control console space area. eye ,Y eye ).

[0065] In step 405, the control exerted by the driver on the vehicle is determined, for example by reading multiple control parameters P of the vehicle at the current time t from the onboard CAN bus. CTL These control parameters include, for example, steering angle θ, throttle position A, and brake pressure B.

[0066] In step 407, the collected acceleration data P MOV Perform data preprocessing operations, including using a low-pass filter on the raw data P. MOV Filtering is performed, and to facilitate subsequent standardization, the filtered information P is... MOV Perform data normalization to convert acceleration data into a standard format. As an example, normalization can be performed using the following formula:

[0067]

[0068] This generates the normalized motion parameters P. MOV ′={a norm}

[0069] In step 409, a coordinate transformation is performed on the coordinates of the point of view SP to determine the coordinates (x, y) of the point of view SP in the planar coordinate system of the central control display screen. eye ,y eye This represents the driver's gaze focus position on the central control screen, and in this example, it is also referred to as the eye gaze parameter.

[0070] In step 411, based on the multidimensional motion state information P MOV Control information P CTL and eye gaze information SP(x) eye ,y eyeThe display position of the content on the vehicle screen is determined. As mentioned earlier, as an example, one or more display compensation models (TNN1) trained by deep learning are used to process this information to estimate the display position compensation amount (Δx) of the content. screen Δy screen Following another example, another neural network model, TNN2, can also be trained on P. MOV Control information P CTL The process determines the first position compensation amounts Δx1 and Δy1; then, it adjusts the first position compensation amounts using the dynamic changes (ΔSx, ΔSy) of the gaze focus position to generate the display position compensation amount (Δx1, Δy1). screen Δy screen Then proceed to step 413.

[0071] In step 413, based on the display position compensation amount Δx screen Δy screen To update the display position of each pixel on the screen content (x adjust (t), y adjust (t)). In one example, the update positions can be determined directly based on the following formula:

[0072] x adjust (t)=x screen (t-1)+Δx screen

[0073] y adjust (t)=y screen (t-1)+Δy screen .

[0074] Therefore, controller 103 controls the central control display to update the position (x) adjust (t), y adjust (t) is used to display the individual pixels of the screen content.

[0075] In another example, to avoid abrupt screen jumps caused by adjustments, a smoothing factor is used to smooth the adjustment process. Therefore, in step 413, a smoothing factor γ can be set to adjust the position of the displayed content from its original screen position (x) at time t-1. screen (t-1), y screen (t-1) to the new display position at time t (x) adjust (t),y adjust The smooth transition of (t) can be calculated using the following formula:

[0076] γ·x screen (t-1)+(1-γ)·x adjust (t)→x adjust(t)

[0077] γ·y screen (t-1)+(1-γ)·y adjust (t)→y adjust (t)

[0078] Here, the value of γ is typically between 0 and 1. Therefore, the screen position update process can be made smoother by using the aforementioned smoothing factor γ, avoiding frequent jumps.

[0079] In another embodiment of the invention, when changes are detected in acceleration sensor data a and vehicle control information such as steering angle θ, throttle position A, brake pressure B, and changes in eye focus, updates x are avoided when parameter changes are minor. screen (t) and y screen (t) To save computational costs, etc., it also includes step 406, which determines whether the parameter changes meet predetermined conditions, such as acceleration a in the lateral, longitudinal, and vertical directions. x ,a y ,a z Set acceleration thresholds, and set corresponding throttle opening thresholds, turning angle thresholds, and brake pressure thresholds for control parameters θ, A, B, etc. Specifically, in step 406, when the detected acceleration a is determined... x ,a y ,a z The subsequent steps 407-413 to update the position x are only executed when any one or more of the parameters θ, A, B, etc., trigger the corresponding threshold condition. screen With y screen It should be noted here that, in Figure 4 In the described process, not every step or the order in which steps are executed is necessary. For example, step 407 is not necessary when the vehicle's acceleration change is detected to be less than a threshold condition.

[0080] The above example illustrates the screen adjustment method of this invention using the content on a central control screen as an example. Such a central control screen can be used in vehicle entertainment and information systems. When performing applications such as music playback, video playback, and internet surfing, the dynamic screen adjustment technology of this invention is employed to reduce screen content shaking during driving, improving passenger comfort and entertainment experience, and reducing discomfort caused by visual shaking, such as visual fatigue and motion sickness. Of course, this invention is not limited to the central control screen; for example, it can also be used on a digital instrument panel located in front of the driver. By utilizing the dynamic screen adjustment technology of this invention, key information (such as speed, RPM, and navigation information) can be clearly displayed even on bumpy roads, helping the driver to better concentrate, thereby improving driving safety and reducing the risk of accidents during driving.

[0081] While exemplary display control methods and systems of the present invention have been described above with reference to specific examples, it is understood that the method steps and functions described herein can be implemented as electronic hardware, computer software, modules, or combinations thereof. Furthermore, another embodiment of the present invention provides a machine-readable medium storing machine-readable instructions, which, when executed by a processor, cause the processor to perform any of the methods disclosed herein. This processor may be part of an in-vehicle infotainment system. Moreover, the present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art will understand that more embodiments of the present invention can be obtained by combining the different embodiments described above, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for controlling the display of content on a vehicle screen, comprising: Acquire multi-dimensional motion state information of the vehicle; Obtain the control information of the vehicle; Obtain the eye gaze information of the vehicle driver; The display position of the screen content on the vehicle screen is adjusted based on the multidimensional motion state information, control information, and eye gaze information.

2. The display control method as claimed in claim 1, wherein The multidimensional motion state information includes accelerations in different directions measured at different locations on the vehicle; The control information includes at least one of the following: steering angle, throttle position, and brake pressure; as well as Obtaining eye gaze information includes: Capture the driver's eye focus information on the vehicle screen; The landing point information is converted into the gaze focus position in the display coordinate system of the vehicle screen.

3. The display control method of claim 2, wherein the gaze focus position is dynamically changing; Adjusting the display position includes: The display position of the screen content is determined based on the multidimensional motion state information and control information so that it remains unchanged relative to the dynamically changing gaze focus position.

4. The display control method of claim 3, wherein determining the display position includes: The multidimensional motion state information and control information are processed using a trained first neural network model to generate a first position compensation amount; The first position compensation amount is adjusted by using the dynamic change in the gaze focus position to generate a display position compensation amount; as well as The display position of the screen content on the vehicle screen is determined based on the display position compensation amount.

5. The display control method of claim 3, wherein determining the display position includes: The multidimensional motion state information, control information, and gaze focus position are processed using a trained second neural network model to generate the display position compensation amount of the screen display content; The display position of the screen content on the vehicle screen is adjusted based on the display position compensation amount.

6. The display control method of claim 4 or 5, wherein adjusting the display position based on the display position compensation amount further comprises: A first display position is generated based on the aforementioned display position compensation amount; Obtain the previous display position of the screen content; A smoothing factor is used to process the first display position and the previous display position to generate a second display position; The screen content is displayed at the second display position.

7. The display control method of claim 6, further comprising: After displaying the screen content at the second display position, update information is received for at least one of the multidimensional motion state information, control information, and eye gaze information; The second display position is updated based on the at least one update information.

8. A display control system for a vehicle screen, comprising: Multiple sensors are used to detect the vehicle's multidimensional motion state information; An image sensor is used to capture the driver's gaze information on the vehicle screen. The controller is configured to adjust the display position of the screen content on the vehicle screen based on the multidimensional motion state information, the vehicle control information, and the gaze information.

9. The display control system of claim 7, wherein The multidimensional motion state information includes accelerations in different directions measured at different locations on the vehicle; The control information includes at least one of the following: steering angle, throttle position, and brake pressure; The eye gaze information includes the focus information of the driver's eye gaze on the vehicle screen; The controller is further configured to convert the focus information into gaze focus position data in the display coordinate system of the vehicle screen.

10. The display control system of claim 9, wherein the gaze focus position is dynamically changing. The controller is further configured as follows: The display position of the screen content is determined based on the multidimensional motion state information and control information so that it remains unchanged relative to the dynamically changing gaze focus position.

11. The display control system of claim 10, wherein the controller is further configured to: The multidimensional motion state information and control information are processed using a trained first neural network model to determine the first position compensation amount; The first position compensation amount is adjusted by the dynamic change in the gaze focus position to generate a display position compensation amount; The display position of the screen content on the vehicle screen is adjusted based on the display position compensation amount.

12. The display control system of claim 10, wherein the controller is further configured to: The multidimensional motion state information, gaze focus position data, and vehicle control information are processed using a trained second neural network model to generate the display position compensation amount of the screen display content. The display position of the screen content on the vehicle screen is adjusted based on the display position compensation amount.

13. The display control system of claim 11 or 12, wherein the controller is further configured to: A first display position is generated based on the aforementioned display position compensation amount; Obtain the previous display position of the screen content; A smoothing factor is used to process the first display position and the previous display position to generate a second display position; The screen content is displayed at the second display position.

14. The display control system of claim 13, wherein the controller is further configured to: After displaying the screen content at the second display position, update information of at least one of the multidimensional motion state information, control information, and gaze focus position is obtained; The second display position is updated based on the at least one update information.

15. The display control system of claim 11 or 12, wherein the controller is further configured to: Determine whether any of the following conditions are met: whether the acceleration exceeds an acceleration threshold, and whether the control information exceeds a control threshold; When the conditions are met, the display position of the content on the vehicle screen is adjusted.

16. A computer program product comprising a computer-readable program, wherein the program, when executed by one or more processors, causes the processors to perform the method of any one of claims 1-7.

Images