Content display method, device and system and storage medium

Abstract

The invention discloses a content display method, device and system and a storage medium. The method comprises the steps that the eyeball center position of a driver is acquired; determining a shielded area of an instrument display screen in the visual field of the driver according to the eyeball center position of the driver; determining compensation display content according to the content of the shielded area of the instrument display screen; and displaying the compensation display content in a steering wheel visual component. By adopting the scheme provided by the invention, a driver can completely obtain the instrument information which is possibly shielded originally, driving decision-making errors caused by information loss are avoided, meanwhile, the size of a steering wheel or a display area does not need to be changed, the probability of misoperation is reduced, and the safety of vehicle driving is improved.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and in particular to a content display method, device, system, and storage medium. Background Technology

[0002] In automotive driving scenarios, the steering wheel is a crucial control component, often made of opaque materials. The problem of the steering wheel obstructing the view of the instrument cluster has long been a prominent issue in automotive design. The instrument cluster is a key window for drivers to obtain real-time vehicle information, displaying important data such as speed, RPM, fuel level, and fault indicators. If the steering wheel obstructs the instrument cluster, the driver cannot obtain a timely and comprehensive understanding of the vehicle's status, significantly impacting the accuracy and timeliness of driving decisions, thus posing a serious threat to driving safety. However, a common problem in automotive design is that the steering wheel obstructs the view of the instrument cluster, making it difficult for drivers to monitor vehicle conditions in real time, potentially threatening driving safety. In the current technological context, engineers have attempted to balance steering wheel comfort with instrument cluster visibility by reducing the steering wheel diameter or eliminating the instrument cluster altogether, instead displaying relevant information on the central control screen. However, these approaches all affect drivers' existing operating habits and are equally detrimental to driving safety.

[0003] Therefore, how to provide a content display method to improve vehicle driving safety has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a content display method, apparatus, system, and storage medium to improve vehicle driving safety.

[0005] This application provides a content display method, including:

[0006] Obtain the center position of the driver's eyeball;

[0007] Determine the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball;

[0008] The compensation display content is determined based on the content of the obscured area of ​​the instrument display screen;

[0009] The compensation content is displayed on the steering wheel visualization component.

[0010] The beneficial effects of this application are as follows: By obtaining the center position of the driver's eyeball, and then accurately determining the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball, the compensation display content is determined based on the content of the obstructed area of ​​the instrument display screen, and the compensation display content is displayed on the steering wheel visualization component. Because this solution allows the driver to fully obtain the instrument information that might otherwise be obstructed, it avoids driving decision errors caused by missing information. At the same time, it does not require changing the steering wheel size or the display area, reducing the probability of operational errors and improving vehicle driving safety.

[0011] In one embodiment, obtaining the center position of the driver's eyeball includes:

[0012] Multiple two-dimensional image coordinates of the driver's eyes are obtained by multiple eye-tracking cameras at fixed positions, with each eye-tracking camera corresponding to one two-dimensional image coordinate.

[0013] Based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera, the two-dimensional image coordinates corresponding to each eye-tracking camera are converted into a gaze direction vector in three-dimensional space;

[0014] The gaze direction vector corresponding to each eye-tracking camera is represented as a spatial straight line equation;

[0015] Solve for the optimal intersection point of the equations of each spatial line, and use the optimal intersection point as the three-dimensional coordinates of the driver's eye center position.

[0016] In one embodiment, determining the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball includes:

[0017] Obtain the set of lines connecting the center position of the driver's eyeballs to the content displayed on the instrument panel display;

[0018] Determine the subset of lines in the set of lines that intersect with the steering wheel based on the steering wheel rotation angle;

[0019] The area where the intersection of the set of connecting lines and the instrument display screen is located is determined as the area where the instrument display screen is obscured in the driver's field of vision.

[0020] In one embodiment, displaying the compensation display content in the steering wheel visualization component includes:

[0021] Determine the display position of the compensation display content within the steering wheel visualization component;

[0022] Adjust the display size of the compensation content according to the display position and the size of the display position;

[0023] The compensation content after adjusting the display size will be displayed at the specified display position.

[0024] In one embodiment, determining the display position of the compensation display content within the steering wheel visualization component includes:

[0025] Determine the steering wheel rotation angle;

[0026] The position of the visual component on the steering wheel, located at the center of the driver's field of vision, is determined based on the steering wheel rotation angle.

[0027] The position of the steering wheel visualization component located at the center of the driver's field of vision is determined as the display position of the compensation display content in the steering wheel visualization component.

[0028] In one embodiment, displaying the compensation display content in the steering wheel visualization component includes:

[0029] Determine the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument display screen;

[0030] The compensation display content is displayed through the steering wheel visualization component, and the display angle of the compensation display content is adjusted so that the compensation display content is seamlessly spliced ​​with the unobstructed content of the instrument display screen in the driver's field of vision.

[0031] In one embodiment, the steering wheel rotation angle is determined as follows:

[0032] The rotation angle of the steering column is determined by an angle sensor, wherein the angle sensor is located at the rotation center axis for adjusting the up and down angle of the steering column.

[0033] The steering wheel rotation angle is determined based on the rotation angle of the steering wheel column.

[0034] This application also provides a content display device, including:

[0035] The acquisition module is used to acquire the center position of the driver's eyeball;

[0036] The first determining module is used to determine the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball;

[0037] The second determining module is used to determine the compensation display content based on the content of the obscured area of ​​the instrument display screen;

[0038] The display module is used to display the compensation display content in the steering wheel visualization component.

[0039] In one embodiment, the acquisition module includes:

[0040] The first acquisition submodule is used to acquire multiple two-dimensional image coordinates of the driver's eyes through multiple eye-tracking cameras at fixed positions, wherein each eye-tracking camera corresponds to one two-dimensional image coordinate.

[0041] The transformation submodule is used to convert the two-dimensional image coordinates corresponding to each eye-tracking camera into a gaze direction vector in three-dimensional space based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera.

[0042] The transformation submodule is also used to represent the gaze direction vector corresponding to each eye-tracking camera as a spatial straight line equation;

[0043] The solution submodule is used to solve for the optimal intersection point of each spatial line equation and use the optimal intersection point as the three-dimensional coordinates of the driver's eye center position.

[0044] In one embodiment, the first determining module includes:

[0045] The second acquisition submodule is used to acquire the set of lines connecting the center position of the driver's eyeballs and the content displayed on the instrument display screen;

[0046] The first determining submodule is used to determine the set of connecting lines that intersect with the steering wheel in the set of connecting lines based on the steering wheel rotation angle.

[0047] The second determining submodule is used to determine that the area where the intersection of the set of connecting lines and the instrument display screen is located is the obstructed area of ​​the instrument display screen in the driver's field of vision.

[0048] In one embodiment, the display module includes:

[0049] The third determining submodule is used to determine the display position of the compensation display content in the steering wheel visualization component;

[0050] An adjustment submodule is used to adjust the display size of the compensation content according to the display position and the size of the display position;

[0051] The first display submodule is used to display the compensation content after adjusting the display size at the display position.

[0052] In one embodiment, the third determining submodule is further configured to:

[0053] Determine the steering wheel rotation angle;

[0054] The position of the visual component on the steering wheel, located at the center of the driver's field of vision, is determined based on the steering wheel rotation angle.

[0055] The position of the steering wheel visualization component located at the center of the driver's field of vision is determined as the display position of the compensation display content in the steering wheel visualization component.

[0056] In one embodiment, the display module includes:

[0057] The fourth determining submodule is used to determine the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument display screen;

[0058] The second display submodule is used to display the compensation display content through the steering wheel visualization component and adjust the display angle of the compensation display content so that the compensation display content is seamlessly spliced ​​with the unobstructed content of the instrument display screen in the driver's field of vision.

[0059] In one embodiment, the steering wheel rotation angle is determined as follows:

[0060] The rotation angle of the steering column is determined by an angle sensor, wherein the angle sensor is located at the rotation center axis for adjusting the up and down angle of the steering column.

[0061] The steering wheel rotation angle is determined based on the rotation angle of the steering wheel column.

[0062] This application also provides a content display system, including:

[0063] At least one processor; and,

[0064] A memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions that can be executed by the at least one processor to implement the content display method described in any of the above embodiments.

[0066] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor corresponding to a content display system, enables the content display system to implement the content display method described in any of the above embodiments.

[0067] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0068] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. In the drawings:

[0070] Figure 1 This is a flowchart of a content display method according to an embodiment of this application;

[0071] Figure 2 This is a front view of a visual steering wheel according to one embodiment of this application;

[0072] Figure 3 This is a visual steering wheel side view according to one embodiment of the present application;

[0073] Figure 4 This is a cross-sectional view of a visual steering wheel display portion in one embodiment of this application;

[0074] Figure 5 This is a schematic diagram of an eye-tracking camera tracking and positioning the driver's eyeballs in one embodiment of this application;

[0075] Figure 6 This is the display principle of a flexible curved screen in one embodiment of this application;

[0076] Figure 7 This is a schematic diagram illustrating the effect of placing the steering wheel in one embodiment of this application;

[0077] Figure 8 This is a schematic diagram of the structure of a content display device according to an embodiment of this application;

[0078] Figure 9 This is a schematic diagram of the hardware structure of a content display system according to an embodiment of this application. Detailed Implementation

[0079] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0080] Figure 1 This is a flowchart of a content display method according to an embodiment of this application, such as... Figure 1 As shown, the method can be implemented as follows: S101-S104:

[0081] In step S101, the position of the center of the driver's eyeball is obtained;

[0082] In step S102, the obstructed area of ​​the instrument display screen in the driver's field of vision is determined based on the center position of the driver's eyeball;

[0083] In step S103, the compensation display content is determined based on the content of the obscured area of ​​the instrument display screen;

[0084] In step S104, the compensation display content is displayed on the steering wheel visualization component.

[0085] In this application, to determine the portion of the instrument panel display screen obscured by the steering wheel within the driver's field of vision, it is first necessary to obtain the center position of the driver's eyeball. There are various methods for obtaining this information, such as capturing images of the driver using a camera and then tracking the center position of the driver's eyeball.

[0086] Figure 2 This is a front view of a visual steering wheel according to one embodiment of this application. Figure 3 Its side view, such as Figure 2 and Figure 3 As shown, the visual steering wheel includes a steering wheel visualization component 101, an instrument display screen 102, an eye-tracking camera 103, an angle sensor 104, and a controller assembly 105. The instrument display screen 102 can be located on a portion of the steering wheel or on the entire steering wheel; the angle sensor 104 is optional. If the steering wheel is fixed and not adjustable, the angle sensor 104 is not needed; if the steering wheel is four-way adjustable (i.e., it can adjust not only the vertical angle but also the forward / backward position), a displacement sensor can be added to detect the movement of the steering wheel along the column direction, thereby calculating the relative position and angle between the steering wheel and the instrument display screen 102. Figure 4 This is a cross-sectional view of a visual steering wheel display portion according to one embodiment of this application, such as... Figure 4 As shown, the steering wheel visualization component 101 consists of a resin housing 1011 and a flexible curved screen 1012.

[0087] To obtain the center position of the driver's eyeball, multiple two-dimensional image coordinates of the driver's eyeball can be obtained by multiple fixed-position eye-tracking cameras. Each eye-tracking camera corresponds to a two-dimensional image coordinate, and based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera, the two-dimensional image coordinates corresponding to each eye-tracking camera are converted into a gaze direction vector in three-dimensional space. Figure 5 This is a schematic diagram illustrating the tracking and positioning of the driver's eye using an eye-tracking camera in one embodiment of this application. Figure 5As shown, there are three eye-tracking cameras 103, positioned in different locations within the vehicle (e.g., the A-pillar, dashboard, etc.). The cameras are directed towards the driver's eyes, tracking the driver's gaze from different directions. By identifying the driver's eye observation angle, the three cameras fit and calculate the eye field of view detected from three different directions to determine the driver's accurate gaze direction. Since similar technologies are already maturely applied in eye trackers, and the internal eye-tracking algorithm is not the focus of this invention, it will not be elaborated upon here. Geometrically, the intersection of the lines connecting the three eye-tracking cameras to the driver's eyes represents the driver's eye position. Assuming the three-dimensional coordinates of eye-tracking camera number one are (0, 0, 0), and given the fixed positions of the three cameras, the coordinates of eye-tracking cameras number two and three are (X2, Y2, Z2) and (X3, Y3, Z3), respectively. The driver's eye coordinates can then be calculated using geometric principles. Specifically, three fixed-position eye-tracking cameras acquire two-dimensional image coordinates of the driver's eyes. Based on the intrinsic parameter matrix and distortion coefficients of each camera, the image coordinates are converted into gaze direction vectors (V1, V2, V3) in three-dimensional space. The gaze direction vectors corresponding to each eye-tracking camera are represented as spatial straight line equations; the gaze direction vectors of each camera are also represented as spatial straight line equations, where:

[0088] The equation of the line of sight of camera number one is: L1=(0,0,0)+t1·V1;

[0089] The equation of the line of sight of the second camera is: L2=(X2,Y2,Z2)+t2·V2;

[0090] The equation of the line of sight of camera number three is: L3=(X3,Y3,Z3)+t3·V3;

[0091] Where t1, t2, and t3 are parameter variables in the parametric equation, used to represent the point positions on the line of sight equation of the camera. When the parameter variable is equal to zero, it represents the initial position of the camera; when the parameter variable is greater than zero, it represents the distance moved in the positive direction along the direction vector; when the parameter variable is less than zero, it represents the distance moved in the reverse direction.

[0092] By simultaneously solving the linear equations of multiple cameras, the optimal intersection point of each spatial linear equation can be obtained, and this optimal intersection point can be used as the three-dimensional coordinates of the driver's eye center position. For example, the least squares method can be used to solve for the optimal intersection point (x, y, z) of three spatial lines, minimizing the sum of the squared distances from this point to the three lines, thus obtaining the three-dimensional coordinates of the driver's eye center. Furthermore, the calculated eye center coordinates can be smoothed in real time using Kalman filtering to eliminate measurement noise, and the final result is transmitted to the controller group 105 for occlusion compensation calculation.

[0093] Based on the center position of the driver's eyeball, the obstructed area of ​​the instrument display screen in the driver's field of vision is determined. Angle sensor 104 is located at the rotation center axis of the steering column for adjusting the vertical angle, and is used to detect the angle information of the adjustable steering wheel. For example... Figure 5 As shown, after detecting the rotation angle of the steering column, the angle sensor 104 can calculate the angle between the steering wheel and the vertical line. Since the instrument display 102 is fixed and the distance L between the steering wheel and the rotation axis of the steering column remains unchanged, the angle and distance between the steering wheel visualization component 101 and the instrument display 102 can be calculated. Figure 6 The flexible curved screen display principle in one embodiment of this application is as follows: Figure 6 As shown, first, a set of lines connecting the center position of the driver's eyeball and the content displayed on the instrument display screen is obtained; then, based on the steering wheel rotation angle, a subset of lines intersecting with the steering wheel is determined; finally, the area where the intersection of the subset of lines with the instrument display screen is located is determined as the obscured area of ​​the instrument display screen in the driver's field of vision. Figure 6 As shown, the flexible curved screen 1012 in the steering wheel visualization component 101 displays the content shown on the instrument cluster display after processing it using a specific algorithm. This allows the driver to see the corresponding content on the flexible curved screen while observing the instrument cluster, preventing obstruction. Specifically, the algorithm connects the driver's eyeball 200 to the content 107 displayed on the instrument cluster display 102 using a set of lines. The set of intersections between this set of lines and the steering wheel visualization component 1012 represents the content to be displayed on the flexible screen.

[0094] The compensation display content is determined based on the content of the obscured area of ​​the instrument display screen, and then displayed on the steering wheel visualization component. Specifically, in one embodiment, the obscured area may be recombined and displayed on the steering wheel visualization component; Figure 7 This is a schematic diagram illustrating the effect of placing the steering wheel in one embodiment of this application, as shown below. Figure 7As shown, this involves seamlessly stitching the content of the obscured area with the unobscured content of the instrument panel screen after calculation. In one embodiment, the display position of the compensation display content in the steering wheel visualization component is first determined. For example, to prevent the display from deflecting due to steering wheel rotation, the steering wheel rotation angle can be determined. Based on the steering wheel rotation angle, the position of the compensation display content in the center of the driver's field of vision in the steering wheel visualization component is determined. This position is then determined as the display position of the compensation display content in the steering wheel visualization component. The display size of the compensation content is then adjusted according to the display position and its dimensions. The adjusted compensation content is then displayed at the specified position. In another embodiment of this application, the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument panel display screen is determined. The compensation display content is displayed through the steering wheel visualization component, and the display angle of the compensation display content is adjusted to ensure seamless stitching of the compensation display content with the unobscured content of the instrument panel display screen in the driver's field of vision.

[0095] During display, the system can receive voice commands and prioritize display information; it dynamically adjusts the display ratio of the compensation area based on voice keywords (such as "vehicle speed" and "fuel consumption"). The voice control module works in conjunction with the vehicle's microphone array using beamforming. During display, it can also acquire cabin illuminance values ​​via an ambient light sensor; dynamically adjusting the brightness and color temperature of the flexible curved screen based on these values. Night mode automatically enhances contrast to over 150%. Furthermore, it can detect driver gestures via a capacitive sensor array built into the steering wheel; when a preset gesture is recognized, it adjusts the layout of the compensation display content. Gesture operations include: pinching to zoom, swiping with three fingers to turn pages, and clenching a fist to confirm.

[0096] In the above method, the steering wheel rotation angle is determined as follows: the rotation angle of the steering wheel column is determined according to an angle sensor, wherein the angle sensor is located at the rotation center axis of the steering wheel column for adjusting the up and down angles; the steering wheel rotation angle is determined according to the rotation angle of the steering wheel column.

[0097] Furthermore, in another embodiment of this application, the duration of the eye-tracking compensation area can be calculated; when the duration exceeds a threshold, an attention distraction alarm is triggered; the alarm methods include steering wheel vibration and red flashing. Alternatively, blink frequency and pupil diameter changes can be calculated using eye-tracking data; when the blink frequency exceeds a first preset range or the pupil diameter change exceeds a second preset range, fatigue characteristics are identified; when fatigue characteristics are detected, a red warning frame is overlaid on the steering wheel display area, and seat vibration and high-speed air conditioning mode can be triggered simultaneously.

[0098] As described above, this invention uses an eye-tracking camera 103 and an angle sensor 104 to determine the positions of the steering wheel and the driver's eyes 200. Then, utilizing simple optical and human visual principles, the content of the area on the instrument display screen 102 obscured by the steering wheel is displayed on the flexible curved screen 1012 of the steering wheel visualization component 101. This allows the driver to indirectly see the content on the instrument display screen 102, thereby significantly decoupling the relative positions of the instrument display screen 102 and the steering wheel in automotive interior design. This increases the design diversity of the dashboard and effectively solves the common automotive design problem of the inability to simultaneously balance steering wheel operating comfort and instrument cluster visibility.

[0099] The solution provided in this application offers two main advantages: First, it ensures comprehensive information acquisition. By acquiring the center position of the driver's eyeball, the obstructed area of ​​the instrument display screen in the driver's field of vision is accurately determined. Then, based on the content of the obstructed area, compensation display content is determined and displayed on the steering wheel visualization component. In this way, regardless of how the steering wheel is turned, the driver can obtain complete and clear instrument information that might otherwise be obstructed, avoiding driving decision errors due to missing information and greatly improving driving safety and reliability. Second, it does not change driving habits. Unlike existing technologies that eliminate the instrument cluster and transfer information to the central control screen, this application displays compensation content on the steering wheel visualization component. The steering wheel is a component that the driver continuously touches and operates during driving, and their gaze naturally sweeps across it occasionally. Displaying key information here eliminates the need for the driver to turn their head significantly or change their original instrument viewing habits, reducing distractions during driving and allowing the driver to focus more on the road and traffic conditions ahead, further improving driving safety. It also eliminates the need for a smaller diameter steering wheel to reduce obstruction, without sacrificing the driver's comfort and stability when operating the steering wheel. Drivers can hold and operate the steering wheel in their own preferred way, which can effectively reduce hand fatigue and the probability of operational errors during long drives, providing drivers with a more comfortable and safer driving experience.

[0100] The beneficial effects of this application are as follows: It obtains the center position of the driver's eyeball, then accurately determines the obstructed area of ​​the instrument display screen in the driver's field of vision based on this position, and then determines the compensation display content based on the content of the obstructed area, displaying it in the steering wheel visualization component. This solution allows the driver to obtain complete and clear instrument information that might otherwise be obstructed, regardless of how the steering wheel is turned, avoiding driving decision errors due to missing information and improving driving safety and reliability. Compared with existing technologies, it does not require changing the steering wheel size, allowing the driver to hold and operate the steering wheel in their usual way, reducing the probability of operational errors; it also eliminates the need for the driver to turn their head significantly or change their original habits of viewing instrument information, reducing distracting driving behaviors and improving vehicle driving safety.

[0101] In one embodiment, step S101 above can be implemented as steps A1-A4 as follows:

[0102] In step A1, multiple two-dimensional image coordinates of the driver's eyes are acquired by multiple eye-tracking cameras at fixed positions, wherein each eye-tracking camera corresponds to one two-dimensional image coordinate.

[0103] In step A2, based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera, the two-dimensional image coordinates corresponding to each eye-tracking camera are converted into a gaze direction vector in three-dimensional space;

[0104] In step A3, the gaze direction vector corresponding to each eye-tracking camera is expressed as a spatial straight line equation;

[0105] In step A4, the optimal intersection point of each spatial line equation is solved, and the optimal intersection point is used as the three-dimensional coordinates of the driver's eye center position.

[0106] In one embodiment, step S102 above can be implemented as steps B1-B3 as follows:

[0107] In step B1, a set of lines connecting the center position of the driver's eyeballs to the content displayed on the instrument panel is obtained;

[0108] In step B2, the subset of lines that intersect with the steering wheel is determined from the set of lines based on the steering wheel rotation angle;

[0109] In step B3, the area where the intersection of the set of connecting lines and the instrument display screen is located is determined as the obscured area of ​​the instrument display screen in the driver's field of vision.

[0110] In one embodiment, step S104 above can be implemented as steps C1-C3 as follows:

[0111] In step C1, the display position of the compensation display content in the steering wheel visualization component is determined;

[0112] In step C2, the display size of the compensation content is adjusted according to the display position and the size of the display position;

[0113] In step C3, the compensation content after adjusting the display size is displayed at the specified display position.

[0114] In one embodiment, step C1 above can be implemented as steps C11-C13:

[0115] In step C11, the steering wheel rotation angle is determined;

[0116] In step C12, the position of the steering wheel visualization component located at the center of the driver's field of vision is determined based on the steering wheel rotation angle;

[0117] In step C13, the position of the steering wheel visualization component located at the center of the driver's field of vision is determined as the display position of the compensation display content in the steering wheel visualization component.

[0118] In one embodiment, step S104 above can be implemented as steps D1-D2 as follows:

[0119] In step D1, the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument display screen is determined;

[0120] In step D2, the compensation display content is displayed through the steering wheel visualization component, and the display angle of the compensation display content is adjusted so that the compensation display content is seamlessly spliced ​​with the unobstructed content of the instrument display screen in the driver's field of vision.

[0121] In one embodiment, the method may also be implemented as follows:

[0122] Real-time calibration of the geometric continuity of the stitched images ensures that the driver can see a complete visual view of the instrument cluster.

[0123] In one embodiment, the determination of the steering wheel rotation angle can be implemented as follows: steps E1-E2:

[0124] In step E1, the rotation angle of the steering column is determined according to the angle sensor, wherein the angle sensor is located at the rotation center axis for adjusting the up and down angle of the steering column.

[0125] In step E2, the steering wheel rotation angle is determined based on the rotation angle of the steering wheel column.

[0126] Figure 8 This is a schematic diagram of the structure of a content display device according to an embodiment of this application, as shown below. Figure 8 As shown, the device includes:

[0127] The acquisition module 801 is used to acquire the center position of the driver's eyeball;

[0128] The first determining module 802 is used to determine the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball;

[0129] The second determining module 803 is used to determine the compensation display content based on the content of the obscured area of ​​the instrument display screen;

[0130] Display module 804 is used to display the compensation display content in the steering wheel visualization component.

[0131] In one embodiment, the acquisition module includes:

[0132] The first acquisition submodule is used to acquire multiple two-dimensional image coordinates of the driver's eyes through multiple eye-tracking cameras at fixed positions, wherein each eye-tracking camera corresponds to one two-dimensional image coordinate.

[0133] The transformation submodule is used to convert the two-dimensional image coordinates corresponding to each eye-tracking camera into a gaze direction vector in three-dimensional space based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera.

[0134] The transformation submodule is also used to represent the gaze direction vector corresponding to each eye-tracking camera as a spatial straight line equation;

[0135] The solution submodule is used to solve for the optimal intersection point of each spatial line equation and use the optimal intersection point as the three-dimensional coordinates of the driver's eye center position.

[0136] In one embodiment, the first determining module includes:

[0137] The second acquisition submodule is used to acquire the set of lines connecting the center position of the driver's eyeballs and the content displayed on the instrument display screen;

[0138] The first determining submodule is used to determine the set of connecting lines that intersect with the steering wheel in the set of connecting lines based on the steering wheel rotation angle.

[0139] The second determining submodule is used to determine that the area where the intersection of the set of connecting lines and the instrument display screen is located is the obstructed area of ​​the instrument display screen in the driver's field of vision.

[0140] In one embodiment, the display module includes:

[0141] The third determining submodule is used to determine the display position of the compensation display content in the steering wheel visualization component;

[0142] An adjustment submodule is used to adjust the display size of the compensation content according to the display position and the size of the display position;

[0143] The first display submodule is used to display the compensation content after adjusting the display size at the display position.

[0144] In one embodiment, the third determining submodule is further configured to:

[0145] Determine the steering wheel rotation angle;

[0146] The position of the visual component on the steering wheel, located at the center of the driver's field of vision, is determined based on the steering wheel rotation angle.

[0147] The position of the steering wheel visualization component located at the center of the driver's field of vision is determined as the display position of the compensation display content in the steering wheel visualization component.

[0148] In one embodiment, the display module includes:

[0149] The fourth determining submodule is used to determine the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument display screen;

[0150] The second display submodule is used to display the compensation display content through the steering wheel visualization component and adjust the display angle of the compensation display content so that the compensation display content is seamlessly spliced ​​with the unobstructed content of the instrument display screen in the driver's field of vision.

[0151] In one embodiment, the steering wheel rotation angle is determined as follows:

[0152] The rotation angle of the steering column is determined by an angle sensor, wherein the angle sensor is located at the rotation center axis for adjusting the up and down angle of the steering column.

[0153] The steering wheel rotation angle is determined based on the rotation angle of the steering wheel column.

[0154] Figure 9 This is a schematic diagram of the hardware structure of a content display system according to an embodiment of this application, such as... Figure 9 As shown, the content display system includes:

[0155] At least one processor 920; and,

[0156] Memory 904 is communicatively connected to the at least one processor 920; wherein,

[0157] The memory 904 stores instructions that can be executed by the at least one processor 920 to implement the content display method described in any of the above embodiments.

[0158] Reference Figure 9 The content shows that the system 900 may include one or more of the following components: processing component 902, memory 904, power supply component 906, multimedia component 908, audio component 910, input / output (I / O) interface 912, sensor component 914, and communication component 916.

[0159] Processing component 902 typically controls the overall operation of content display system 900. Processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0160] Memory 904 is configured to store various types of data to support the operation of content display system 900. Examples of this data include instructions for any application or method operating on content display system 900, such as text, images, video, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0161] Power supply component 906 provides power to various components of content display system 900. Power supply component 906 may include power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to vehicle control system 900.

[0162] Multimedia component 908 includes a screen that provides an output interface between content display system 900 and user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 may also include a front-facing camera and / or a rear-facing camera. When content display system 900 is in an operating mode, such as shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0163] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when the content display system 900 is in an operating mode, such as alarm mode, recording mode, voice recognition mode, and voice output mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0164] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0165] Sensor assembly 914 includes one or more sensors for providing status assessments of various aspects of the content display system 900. For example, sensor assembly 914 may include a sound sensor. Additionally, sensor assembly 914 may detect the on / off state of the content display system 900, the relative positioning of components such as the display and keypad of the content display system 900, and the operational status of the content display system 900 or one of its components. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include a pressure sensor or a temperature sensor.

[0166] Communication component 916 is configured to enable content display system 900 to provide wired or wireless communication capabilities with other devices and cloud platforms. Content display system 900 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0167] In an exemplary embodiment, the content display system 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the content display method described in any of the above embodiments.

[0168] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor corresponding to a content display system, enables the content display system to implement the content display method described in any of the above embodiments.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0170] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A content display method, characterized in that, include: Obtain the center position of the driver's eyeball; Determine the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball; The compensation display content is determined based on the content of the obscured area of ​​the instrument display screen; The compensation content is displayed on the steering wheel visualization component.

2. The method as described in claim 1, characterized in that, The step of obtaining the center position of the driver's eyeball includes: Multiple two-dimensional image coordinates of the driver's eyes are obtained by multiple eye-tracking cameras at fixed positions, with each eye-tracking camera corresponding to one two-dimensional image coordinate. Based on the intrinsic parameter matrix and distortion coefficient of each eye-tracking camera, the two-dimensional image coordinates corresponding to each eye-tracking camera are converted into a gaze direction vector in three-dimensional space; The gaze direction vector corresponding to each eye-tracking camera is represented as a spatial straight line equation; Solve for the optimal intersection point of the equations of each spatial line, and use the optimal intersection point as the three-dimensional coordinates of the driver's eye center position.

3. The method as described in claim 1, characterized in that, The step of determining the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball includes: Obtain the set of lines connecting the center position of the driver's eyeballs to the content displayed on the instrument panel; Determine the subset of lines in the set of lines that intersect with the steering wheel based on the steering wheel rotation angle; The area where the intersection of the set of connecting lines and the instrument display screen is located is determined as the area where the instrument display screen is obscured in the driver's field of vision.

4. The method as described in claim 1, characterized in that, The step of displaying the compensation display content in the steering wheel visualization component includes: Determine the display position of the compensation display content within the steering wheel visualization component; Adjust the display size of the compensation content according to the display position and the size of the display position; The compensation content after adjusting the display size will be displayed at the specified display position.

5. The method as described in claim 4, characterized in that, Determining the display position of the compensation display content in the steering wheel visualization component includes: Determine the steering wheel rotation angle; The position of the visual component on the steering wheel, located at the center of the driver's field of vision, is determined based on the steering wheel rotation angle. The position of the steering wheel visualization component located at the center of the driver's field of vision is determined as the display position of the compensation display content in the steering wheel visualization component.

6. The method as described in claim 1, characterized in that, The step of displaying the compensation display content in the steering wheel visualization component includes: Determine the boundary line between the area obscured by the steering wheel and the unobscured area in the instrument display screen; The compensation display content is displayed through the steering wheel visualization component, and the display angle of the compensation display content is adjusted so that the compensation display content is seamlessly spliced ​​with the unobstructed content of the instrument display screen in the driver's field of vision.

7. The method as described in claim 3 or 5, characterized in that, The method for determining the steering wheel rotation angle is as follows: The rotation angle of the steering column is determined by an angle sensor, wherein the angle sensor is located at the rotation center axis for adjusting the up and down angle of the steering column. The steering wheel rotation angle is determined based on the rotation angle of the steering wheel column.

8. A content display device, characterized in that, include: The acquisition module is used to acquire the center position of the driver's eyeball; The first determining module is used to determine the obstructed area of ​​the instrument display screen in the driver's field of vision based on the center position of the driver's eyeball; The second determining module is used to determine the compensation display content based on the content of the obscured area of ​​the instrument display screen; The display module is used to display the compensation display content in the steering wheel visualization component.

9. A content display system, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the content display method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor corresponding to the content display system, the content display system is able to implement the content display method as described in any one of claims 1-7.

Images