Display method and device, vehicle, storage medium and computer program product

Abstract

The invention relates to a display method and device, a vehicle, a storage medium and a computer program product, relates to the technical field of vehicle information display, and can be applied to an intelligent cabin. The method comprises the following steps: monitoring sight line data of a user through at least one camera; according to the sight line data, stay information of the sight line of the user in a target area is determined, and the target area is a display area of the in-vehicle display device; and if the stay information meets the preset condition, adjusting the display elements in the target area. Therefore, the display elements in the target area can be adjusted according to the sight line of the user, manual operation of the user is not needed, and the driving and riding safety is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicle information display, and in particular to a display method, device, vehicle, storage medium, and computer program product. Background Art

[0002] Vehicles are equipped with display devices that display various types of information, such as vehicle speed, range, navigation, and multimedia entertainment. In the prior art, the information displayed in the display area of ​​an in-vehicle display device is typically fixed. If a user wishes to adjust the displayed information, manual adjustment is required, which is cumbersome and detrimental to driving safety. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a display method, device, vehicle, storage medium and computer program product.

[0004] According to a first aspect of an embodiment of the present disclosure, a display method is provided, comprising: monitoring a user's line of sight data through at least one camera; determining, based on the line of sight data, the stay information of the user's line of sight in a target area, wherein the target area is a display area of ​​an in-vehicle display device; if the stay information meets a preset condition, adjusting the display elements within the target area.

[0005] According to a second aspect of an embodiment of the present disclosure, a display device is provided, including: a monitoring module, configured to monitor a user's line of sight data through at least one camera; a determination module, configured to determine the user's line of sight staying information in a target area based on the line of sight data, wherein the target area is a display area of ​​an in-vehicle display device; and an adjustment module, configured to adjust the display elements within the target area if the staying information meets a preset condition.

[0006] According to a third aspect of an embodiment of the present disclosure, a display device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: monitor a user's line of sight data through at least one camera; determine, based on the line of sight data, information on where the user's line of sight stays in a target area, wherein the target area is a display area of ​​an in-vehicle display device; and if the stay information meets a preset condition, adjust the display elements within the target area.

[0007] According to a fourth aspect of an embodiment of the present disclosure, a vehicle is provided, comprising: at least one camera for capturing a facial image of a user, wherein the facial image is used to determine line of sight data of the user; and a display device according to the third aspect.

[0008] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0009] According to a sixth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0010] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the user's line of sight data is monitored by at least one camera, and the information on the user's line of sight staying in the target area can be determined based on the line of sight data, wherein the target area is the display area of ​​the in-vehicle display device. If the information on the user's line of sight staying in the target area meets the preset conditions, it indicates that the user is paying more attention to the target area, and the display elements in the target area are adjusted at this time. Through this technical solution, the display elements in the target area can be adjusted according to the user's line of sight without the need for manual operation by the user, thereby improving driving safety. In addition, the display elements in the target area are adjusted only when the user's line of sight staying in the target area meets the preset conditions, and the display elements in the target area will not be adjusted in response to the user's line of sight just because the user's line of sight instantaneously sweeps over the target area, thereby avoiding frequent adjustments that distract the user and reduce the user experience.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0013] Figure 1 The figure is a flowchart of a display method according to an exemplary embodiment.

[0014] Figure 2 FIG. 1 is a schematic diagram showing a display area of ​​a display device according to an exemplary embodiment.

[0015] Figure 3 The figure is a schematic diagram showing display elements in a display area of ​​a display device according to an exemplary embodiment.

[0016] Figures 4 to 10 is a schematic diagram showing adjustment of display elements according to different exemplary embodiments.

[0017] Figure 11This is a schematic diagram showing a situation in which a third user interferes with the sight line of a target area according to an exemplary embodiment.

[0018] Figures 12 to 13 is a schematic diagram showing adjustment of display elements according to different exemplary embodiments.

[0019] Figures 14 and 15 is a schematic diagram showing adjustment of display elements according to different exemplary embodiments.

[0020] Figure 16 and Figure 17 A schematic diagram of a scene of display element adjustment according to an exemplary embodiment of the present disclosure is shown.

[0021] Figure 18 is a structural block diagram of a display device according to an exemplary embodiment.

[0022] Figure 19 is a structural block diagram of a display device according to yet another exemplary embodiment.

[0023] Figure 20 The figure is a structural block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0025] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0026] Figure 1 This is a flow chart of a display method according to an exemplary embodiment. The method can be applied to a vehicle, for example, to a controller in a vehicle. For example, the controller can include, but is not limited to, at least one of the following: a DMS controller in a driver monitoring system (DMS), a cockpit domain controller, an advanced driver assistance system (ADAS) domain controller, a central computing platform, etc. Figure 1 As shown, the method may include the following steps.

[0027] S101: Monitor user's sight data through at least one camera.

[0028] At least one camera can be installed in the vehicle to monitor the user's gaze data. For example, the at least one camera is used to monitor the gaze data of at least one user, with different cameras used to monitor the gaze data of different users. The at least one camera can include a camera in the DMS system. The camera can be used to capture the user's facial image, and then determine the user's gaze data by performing image recognition on the facial image captured by the camera.

[0029] The sight line data may include, but is not limited to, at least one of the following: a sight line direction, a gaze point position, or a gaze range. Based on the sight line data, the position or area currently being gazed at by the user may be determined.

[0030] S102 , determining information on the user's sight line staying in a target area according to the sight line data, wherein the target area is a display area of ​​an in-vehicle display device.

[0031] The stay information may include at least one of the following: stay duration, stay number. When the sight line data indicates that the user's sight line falls into the target area, the stay duration and / or stay number of the user's sight line in the target area may be counted.

[0032] The in-vehicle display device may include one or more of the following: a physical screen, a projection display device. Furthermore, the in-vehicle display device may include one or more display areas. If the in-vehicle display device includes a physical screen, the screen area of ​​the physical screen may serve as a single display area, or may be divided into multiple display areas. If the in-vehicle display device includes a projection display device, the projection area of ​​the projection display device may serve as a single display area, or the projection area may be divided into multiple display areas.

[0033] For example, the in-vehicle display device is a projection display device, which can be used to project onto the front windshield of the vehicle to form a display area extending from one side of the front windshield to the other side. For example, the display device may include one or more projection devices (such as an optical machine, a projector, etc.), each of which projects onto the front windshield of the vehicle to form one or more display areas extending from one side of the front windshield to the other side. For example, Figure 2As shown, the display device includes three projection devices, which project images onto the left, center, and right sides of the front windshield 201, respectively, to form three display areas. These three display areas extend from the left side to the right side of the front windshield 201. The left display area is hereinafter referred to as the first display area 202, the center display area is hereinafter referred to as the second display area 203, and the right display area is hereinafter referred to as the third display area 204.

[0034] In another embodiment, the display area of ​​the display device extends from one side of the vehicle center console to the other. In this embodiment, the display device can be a physical screen. For example, the display device can include at least one physical screen, and the screen area of ​​the physical screen can serve as a single display area or can be divided into multiple display areas. In an exemplary embodiment, the display device can be a long strip of physical screen, and the screen area of ​​the long strip of physical screen can be divided into at least two display areas.

[0035] In the embodiment of the present disclosure, when the in-vehicle display device includes at least two display areas, the lateral widths of the at least two display areas are fixed. Figure 2 As shown, the horizontal widths W1 , W2 and W3 of the first display area 202 , the second display area 203 and the third display area 204 are all fixed values.

[0036] In addition, in the case where the in-vehicle display device includes at least two display areas, the at least two display areas can be arranged horizontally, for example Figure 2 As shown, the three display areas are arranged horizontally. However, in other embodiments, the at least two display areas may also be arranged vertically, such as along the A-pillar on the front windshield, which is not limited in the present disclosure.

[0037] The target area is the display area of ​​the in-vehicle display device. If the in-vehicle display device includes at least two display areas, then the target area is one of the at least two display areas. Figure 2 As shown in the example, the display device includes three display areas, and the target area may be one of the three display areas, for example, the target area is the second display area 203 .

[0038] In one embodiment, when the in-vehicle display device includes at least two display areas, different display areas can be used to display different types of vehicle information.

[0039] For example, vehicle information can be divided into instrument information, navigation information, and entertainment information. The in-vehicle display device includes three display areas, and the three display areas are used to display instrument information, navigation information, and entertainment information respectively.

[0040] S103: If the stay information meets the preset conditions, adjust the display elements in the target area.

[0041] As mentioned above, the stay information may include a stay duration. In this case, the preset condition may include: the stay duration reaches a preset duration. The preset duration may be pre-set according to actual needs, for example, to 3 seconds.

[0042] In another embodiment, the stay information may include the number of stays. In this case, the preset condition may include: the number of stays reaches a preset number. The preset number can be preset according to actual needs, for example, three times.

[0043] In another embodiment, the stay information may include both the stay duration and the number of stays. In this case, the preset condition may include: the number of stays reaches a preset number and the stay duration of each stay reaches a preset duration.

[0044] In this disclosure, display elements refer to user interface (UI) elements displayed in the display area of ​​an in-vehicle display device. UI elements are used to present vehicle information to the user. For example, UI elements may include UI elements for displaying the current vehicle speed, UI elements for displaying a navigation map interface, UI elements for displaying the remaining power battery charge, UI elements for displaying the title of the currently playing multimedia content, and so on. Furthermore, UI elements may be displayed in at least one of the following formats: images, graphics, text, and animations.

[0045] Figure 3 A schematic diagram of display elements in a display area of ​​a display device is shown. Figure 3 As shown, the display device includes a first display area 202, a second display area 203, and a third display area 204. The first display area 202 is used to display instrument information, such as vehicle speed, mileage, remaining battery charge, and driving energy consumption. The second display area 203 is used to display vehicle navigation information, such as the navigation map interface, vehicle turn prompts, remaining navigation distance, remaining navigation time, etc. The third display area 204 is used to display entertainment information, such as the title of multimedia content, the playback volume of multimedia content, and the playback mode of multimedia content.

[0046] In one example, if Figure 3As shown, the first display area 202 displays three UI elements: UI element A for displaying vehicle speed, UI element B for displaying mileage, and UI element C for displaying driving energy consumption. The second display area 203 displays three UI elements: UI element D for displaying vehicle turn prompt information, UI element E for displaying the remaining navigation distance, and UI element F for displaying the navigation map interface. The third display area 204 displays three UI elements: UI element G for displaying the name of multimedia content, UI element H for displaying the playback volume of multimedia content, and UI element I for displaying the playback mode of multimedia content.

[0047] It should be noted that the display elements displayed in each display area described above are merely exemplary. The display elements displayed in each display area can be pre-set by the user according to their needs. During vehicle driving, the method provided by the present disclosure can dynamically adjust the display elements in the corresponding display area based on the user's line of sight.

[0048] If the user's gaze stays in the target area and meets a preset condition, indicating that the user is currently paying close attention to the target area, the display elements in the target area are adjusted. Adjusting the display elements in the target area includes but is not limited to at least one of the following: adjusting the display position of the display element; adjusting the display ratio or size of the display element; and adjusting the brightness of the display element.

[0049] For example, if the user's gaze stays within a target area and meets preset conditions, the display positions of display elements within the target area can be adjusted. For example, higher-priority display elements can be displayed at a location with a better field of view for the user. The priority of display elements will be described in detail below. Display positions within the display area with a better field of view for the user (including an optimal display position with the best field of view) can be pre-calibrated.

[0050] For example, if the user's sight stay information in the target area meets the preset conditions, the display ratio or size of the display elements in the target area can be adjusted. Figure 4 As shown, the target area is the second display area 203. If the information where the user's line of sight stays in the second display area 203 meets the preset conditions, the display elements (UI element D, UI element F and UI element E) in the second display area 203 are enlarged and displayed so that the user can see the information presented by the display elements in the second display area 203 more clearly.

[0051] As another example, if the user's gaze stays in the target area and meets the preset conditions, the brightness of the display elements in the target area can be adjusted, for example, highlighted, so that the user can see the information presented by the display elements in the target area more clearly.

[0052] In other embodiments, multiple or all of the above-mentioned adjustment operations may be performed. For example, while adjusting the display position of a display element, the display ratio or size of the display element may also be adjusted to prevent the display element from overlapping or exceeding the display range of the display area. Alternatively, while adjusting the display position of a display element, the brightness of the display element may also be adjusted. Alternatively, while adjusting the display ratio or size of a display element, the brightness of the display element may also be adjusted. Alternatively, while adjusting the display position of a display element, the display ratio or size of the display element may also be adjusted, and the brightness of the display element may also be adjusted.

[0053] In the display method provided by the present disclosure, the user's line of sight data is monitored by at least one camera, and the information of the user's line of sight staying in the target area can be determined based on the line of sight data, wherein the target area is the display area of ​​the in-vehicle display device. If the information of the user's line of sight staying in the target area meets the preset conditions, it indicates that the user is paying more attention to the target area, and the display elements in the target area are adjusted at this time. Through this technical solution, the display elements in the target area can be adjusted according to the user's line of sight without the need for manual operation by the user, thereby improving driving safety. In addition, the display elements in the target area are adjusted only when the information of the user's line of sight staying in the target area meets the preset conditions, and the display elements in the target area will not be adjusted in response to the user's line of sight just because the user's line of sight instantly sweeps over the target area, thereby avoiding the situation where frequent adjustments distract the user and reduce the user experience.

[0054] In the embodiments of the present disclosure, the users in the vehicle may include but are not limited to the first user and / or the second user. That is, in one possible implementation of the present disclosure, the sight lines of multiple users may be monitored, and a strategy for adjusting display elements within the display area may be determined based on the sight lines of the multiple users.

[0055] In an exemplary embodiment, a first user may be in the driving seat of a vehicle, and a second user may be in the non-driving seat of the vehicle. The first user in the driving seat of the vehicle is related to the driving safety of the vehicle. Therefore, in this embodiment, the priority of the user in the driving seat is higher than that of the user in the non-driving seat. During the driving of the vehicle, the changes in the line of sight of the user in the driving seat are given priority. For example, when it is detected that the lines of sight of the user in the driving seat and the user in the non-driving seat fall into the same target area at the same time, the display elements in the target area are adjusted based on the line of sight of the user in the driving seat. In this way, the adjustment needs of the user in the driving seat can be met first, which helps to improve driving safety.

[0056] The second user is located in the non-driving seat of the vehicle, including but not limited to: the second user is located in the co-pilot seat of the vehicle; or the second user is located in the back seat of the vehicle.

[0057] In another exemplary embodiment, the first user may be located in a first non-driving seat of the vehicle, and the second user may be located in a second non-driving seat of the vehicle. Typically, the first non-driving seat and the second non-driving seat of the vehicle may be the passenger seat and the rear seat of the vehicle, respectively, or both may be the rear seat of the vehicle.

[0058] Different user priorities can be set for users in different non-driving seats. For example, the priority of the user in the front passenger seat can be set higher than the priority of users in the rear seats, and the priority of the user in the middle of the rear seats can be set higher than the priority of users on either side of the rear seats. Alternatively, the priority of users in non-driving seats can be determined based on the user's age, clothing information, etc. For example, the user's age information can be determined through image recognition and / or detection data from the weight sensor on the seat, and then the user can be determined to be an adult or a child. The priority of adults is higher than that of children. For another example, the user's clothing information, such as whether they are wearing sunglasses or headphones, can be determined through image recognition. If they are wearing sunglasses or headphones, it means that the user is less interested in the content displayed in the display area at this time. Therefore, the priority of non-driving users wearing items such as sunglasses or headphones is lower than that of non-driving users who are not wearing such items.

[0059] As mentioned above, the sight lines of multiple users may be monitored, and the adjustment strategy for display elements in the display area may be determined based on the sight lines of the multiple users. In this case, step S103 may further include: If the stay information meets the preset conditions, the display elements in the target area are adjusted according to the sight line of the third user, wherein the third user is the user with the highest priority among the users corresponding to the stay information meeting the preset conditions.

[0060] In this embodiment, there are at least two users. Each user's line of sight is monitored through steps S101 and S102 to determine each user's line of sight's stay information in the target area. If only one user's corresponding stay information meets the preset conditions, the third user is that user. If multiple users' corresponding stay information all meet the preset conditions, the third user is the user with the highest priority among these users.

[0061] In this way, by adjusting the display elements in the target area according to the sight line of the highest-priority user among the users corresponding to the dwell information that meets the preset conditions, the needs of the high-priority users can be preferentially met, and the viewing experience of the high-priority users on the display elements in the target area can be improved.

[0062] In one embodiment, adjusting display elements within the target area according to the line of sight of the third user may further include: If the third user is the user with the highest priority among all users, the display elements in the target area are adjusted according to the line of sight of the third user.

[0063] That is, in this embodiment, once the information on the sight of the user with the highest priority among all users staying in the target area meets the preset conditions, the display elements in the target area are adjusted according to the sight of the user with the highest priority among all users, regardless of whether the information on the sight of other users staying in the target area meets the preset conditions.

[0064] For example, in a vehicle, the user in the driving seat has the highest priority. If the information that the driver's seat user's line of sight stays in the target area meets the preset conditions, the display elements in the target area will be adjusted based on the line of sight of the driver's seat user, without considering the line of sight of users in other positions.

[0065] In another embodiment, adjusting display elements within the target area according to the line of sight of the third user may further include: If the third user is not the user with the highest priority among all users, then when the fourth user is not looking at the target area, the display elements in the target area are adjusted according to the line of sight of the third user, wherein the priority of the fourth user is higher than that of the third user.

[0066] That is, in this embodiment, if the third user is not the highest priority user among all users, the display elements within the target area can only be adjusted based on the third user's line of sight if a user with a higher priority than the third user (i.e., the fourth user) is not looking at the target area. Whether the fourth user is looking at the target area can be determined by determining whether the fourth user's line of sight falls within the target area based on the fourth user's line of sight data.

[0067] For example, there are two users in the vehicle, namely the user in the driver's seat and the user in the front passenger seat, and the information about the front passenger seat user's sight staying in the target area meets the preset conditions. Since the front passenger seat user is not the user with the highest priority among all users (because the user in the driver's seat has the highest priority among all users), whether to adjust the display elements in the target area according to the front passenger seat user's sight depends on whether the user in the driver's seat is currently looking at the target area. If the user in the driver's seat is not looking at the target area, the display elements in the target area are adjusted according to the front passenger seat user's sight. If the user in the driver's seat is looking at the target area (even if the information about the driver's seat user's sight staying in the target area does not meet the preset conditions at this time), the display elements in the target area are not adjusted according to the front passenger seat user's sight.

[0068] This satisfies the needs of third users and enhances their viewing experience of the display elements in the target area, while also not impacting the viewing experience of the higher-priority fourth user. Furthermore, while the vehicle is in motion, the above technical solution can assist third users in monitoring vehicle information, preventing distraction for the primary driver and improving driving safety.

[0069] In an embodiment of the present disclosure, adjusting the display element in the target area includes adjusting the display position of the display element in the target area. For example, the display position of the display element in the target area can be adjusted in the following manner: Determining a target display element and a target display position of the target display element within the target area according to the priority of the display element within the target area; In response to the target display position not being consistent with the current display position of the target display element, the target display element is adjusted to the target display position.

[0070] Exemplarily, the target display element may be a preset number (eg, the first two) of display elements in the target area, sorted from high to low in terms of priority, or a display element whose priority is higher than a preset threshold.

[0071] In one embodiment, the priority of the display elements can be pre-set. Figure 3As shown in the example, in the first display area 202, vehicle speed is the most important information, so UI element A for indicating vehicle speed has the highest priority, followed by UI element B for indicating mileage and UI element C for displaying driving energy consumption. In the second display area 203, the navigation map interface is the most important information, so UI element F for indicating the navigation map interface has the highest priority, followed by UI element D for displaying vehicle turn prompt information and UI element E for displaying the remaining navigation distance. In the third display area 204, UI element G for displaying the name of multimedia content has the highest priority, followed by UI element H for displaying the playback volume of multimedia content and UI element I for displaying the playback mode of multimedia content. It is worth noting that the priority ranking of the above UI elements is merely exemplary. The priority ranking of each display element can be set by the user according to needs, and this disclosure does not limit this.

[0072] In the display area of ​​the display device, multiple display positions can be pre-set, each display position being used to display a UI element. Furthermore, among the multiple display positions in the display area, there is an optimal display position that provides the user with the best field of view. The optimal display position of the display area for the user can be determined by comprehensively considering factors such as the distance from the user and the curvature of the front windshield.

[0073] The following combination Figure 3 , and takes the main driver user and the front passenger user as examples for illustration. The first display area 202 is located in front of the main driver, the third display area 204 is located in front of the front passenger, and the second display area 203 is located between the main driver and the front passenger. In the first display area 202, the display position 2023 on the right is closest to the front passenger, so this display position 2023 is the best display position for the front passenger user. In addition, affected by the curvature of the front windshield on both sides, this display position 2023 is located in the front windshield area with a relatively gentle curvature. Therefore, compared with the display position 2021 on the left and the display position 2022 in the middle, this display position 2023 is also the best display position for the main driver.

[0074] For another example, in the second display area 203, the left display position 2031 is closest to the driver's seat, and therefore is the optimal display position for the driver. The right display position 2033 is closest to the passenger seat, and therefore is the optimal display position for the passenger seat. In other embodiments, if there is a rear-seat user, then display position 2032 is the optimal display position for that rear-seat user.

[0075] For example, in the third display area 204, the left display position 2041 is closest to the driver's seat, and therefore is the optimal display position for the driver. Furthermore, due to the curvature of the windshield, display position 2041 is located within the relatively flat windshield area, making it the optimal display position for the passenger seat, compared to the right display position 2043 and the center display position 2042.

[0076] When determining the target display position of the target display element, an optimal display position in the display area for the third user may be considered.

[0077] In one embodiment, for different display areas of the display device, the UI style or UI theme of the UI elements in each display area may be different. Therefore, in order not to affect the overall visual experience of the user of the display area after adjustment, the display elements in different display areas are generally not displayed across areas. That is, in this embodiment, the target display position is determined within the target area. In other words, the target display position is located within the target area. For example, the target display position is the optimal display position in the target area for the third user.

[0078] Still Figure 3 For example, assuming that the third user is the primary driver, the target display element is the UI element F with the highest priority in the second display area 203. In the second display area 203, the display position 2031 is the best for the primary driver, so it can be determined that the display position 2031 is the target display position of the UI element F. However, at this time, the display position 2032 where the UI element F is located is different from the target display position, so the controller can adjust the UI element F to the target display position, that is, the display position 2031. For example, Figure 5 As shown in FIG, this adjustment can be achieved by swapping the display positions of UI element D and UI element F. Alternatively, as shown in FIG. Figure 6 As shown, the controller can adjust UI element F to the target display position, i.e., display position 2031, adjust UI element E to display position 2032, and adjust UI element D to display position 2033. That is, in the present disclosure, when the controller adjusts the target display element to the target display position, it does not limit the synchronous adjustment strategy for the display positions of other display elements in the target area. Or, as Figure 7 As shown, the controller can adjust UI element F to the target display position, that is, display position 2031, and at the same time adjust the display ratio or size of UI element D and UI element F at display position 2031 to avoid the situation where UI elements cover or exceed the display range of the display area.

[0079] In another embodiment, Figure 3 For example, assuming that the third user is the passenger user, the target display element is the UI element F with the highest priority in the second display area 203. In the second display area 203, the display position 2033 is the best for the passenger user, so it can be determined that the display position 2033 is the target display position of the UI element F. However, at this time, the display position 2032 where the UI element F is located is different from the target display position, so the controller can adjust the UI element F to the target display position, that is, the display position 2033. For example, Figure 8 As shown in FIG, this adjustment can be achieved by swapping the display positions of UI element E and UI element F. Alternatively, as shown in FIG. Figure 9 As shown, the controller can adjust UI element F to the target display position, i.e., display position 2033, adjust UI element E to display position 2031, and adjust UI element D to display position 2032. That is, in the present disclosure, when the controller adjusts the target display element to the target display position, it does not limit the synchronous adjustment strategy for the display positions of other display elements in the target area. Or, as Figure 10 As shown, the controller can adjust UI element F to the target display position, that is, display position 2033, and at the same time adjust the display ratio or size of UI element E and UI element F at display position 2033 to avoid the situation where UI elements cover or exceed the display range of the display area.

[0080] In another embodiment, the in-vehicle display device includes at least two display areas, the target area is one of the display areas of the display device, and the target display position is located in a display area of ​​the display device different from the target area.

[0081] In some scenarios, a third user's field of view of the target area may be obstructed by visual interference, such as a phone holder or other in-vehicle accessory installed in the vehicle cabin. In this case, the target display position can be determined in a display area other than the target area. For example, the presence of visual interference (e.g., an obstructing object) within the third user's field of view of the target area can be determined by recognizing the in-cabin environment image captured by the vehicle's in-vehicle camera.

[0082] by Figure 11Take an example to illustrate, the third user is the main driving user, the target area is the second display area 203, and the target display element is the UI element F with the highest priority in the second display area 203. Since there is visual interference within the third user's field of view of the target area, the target display position of the UI element F is determined in other display areas. For example, the target display position is determined in the third display area 204. However, it should be noted that although the following description is given using the third display area 204 as an example, those skilled in the art will understand that the target display position can also be determined in the first display area 202. In the third display area 204, display position 2041 is optimal for the third user, and therefore, display position 2041 is determined to be the target display position of UI element F. The controller can adjust the UI element F to display position 2041. For example, Figure 12 As shown in FIG, this adjustment can be achieved by swapping the display positions of UI element F and UI element G. Alternatively, as shown in FIG. Figure 13 As shown, the controller can adjust UI element F to display position 2041, and at the same time adjust the display ratio or size of UI element G and UI element F at display position 2041 to avoid the situation where UI elements cover or exceed the display range of the display area.

[0083] In other scenarios, when the priority of the target display element is so high that it exceeds the priority of the display elements in another display area, the target display position can be determined in the other display area, wherein the other display area is a display area with a better field of view for the third user. For example, the target area is the second display area 203, and the third user is the main driver. At this time, the first display area 202 has a better field of view for the main driver. Therefore, when the priority of the target display element in the second display area 203 is so high that it exceeds the priority of the display elements in the first display area 202, for example, the priority of UI element F is higher than the priority of each display element in the first display area 202, then the target display position (display position 2023) for the third user can be determined in the first display area 202, and the target display element can be adjusted to the target display position in the first display area 202. For example, as Figure 14 As shown, the UI element F is adjusted to the display position 2023 in the first display area 202. This adjustment can be achieved by exchanging the display positions of the UI element F and the UI element A, or as shown in FIG. Figure 15 As shown, the UI element F is adjusted to the display position 2023, and the display ratio or size of the UI element A and the UI element F at the display position 2023 is adjusted to avoid the situation where the UI elements cover or exceed the display range of the display area.

[0084] If the third user is the passenger, the third display area 204 offers a better field of view. Therefore, when the priority of the target display element in the second display area 203 exceeds the priority of the display elements in the third display area 204—for example, if the priority of UI element F is higher than the priority of each display element in the third display area 204—a target display position (display position 2041) for the third user can be determined in the third display area 204, and the target display element can be adjusted to the target display position in the third display area 204.

[0085] In an exemplary embodiment, the priority of the display element in the target area may be determined according to at least one of the following information: the importance of the display element; the degree of attention paid by the third user to the display element; and the information category corresponding to the display element.

[0086] The importance of a display element can be pre-set at the vehicle's factory or manually set by the user, and this disclosure does not specifically limit this. For example, vehicle speed is the most important indicator in instrument information. In navigation information, the navigation map interface is the most important indicator. The higher the importance of a display element, the higher its priority.

[0087] The degree of attention paid by the third user to the display element can be determined based on the interaction information between the third user and the display element. In one embodiment, the interaction information may include the third user's line of sight attention information on the display element. For example, the line of sight of the third user can be monitored by a camera to determine the specific coordinates of the third user's line of sight, and then the line of sight interaction between the third user and the display element corresponding to the coordinates can be recorded, and the third user's level of attention to the display element can be updated accordingly. In other embodiments, the interaction information may include information on the interactive operations performed by the third user on the display element through switches or buttons, or voice, or gestures. When it is detected that the third user has interacted with the display element, the third user's level of attention to the display element can be updated accordingly. The level of attention can be characterized by at least one of the following information: number of attentions, frequency of attention, duration of attention, etc. The higher the third user's level of attention to the display element, the higher the priority of the display element.

[0088] The information categories of display elements may include, but are not limited to, instrument information, navigation information, or entertainment information. The priority of each category of display elements can be pre-set for each user. For example, if the third user is the primary driver, the display elements of instrument information and navigation information have a higher priority than the display elements of entertainment information. If the third user is the passenger driver, the display elements of entertainment information and navigation information have a higher priority than the display elements of instrument information.

[0089] The priority of a display element can be determined based on any one or more of the three types of information above. When determining the priority of a display element based on multiple types of the three types of information above, as an example, a priority value for the display element can be determined based on each type of information, and then a weighted average of these priority values ​​can be taken to obtain the final priority information for the display element. The weights corresponding to the various types of information can be pre-set. As an example, the weight corresponding to the importance of the display element and the weight corresponding to the third user's attention to the display element are greater than the weight corresponding to the information category of the display element.

[0090] Figure 16 and Figure 17 A schematic diagram of a scenario for adjusting display elements according to an exemplary embodiment of the present disclosure is shown. In this implementation scenario, the target area is the second display area 203. Three UI elements are displayed on the second display area 203, namely a UI element 301 for displaying vehicle speed and speed limit information, a UI element 302 for displaying the speed information of the drive motor, and a UI element 303 for displaying the current driving mode of the vehicle. Among them, UI element 301 is the display element with the highest priority in the second display area 203. The target display element is set to be the display element with the highest priority in the second display area 203, so UI element 301 is the target display element. These three UI elements have initial display positions in the second display area 203, for example, from left to right, they are UI element 302, UI element 301 and UI element 303. Users include the main driver and the co-driver. The sight lines of the main driver and the co-driver are monitored. When the stay information of the main driver's sight line in the second display area 203 meets the preset conditions, the UI element 301 is adjusted to the left position in the second display area 203, such as Figure 16 When the main driver is not looking at the second display area 203, and the passenger user's sight line stay information in the second display area 203 meets the preset conditions, the UI element 301 is adjusted to the right position in the second display area 203, as shown. Figure 17 Afterwards, once the driver's sight line stays in the second display area 203 and the preset condition is met again, the UI element 301 is adjusted to the left position in the second display area 203 again.

[0091] Figure 18 FIG. 1 is a block diagram of a display device according to an exemplary embodiment. Figure 18 The display device includes a monitoring module 1401 , a determining module 1402 and an adjusting module 1403 .

[0092] The monitoring module 1401 is configured to monitor the user's sight data through at least one camera; The determining module 1402 is configured to determine information about the user's sight line staying in a target area based on the sight line data, wherein the target area is a display area of ​​the in-vehicle display device; The adjustment module 1403 is configured to adjust the display elements in the target area if the stay information meets a preset condition.

[0093] In one embodiment, the stay information includes a stay duration, and the preset condition includes: the stay duration reaches a preset duration.

[0094] In one embodiment, the stay information includes the number of stays, and the preset condition includes: the number of stays reaches a preset number.

[0095] In one embodiment, the sight line data includes a sight line direction.

[0096] In one embodiment, the display device is configured to project onto a front windshield of a vehicle to form a display area extending from one side of the front windshield to the other side.

[0097] In one embodiment, the display area of ​​the display device extends from one side of the vehicle center console to the other side.

[0098] In one embodiment, the display device includes at least two display areas, the lateral widths of the at least two display areas are fixed, and the target area is one of the display areas of the display device.

[0099] In one embodiment, the at least two display areas are arranged horizontally.

[0100] In one embodiment, the user includes a first user and / or a second user, the first user is located in a driving seat of the vehicle, and the second user is located in a non-driving seat of the vehicle.

[0101] In one embodiment, the second user is located in a non-driving seat of the vehicle, including: the second user is located in a front passenger seat of the vehicle; or the second user is located in a back seat of the vehicle.

[0102] In one embodiment, the users include a first user and / or a second user, the first user is located in a first non-driving seat of the vehicle, and the second user is located in a second non-driving seat of the vehicle.

[0103] In one embodiment, there are at least two users; the adjustment module 1403 is configured to: if the stop information meets the preset conditions, adjust the display elements in the target area according to the line of sight of a third user, wherein the third user is the user with the highest priority among the users corresponding to the stop information that meets the preset conditions.

[0104] In one embodiment, the adjustment module 1403 is configured to: if the third user is the user with the highest priority among all users, adjust the display elements in the target area according to the line of sight of the third user.

[0105] In one embodiment, the adjustment module 1403 is configured to: if the third user is not the user with the highest priority among all users, then when the fourth user is not looking at the target area, adjust the display elements within the target area according to the line of sight of the third user, wherein the priority of the fourth user is higher than that of the third user.

[0106] In one embodiment, the adjustment module 1403 is configured to: determine the target display element and the target display position of the target display element within the target area according to the priority of the display elements within the target area; and adjust the target display element to the target display position in response to the target display position being inconsistent with the current display position of the target display element.

[0107] In one embodiment, the target display element is a preset number of display elements that are ranked first in descending order of priority within the target area, or a display element whose priority is higher than a preset threshold.

[0108] In one embodiment, the target display position is located within the target area.

[0109] In one embodiment, the display device includes at least two display areas, the lateral widths of the at least two display areas are fixed, the target area is one of the display areas of the display device; the target display position is located in a display area of ​​the display device different from the target area.

[0110] In one embodiment, the priority of the display element in the target area is determined according to at least one of the following information: the importance of the display element; the degree of attention paid by the third user to the display element; and the information category corresponding to the display element.

[0111] Regarding the display device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the display method, and will not be elaborated here.

[0112] Figure 19 FIG. 1 is a block diagram of a display device according to an exemplary embodiment. Figure 19 The display device includes: a processor 1501 and a memory 1502 for storing executable instructions of the processor 1501; wherein the processor 1501 is configured to execute the display method provided by any of the above embodiments.

[0113] Figure 20 1 is a block diagram illustrating a vehicle 1600 according to an exemplary embodiment. For example, vehicle 1600 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or another type of vehicle. Vehicle 1600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0114] Reference Figure 20 Vehicle 1600 may include various subsystems, such as an infotainment system 1610, a perception system 1620, a decision control system 1630, a drive system 1640, a computing platform 1650, and a display system 1660. Vehicle 1600 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of vehicle 1600 may be interconnected via wired or wireless means.

[0115] In some embodiments, the infotainment system 1610 may include a communication system, an entertainment system, a navigation system, and the like.

[0116] In some embodiments, display system 1660 may include at least one camera 1661 for capturing a user's facial image, which may be used by computing platform 1650 to determine the user's gaze direction. Display system 1660 may also include a display device 1662, which may include one or more of the following: a physical screen, a projection display device.

[0117] Perception system 1620 may include several sensors for sensing information about the environment surrounding vehicle 1600. For example, perception system 1620 may include a global positioning system (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.

[0118] The decision control system 1630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0119] Drive system 1640 may include components that provide powered motion for vehicle 1600. In one embodiment, drive system 1640 may include an engine, a power source, a transmission system, and wheels. The engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination thereof. The engine is capable of converting energy provided by the power source into mechanical energy.

[0120] Some or all functions of the vehicle 1600 are controlled by a computing platform 1650. The computing platform 1650 may include at least one processor 1651 and a memory 1652. The processor 1651 may execute instructions 1653 stored in the memory 1652.

[0121] Processor 1651 can be any conventional processor, such as a commercially available CPU. The processor can also include a graphics processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.

[0122] Memory 1652 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0123] In addition to instructions 1653 , memory 1652 may also store data, such as road maps, route information, and vehicle location, direction, speed, etc. The data stored in memory 1652 may be used by computing platform 1650 .

[0124] In the embodiment of the present disclosure, the processor 1651 may execute the instruction 1653 to complete all or part of the steps of the above-mentioned display method.

[0125] In another exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it is used to implement the steps of the above-mentioned display method.

[0126] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and has a code portion for executing the above display method when executed by the programmable device.

[0127] Furthermore, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word exemplary is intended to present concepts in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X applies to Y or Z" is intended to mean any of the natural inclusive permutations. That is, if X applies to Y; X applies to Z; or X applies to both Y and Z, then "X applies to Y or Z" satisfies any of the aforementioned instances. Furthermore, the articles "a" and "an," as used in this application and the appended claims, are generally understood to mean "one or more," unless otherwise specified or clear from the context to indicate a singular form.

[0128] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. With particular regard to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if not structurally equivalent to the disclosed structure. In addition, although particular features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms "include," "have," "have," "have," or variations thereof are used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term "comprising."

[0129] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

[0130] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0131] In the foregoing detailed description, reference is made to the accompanying drawings, which illustrate, by way of illustration, specific aspects of the present disclosure in which it may be practiced. In this regard, terms indicating directions or expressing positional relationships, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," "clockwise," "counterclockwise," "axial," "radial," "circumferential," etc., may be used with reference to the orientation of the figures being described. Since the components of the described devices may be positioned in a plurality of different orientations, the directional terms may be used for illustrative purposes rather than restrictive. It should be understood that other aspects may be utilized and structural or logical changes may be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be taken in a limiting sense.

[0132] It should be understood that, unless otherwise specifically noted, the features of the various embodiments of the present disclosure described herein may be combined with each other. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more thereof; similarly, "at least one of" includes any one of the relevant listed items and any combination of any two or more thereof.

[0133] It should be understood that, unless otherwise expressly specified or limited, the terms "join," "attach," "install," "connect," "connect," "fix," etc. used in the embodiments of the present disclosure should be understood in a broad sense. For example, they can be fixedly connected, detachably connected, or integrated; they can be mechanically connected, electrically connected, or communicable with each other; they can be directly connected, or indirectly connected through an intermediate medium, and they can be internally connected between two elements or an interactive relationship between two elements, unless otherwise expressly limited. For those skilled in the art, the specific meanings of the above terms in this article can be understood according to specific circumstances.

[0134] Additionally, the term "over" as used in reference to a component, element, or material layer being formed "over" or located "over" a surface may be used herein to mean that the component, element, or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements, or layers are disposed between the surface and the component, element, or material layer. However, the term "over" as used in reference to a component, element, or material layer being formed "over" or located "over" a surface may alternatively have a specific meaning: the component, element, or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, e.g., in direct contact with the surface.

[0135] Although terms such as "first", "second" and "third" may be used herein to describe various components, parts, regions, layers or sections, these components, parts, regions, layers or sections are not limited to these terms. On the contrary, these terms are only used to distinguish one component, part, region, layer or section from another component, part, region, layer or section. Therefore, without departing from the teachings of each example, the first component, part, region, layer or section mentioned in the examples described herein may also be referred to as the second component, part, region, layer or section. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one such feature. In the description herein, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise clearly and specifically defined.

[0136] It should be understood that spatially relative terms, such as "above," "upper," "below," and "lower," are used herein to describe the relationship of one element to another element shown in the figures. Such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, an element described as being "above" or "upper" relative to another element would then be "below" or "lower" relative to the other element. Thus, the term "above" encompasses both above and below orientations, depending on the spatial orientation of the device. The device may be oriented in other ways (e.g., rotated 90 degrees or in other orientations), and the spatially relative terms used herein should be interpreted accordingly.

Claims

1. A display method, characterized in that: include: Monitoring the user's gaze data through at least one camera; Determining, based on the sight line data, information about the user's sight line staying in a target area, wherein the target area is a display area of ​​an in-vehicle display device; If the stay information meets a preset condition, the display elements in the target area are adjusted.

2. The display method according to claim 1, wherein: The stay information includes a stay duration, and the preset condition includes: the stay duration reaches a preset duration.

3. The display method according to claim 1, wherein: The stay information includes the number of stays, and the preset condition includes: the number of stays reaches a preset number.

4. The display method according to claim 1, wherein: The sight line data includes a sight line direction.

5. The display method according to claim 1, wherein: The display device is used for projecting onto a front windshield of a vehicle to form a display area extending from one side of the front windshield to the other side.

6. The display method according to claim 1, wherein: The display area of ​​the display device extends from one side of the vehicle center console to the other side.

7. The display method according to claim 1, wherein: The display device includes at least two display areas, the horizontal widths of the at least two display areas are fixed, and the target area is one of the display areas of the display device.

8. The display method according to claim 7, wherein: The at least two display areas are arranged horizontally.

9. The display method according to claim 1, wherein: The users include a first user and / or a second user, the first user is located in a driving seat of the vehicle, and the second user is located in a non-driving seat of the vehicle.

10. The display method according to claim 9, wherein: The second user is located in a non-driving seat of the vehicle, including: The second user is located in the passenger seat of the vehicle; or The second user is located in a rear seat of the vehicle.

11. The display method according to claim 1, wherein: The users include a first user and / or a second user, the first user is located in a first non-driving seat of the vehicle, and the second user is located in a second non-driving seat of the vehicle.

12. The display method according to claim 1, wherein: There are at least two users; If the stay information satisfies a preset condition, adjusting the display elements in the target area includes: If the stay information meets the preset condition, the display elements in the target area are adjusted according to the sight line of a third user, wherein the third user is a user with the highest priority among the users corresponding to the stay information that meets the preset condition.

13. The display method according to claim 12, wherein: The adjusting the display elements in the target area according to the line of sight of the third user includes: If the third user is the user with the highest priority among all users, the display elements in the target area are adjusted according to the line of sight of the third user.

14. The display method according to claim 12, wherein: The adjusting the display elements in the target area according to the line of sight of the third user includes: If the third user is not the user with the highest priority among all users, then when the fourth user is not looking at the target area, the display elements in the target area are adjusted according to the line of sight of the third user, wherein the priority of the fourth user is higher than that of the third user.

15. The display method according to claim 13 or 14, characterized in that: The adjusting the display elements in the target area includes: determining a target display element in the target area and a target display position of the target display element according to the priority of the display elements in the target area; In response to the target display position being inconsistent with the current display position of the target display element, the target display element is adjusted to the target display position.

16. The display method according to claim 15, characterized in that: The target display elements are a preset number of display elements that are ranked first in descending order of priority within the target area, or display elements whose priorities are higher than a preset threshold.

17. The method according to claim 15, characterized in that The target display position is located within the target area.

18. The method according to claim 15, characterized in that The display device includes at least two display areas, the horizontal widths of the at least two display areas are fixed, the target area is one of the display areas of the display device; the target display position is located in a display area of ​​the display device that is different from the target area.

19. The method according to claim 15, characterized in that The priority of the display element in the target area is determined according to at least one of the following information: the importance of the display element; the degree of attention paid by the third user to the display element; and the information category corresponding to the display element.

20. A display device, characterized in that: include: A monitoring module is configured to monitor the user's sight data through at least one camera; a determination module configured to determine, based on the sight line data, information about the user's sight line staying in a target area, wherein the target area is a display area of ​​an in-vehicle display device; The adjustment module is configured to adjust the display elements in the target area if the stay information meets a preset condition.

21. A display device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Monitoring the user's gaze data through at least one camera; Determining, based on the sight line data, information about the user's sight line staying in a target area, wherein the target area is a display area of ​​an in-vehicle display device; If the stay information meets a preset condition, the display elements in the target area are adjusted.

22. A vehicle, characterized in that: include: at least one camera, configured to capture a facial image of a user, wherein the facial image is used to determine line of sight data of the user; The display device according to claim 21.

23. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 19 are implemented.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 19 are implemented.

Images