Vehicle-mounted information display method, system and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle-mounted information display method, system and device, electronic equipment and a storage medium. According to the specific implementation scheme, first information needing to be displayed by mixed reality equipment worn by a user in a vehicle is determined according to the state of the vehicle; determining a target visual area watched by the user in the field of view of the corresponding mixed reality equipment according to the head posture of the user and the sight line of the user; determining second information needing to be displayed by the mixed reality equipment according to the target visual area; determining a display mode of the first information according to the target visual area; and displaying the vehicle-mounted information of the vehicle in the mixed reality equipment according to the first information, the display mode of the first information and the second information. According to the scheme, it can be guaranteed that the vehicle-mounted information is always displayed in front of the eyes of the user, the driving danger caused by out-of-focus sight of the user is avoided, and the driving concentration degree and the driving response time of the user are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method, system, device, electronic device and storage medium for displaying vehicle-mounted information. Background Art

[0002] Currently, the display of in-vehicle information is mainly achieved through in-vehicle display screens or head-up display devices. Both display methods have their own limitations. Among them, the in-vehicle display screen requires the driver to defocus from the front of the vehicle to the in-vehicle display screen, which introduces risks to driving. Head-up display devices can prevent the driver's line of sight from being out of focus, but there are problems such as small display area, high cost, and easy damage to optical devices. Summary of the invention

[0003] The present disclosure provides a method, system, device, electronic device and storage medium for displaying vehicle information to solve or alleviate one or more technical problems in the prior art.

[0004] In one aspect, the present disclosure provides a method for displaying vehicle information, comprising:

[0005] Determining, according to the state of the vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle;

[0006] Determine, based on the user's head posture and the user's sight line, a target visual area that the user is looking at in the field of view of the corresponding mixed reality device;

[0007] Determining, according to the target viewing area, second information to be displayed by the mixed reality device;

[0008] Determining a display mode of the first information according to the target visual area; and

[0009] Display of the vehicle's in-vehicle information is implemented in the mixed reality device according to the first information, the display method of the first information and the second information.

[0010] In another aspect, the present disclosure provides a vehicle information display system, comprising:

[0011] An image acquisition device, used to acquire environmental information corresponding to the vehicle;

[0012] A data collection device, used to collect vehicle information corresponding to the vehicle;

[0013] A controller, electrically connected to the image acquisition device and the data acquisition device, and configured to execute the method of any embodiment of the present disclosure; and

[0014] The mixed reality device is electrically connected to the controller and is used to display the vehicle's onboard information.

[0015] In another aspect, the present disclosure provides a display device for vehicle-mounted information, comprising:

[0016] A first determination module, configured to determine, according to a state of the vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle;

[0017] A second determination module is used to determine a target visual area that the user is looking at in the field of view of the corresponding mixed reality device according to the user's head posture and the user's line of sight;

[0018] A third determination module is used to determine the second information to be displayed by the mixed reality device according to the target visual area;

[0019] a fourth determining module, configured to determine a display mode of the first information according to a target visual area; and

[0020] The display module is used to realize the display of the vehicle's in-vehicle information in the mixed reality device according to the first information, the display mode of the first information and the second information.

[0021] In another aspect, the present disclosure provides an electronic device, comprising:

[0022] At least one processor. And

[0023] A memory in communication with the at least one processor.

[0024] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any embodiment of the present disclosure.

[0025] On the other hand, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any embodiment of the present disclosure.

[0026] On the other hand, the present disclosure provides a computer program product, including a computer program, which implements the method according to any embodiment of the present disclosure when executed by a processor.

[0027] According to the scheme disclosed in the present invention, it can be ensured that the in-vehicle information is always displayed in front of the user, avoiding driving hazards caused by the user's out-of-focus vision, and improving the user's driving concentration and driving reaction time.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments provided according to the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0030] Figure 1 It is a flowchart of a method for displaying in-vehicle information according to an embodiment of the present disclosure.

[0031] Figure 2 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0032] Figure 3 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0033] Figure 4 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0034] Figure 5 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0035] Figure 6 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0036] Figure 7 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0037] Figure 8 It is a schematic diagram of the application of the method for displaying vehicle information according to an embodiment of the present disclosure.

[0038] Fig. 9 is a schematic diagram of a vehicle information display system according to an embodiment of the present disclosure.

[0039] Fig.10 is a schematic diagram of a display device for in-vehicle information according to an embodiment of the present disclosure.

[0040] Fig.11 It is a block diagram of an electronic device used to implement the display of vehicle-mounted information according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The present disclosure will be further described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0042] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0043] like Figure 1 As shown, the embodiment of the present disclosure provides a method for displaying vehicle information, including:

[0044] Step S101: Determine, according to the state of the vehicle, the first information to be displayed by the mixed reality (MR) device worn by the user in the vehicle.

[0045] Step S102: Determine the target visual area that the user is looking at in the field of view of the corresponding mixed reality device according to the user's head posture and the user's line of sight.

[0046] Step S103: Determine the second information to be displayed by the mixed reality device according to the target viewing area.

[0047] Step S104: Determine the display mode of the first information according to the target viewing area.

[0048] Step S105: Based on the first information, the display mode of the first information and the second information, the vehicle's in-vehicle information is displayed in the mixed reality device.

[0049] According to the embodiments of the present disclosure, it should be noted that:

[0050] The state of the vehicle may at least include the driving state of the vehicle and the starting state of the vehicle. The starting state of the vehicle may be understood as the vehicle starting state or the vehicle starting but still in a parked state.

[0051] The user in the vehicle can be understood as a driver or a passenger. When the driver wears a mixed reality device, the method of any embodiment of the present disclosure can be used to achieve safe driving without defocusing the vision. When the passenger wears a mixed reality device, the method of any embodiment of the present disclosure can be used to achieve the same driving perspective as the driver, which can assist the driver in driving safely and remind the driver of possible dangerous situations at any time.

[0052] The user's head posture and user's line of sight can be determined based on the mixed reality device worn by the user, or can be determined based on image capture and recognition of the user's head and face by a camera installed in the car.

[0053] Field of view can be understood as the horizontal field of view (FOV). Figure 2As shown, when the user 200 in the vehicle 100 wears the mixed reality device 300, the horizontal field of view corresponding to the mixed reality device 300 is divided into multiple viewing areas. Figure 2 The upper middle part shows a top view of the user's head and multiple viewing zones divided into horizontal viewing areas based on the top view. The multiple viewing zones include binocular viewing zones and monocular viewing zones; when the user's sight is in the normal viewing zone in the binocular viewing zone, it means that the user is looking straight ahead; when the user's sight is in the peripheral viewing zone in the binocular viewing zone, it means that the user is paying attention to the outer rearview mirror or the A-pillar area; when the user's sight is in the monocular viewing zone, it means that the user is paying attention to the B-pillar or the area between the B-pillar and the C-pillar. When the user's sight is in the binocular viewing zone, it means that the user is driving safely, and when the user's sight is in the monocular viewing zone, it means that the user is not paying attention to the environment in front of the vehicle and is in a dangerous driving state.

[0054] The target viewport can be understood as Figure 2 Any one of the multiple view areas divided in .

[0055] The first information can be understood as information that the user needs to pay regular attention to during the driving process of the vehicle, that is, information that is displayed for a long time. The first information is not limited to the vehicle's external environment information, vehicle speed information, battery information, fuel information, navigation information (navigation arrows, destination buoys), road information, turn signal information, etc. displayed by the exterior rearview mirror and the interior rearview mirror (such as Figures 3 to 8 For example, the red dotted box A in the upper left corner of each figure shows the external environment information of the side and rear of the vehicle displayed by the left exterior rearview mirror; the upper right corner of each figure shows the external environment information of the side and rear of the vehicle displayed by the right exterior rearview mirror; the top middle of each figure shows the external environment information of the rear of the vehicle displayed by the interior rearview mirror; the bottom middle of each figure shows the vehicle speed information, fuel information, power information, etc.; the first information shown in the lower left corner, lower right corner and middle of each view is navigation information and road information, etc. It should be noted that each view is only used for illustration, and the content and position of the first information displayed specifically can be selected and adjusted as needed, and are not limited to those shown in each view.

[0056] When the vehicle is in different states, the first information displayed to the user by the mixed reality device may be different, depending on the vehicle information that is of interest when the vehicle is driving and when the vehicle is started, for example, Figure 3 The figure shows the first information displayed by the mixed reality device when the vehicle is driving, and the first information includes external environment information. Figure 6 The first information displayed by the mixed reality device when the vehicle is started is shown, and the first information includes interior environment information ( Figure 6 The red dotted box A in the lower right corner) and external environment information.

[0057] The mixed reality device can display multiple first information at the same time, and there is no limit on the number of first information.

[0058] The display mode of the first information can be understood as magnifying, reducing, flashing, displaying the first information, etc. The display mode of the first information is set to highlight the first information that the user needs to focus on when the user's line of sight is focused on different visual areas. It should be noted that each first information can include a regular display mode and a special display mode. The regular display mode can be understood as the default display position and display size of the first information set in the mixed reality device. The special display mode can be understood as the non-default display position and / or display size set in the mixed reality device. For example, Figure 3 The content displayed by the left exterior rearview mirror shown in the upper left corner (first information) is a conventional display mode; Figure 4 The content (first information) displayed by the left exterior rearview mirror shown in the upper left corner is a special display mode, that is, the content displayed by the left exterior rearview mirror is magnified. Figure 3 The content displayed by the mixed reality device when the user is driving with his eyes on the front. Figure 4 It is the content displayed by the mixed reality device when the user changes his gaze from looking forward to the peripheral vision area of ​​the eye area, so that the user can see the external environment displayed by the left exterior rearview mirror.

[0059] When the first information includes the content displayed by the rearview mirror, the first information may be information collected by the electronic rearview mirror, or may be information collected by an additional image acquisition device provided at the rearview mirror, which is not specifically limited here.

[0060] The second information can be understood as information that is not often displayed during the driving process of the vehicle, that is, information that is displayed for a short time. The second information includes but is not limited to warning information, prompt information, etc. The second information can be text, static icons or dynamic icons.

[0061] According to the technology of the disclosed embodiment, the first information that the user is concerned about when driving (such as external environment information, vehicle speed information, power information, fuel information, navigation information, road information, etc.) can be always displayed in front of the user, without the user turning his head or shifting his sight to pay attention to the left rear exterior mirror or the vehicle display screen set in different positions, solving the problem of user's defocused vision (i.e., the vision is focused within the binocular vision area) and the driving danger caused by defocused vision, and improving the user's driving concentration and driving reaction time. At the same time, the mixed reality device has stronger human-computer interaction and display customization, which can provide vehicle users with more required personalized information, improve user driving safety and user experience.

[0062] In one implementation, the present disclosure provides a method for displaying in-vehicle information, including steps S101 to S105, wherein step S101: determining, according to the state of the vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle, including:

[0063] Step S1011: According to the state of the vehicle, the environment information corresponding to the vehicle is obtained based on the image acquisition device.

[0064] Step S1012: According to the state, vehicle information corresponding to the vehicle is obtained based on the data acquisition device, wherein the vehicle information includes at least one of driving information, road information, navigation information and multimedia information.

[0065] Step S1013: determine the environmental information and the vehicle information as the first information to be displayed by the mixed reality device worn by the user in the vehicle.

[0066] According to the embodiments of the present disclosure, it should be noted that:

[0067] The image acquisition device may be a camera or a sensor provided in the vehicle.

[0068] The environmental information includes external environmental information of the vehicle and / or interior environmental information of the vehicle.

[0069] The vehicle status includes driving status and starting status.

[0070] According to the technology of the embodiments of the present disclosure, different first information can be displayed to the user in a mixed reality device based on the different states of the vehicle, so as to meet the user's information attention needs when the vehicle is in different states.

[0071] In one implementation, the present disclosure provides a method for displaying vehicle information, including steps S101 to S105, and steps S1011 to S1013, wherein step S1011: acquiring environmental information corresponding to the vehicle based on an image acquisition device according to the state of the vehicle, including:

[0072] When it is determined that the state of the vehicle is a driving state, external environment information of the vehicle is acquired based on a first image acquisition device of the vehicle.

[0073] The external environment information includes at least one of the vehicle's front environment information, side and rear environment information, and rear environment information.

[0074] According to the embodiments of the present disclosure, it should be noted that:

[0075] like Figure 3 , 4, 5, 7, and 8 can be understood as the in-vehicle information displayed by the mixed reality device when the vehicle is in motion. The red dotted box A in the upper left corner of each figure shows the external environment information of the side and rear of the vehicle displayed by the left exterior rearview mirror; the upper right corner of each figure shows the external environment information of the side and rear of the vehicle displayed by the right exterior rearview mirror; the top middle of each figure shows the external environment information of the rear of the vehicle displayed by the interior rearview mirror.

[0076] According to the technology of the embodiments of the present disclosure, it is possible to display all external environmental information that the user is concerned about in the mixed reality device without the user having to turn his head or shift his gaze to pay attention to the left rear exterior mirror or the vehicle-mounted display screens set in different positions. This solves the problem of user's line of sight defocusing and the driving hazards caused by defocusing line of sight, and improves the user's driving concentration and driving reaction time.

[0077] In one implementation, the present disclosure provides a method for displaying vehicle information, including steps S101 to S105, and steps S1011 to S1013, wherein step S1011: acquiring environmental information corresponding to the vehicle based on an image acquisition device according to the state of the vehicle, including:

[0078] When it is determined that the state of the vehicle is the start-up state, the interior environment information of the vehicle and the external environment information of the vehicle are acquired based on the second image acquisition device of the vehicle.

[0079] The external environment information includes at least one of the vehicle's front environment information, side and rear environment information, and rear environment information.

[0080] According to the embodiments of the present disclosure, it should be noted that:

[0081] The interior environment information is used to remind the user whether the occupants of the vehicle are wearing seat belts. The interior environment information can be text information or graphic information.

[0082] According to the technology of the disclosed embodiment, by displaying interior environment information, it is possible to effectively remind the user whether the occupants of the vehicle are wearing seat belts, thereby preventing the occupants from forgetting to wear seat belts or not wearing seat belts, thereby improving driving safety. At the same time, it is possible to display all the external environment information and interior environment information that the user is concerned about in the mixed reality device, without the user having to turn his head or shift his sight to pay attention to the left rear exterior mirror or the vehicle display screen set in different positions, thereby solving the problem of the user's defocused vision and the driving danger caused by the defocused vision, and improving the user's driving concentration and driving reaction time.

[0083] In one implementation, the present disclosure provides a method for displaying in-vehicle information, including steps S101 to S105, wherein step S103: determining, according to a target viewing area, second information to be displayed by a mixed reality device, includes:

[0084] When it is determined that the target visual area is a monocular visual area outside the binocular visual area in the visual field, a warning message is generated, wherein the warning message is used to prompt the user to look at the binocular visual area, and the warning message includes text information and / or icon information.

[0085] The warning information is determined as the second information that the mixed reality device needs to display.

[0086] According to the embodiments of the present disclosure, it should be noted that:

[0087] like Figure 5 As shown, the green dotted box B in the figure shows the second information, which can be text information and / or icon information. When the user's sight is not focused on the binocular vision area, the user is prompted to adjust the sight to focus on the binocular vision area immediately.

[0088] When it is determined that the target viewing area is within the binocular viewing area, it is determined that the second information to be displayed by the mixed reality device is not present.

[0089] According to the technology of the embodiment of the present disclosure, by displaying the second information in the mixed reality device, driving safety can be improved and the user can be warned in real time.

[0090] In one implementation, the present disclosure provides a method for displaying vehicle-mounted information, including steps S101 to S105, wherein step S104: determining a display mode of first information according to a target visual area, includes:

[0091] When it is determined that the target visual area is a peripheral visual area within the binocular visual area in the field of view, target information corresponding to the peripheral visual area is determined, wherein the first information includes the target information, and the target information includes environmental information displayed by an exterior rearview mirror on the same side of the peripheral visual area.

[0092] A display mode of the first information is determined according to the peripheral visual area.

[0093] According to the embodiments of the present disclosure, it should be noted that:

[0094] The display mode of the first information can be understood as magnifying, reducing, flashing, displaying the first information, etc. The display mode of the first information is set to highlight the first information that the user needs to focus on when the user's line of sight is focused on different visual areas. It should be noted that each first information can include a regular display mode and a special display mode. The regular display mode can be understood as the default display position and display size of the first information set in the mixed reality device. The special display mode can be understood as the non-default display position and / or display size set in the mixed reality device. For example, Figure 3 The content displayed by the left exterior rearview mirror shown in the upper left corner (first information) is a conventional display mode; Figure 4 The content (first information) displayed by the left exterior rearview mirror shown in the upper left corner is a special display mode, that is, the content displayed by the left exterior rearview mirror is magnified. Figure 3 The content displayed by the mixed reality device when the user is driving with his eyes on the front. Figure 4 It is the content displayed by the mixed reality device when the user changes his gaze from looking forward to the peripheral vision area of ​​the eye area, so that the user can see the external environment displayed by the left exterior rearview mirror.

[0095] According to the technology of the embodiments of the present disclosure, the first information that the user wants to pay attention to during the current driving process can be inferred based on the user's current line of sight and head posture, and the first information can be highlighted in the mixed reality device, so that the user can use the mixed reality device to see the required information without causing defocus of the line of sight.

[0096] In one implementation, the present disclosure provides a method for displaying vehicle-mounted information, including steps S101 to S105, wherein step S105: according to first information, a display mode of the first information, and second information, displaying the vehicle-mounted information of the vehicle in a mixed reality device includes:

[0097] According to the user's sitting posture information, determine the field of view reference point of the user's corresponding mixed reality device.

[0098] Based on the field of view reference point, according to the first information, the display mode of the first information and the second information, the display of the vehicle's in-vehicle information is implemented in the mixed reality device.

[0099] According to the embodiments of the present disclosure, it should be noted that:

[0100] The sitting posture information at least includes height information of the user's head.

[0101] According to the technology of the embodiments of the present disclosure, it is possible to adaptively adjust the display positions of the first information and the second information in the mixed reality device for users of different heights, thereby ensuring that the first information and the second information can be accurately superimposed and mixed with the road environment ahead seen by the naked eyes of users of different heights in the mixed reality device, thereby avoiding the first information and the second information being superimposed and mixed and displayed at an incorrect position in the road environment ahead.

[0102] In one example, the process of determining the field of view reference point includes: the in-car camera tracks the driver's line of sight position, which is used to display different UI interfaces when the driver observes different positions. The camera is installed in the direction of the steering wheel. According to the camera installation position, its optical center is used as the origin, the X axis points to the camera's observation direction, and the Y axis is parallel to the image plane and points to the right side of the camera, so as to construct a "camera coordinate system". And the camera is collected on the B-pillar to mark the reference point, so as to construct a world coordinate system. Use a function to convert the disparity map and depth map collected by the camera into three-dimensional coordinates. Obtain the feature points of the driver's eye positions, and map the pixel coordinates of the feature points of the eye positions to the camera coordinate system. And convert to the position in the world coordinate system. After obtaining the front camera picture, it is necessary to deflect the camera-captured image based on the driver's sitting posture information calculated by the camera in the cockpit so that the image is transformed to an image based on the driver's perspective and finally presented in the MR device. To achieve the above function, a homography matrix transformation can be used, and the steps are as follows: the homography matrix of the driver's horizon is obtained by the known position relationship between the front camera of the vehicle body and the driver's face capture camera in the cockpit, and the calculated position relationship between the driver's horizon reference and the driver's face capture camera in the cockpit. The converted image of the driver's field of view calculated by the camera in the cockpit will display the image range of the driver's field of view. The color and brightness values ​​of each part within the field of view are calculated in the converted image. These values ​​will be used to select safe and eye-catching interface colors during MR imaging.

[0103] In one implementation, the present disclosure provides a method for displaying vehicle information, including steps S101 to S105, and further including:

[0104] When a color blindness display instruction is received, the vehicle information corresponding to the color blindness display is marked and displayed.

[0105] and / or

[0106] When it is determined that the ambient light in front of the vehicle changes color, the vehicle-mounted information that is consistent with the color change is changed in color and displayed.

[0107] According to the embodiments of the present disclosure, it should be noted that:

[0108] The in-vehicle information corresponding to the color blind display may at least include traffic light information.

[0109] Mark and display, such as Figure 7 As shown in the position indicated by C in the middle, the traffic light is annotated with text and / or graphics, and the text and / or graphics are mixed and superimposed on the traffic light shown by the user in the mixed reality device.

[0110] The vehicle information that matches the color change is changed and displayed, such as Figure 8 As shown in the position indicated by D in the middle, when the external background environment corresponding to the position where the color of the first information displayed by the mixed reality device is located is similar to the color of the icon, the color of the first information can be adaptively adjusted so that the user can distinguish the first information from the corresponding background environment in the mixed reality device.

[0111] The vehicle camera can capture images in front of the vehicle and identify the traffic lights and color changes of the ambient light.

[0112] According to the technology of the embodiments of the present disclosure, the driving needs of special users can be met, so that users can drive safely even if they are color blind, clearly know relevant driving information, and assist users in driving safely.

[0113] like Fig. 9 As shown, the embodiment of the present disclosure provides a vehicle information display system, including:

[0114] The image acquisition device 910 is used to acquire the environment information corresponding to the vehicle.

[0115] The data collection device 920 is used to collect vehicle information corresponding to the vehicle.

[0116] The controller 930 is electrically connected to the image acquisition device 910 and the data acquisition device 920, and is used to execute the method of any embodiment of the present disclosure.

[0117] The mixed reality device 940 is electrically connected to the controller 930 and is used to display the vehicle's onboard information.

[0118] According to the embodiments of the present disclosure, it should be noted that:

[0119] The image acquisition device 910 may include at least one of an in-vehicle camera, an out-vehicle camera, or a vehicle-mounted sensor.

[0120] The data acquisition device 920 may include an in-vehicle bus to obtain required vehicle information from other devices in the vehicle.

[0121] The environmental information may include: the vehicle's exterior environmental information displayed by the exterior rearview mirror and the interior rearview mirror, and / or the vehicle's interior environmental information.

[0122] Vehicle information may include: vehicle speed information, power information, fuel level information, navigation information (navigation arrows, destination buoys), road information, turn indicator light information, etc.

[0123] The controller 930 may be an ECU (electronic control unit), an MCU (microcontroller unit), or the like in a vehicle.

[0124] According to the technology of the embodiments of the present disclosure, the first information that the user is concerned about when driving (such as external environment information, vehicle speed information, power information, fuel information, navigation information, road information, etc.) can be always displayed in front of the user, without the user turning his head or shifting his sight to pay attention to the left rear exterior mirror or the vehicle display screen set in different positions, solving the problem of user's defocused vision and the driving danger caused by defocused vision, and improving the user's driving concentration and driving reaction time. At the same time, the mixed reality device has stronger human-computer interaction and display customization, which can provide vehicle users with more required personalized information and improve user driving safety and user experience.

[0125] like Fig.10 As shown, the embodiment of the present disclosure provides a display device for vehicle-mounted information, including:

[0126] The first determination module 1010 is used to determine the first information that needs to be displayed on the mixed reality device worn by the user in the vehicle according to the state of the vehicle.

[0127] The second determination module 1020 is used to determine the target visual area that the user is looking at in the field of view of the corresponding mixed reality device according to the user's head posture and the user's line of sight.

[0128] The third determination module 1030 is used to determine the second information that the mixed reality device needs to display according to the target viewing area.

[0129] The fourth determination module 1040 is used to determine the display mode of the first information according to the target viewing area.

[0130] The display module 1050 is used to realize the display of the vehicle's in-vehicle information in the mixed reality device according to the first information, the display mode of the first information and the second information.

[0131] In one implementation, the first determining module 1010 is used to:

[0132] The first acquisition submodule is used to acquire the environment information corresponding to the vehicle based on the image acquisition device according to the state of the vehicle.

[0133] The second acquisition submodule is used to acquire vehicle information corresponding to the vehicle based on the data acquisition device according to the state, wherein the vehicle information includes at least one of driving information, road information, navigation information and multimedia information.

[0134] The determination submodule is used to determine the environmental information and the vehicle information as the first information to be displayed by the mixed reality device worn by the user in the vehicle.

[0135] In one implementation, the first acquisition submodule is used to:

[0136] When it is determined that the state of the vehicle is a driving state, external environment information of the vehicle is acquired based on a first image acquisition device of the vehicle.

[0137] or

[0138] When it is determined that the state of the vehicle is the start-up state, the interior environment information of the vehicle and the external environment information of the vehicle are acquired based on the second image acquisition device of the vehicle.

[0139] The external environment information includes at least one of the vehicle's front environment information, side and rear environment information, and rear environment information.

[0140] In one implementation, the third determination module 1030 is used to:

[0141] When it is determined that the target visual area is a monocular visual area outside the binocular visual area in the visual field, a warning message is generated, wherein the warning message is used to prompt the user to look at the binocular visual area, and the warning message includes text information and / or icon information.

[0142] The warning information is determined as the second information that the mixed reality device needs to display.

[0143] In one implementation, the fourth determining module 1040 is configured to:

[0144] When it is determined that the target visual area is a peripheral visual area within the binocular visual area in the field of view, target information corresponding to the peripheral visual area is determined, wherein the first information includes the target information, and the target information includes environmental information displayed by an exterior rearview mirror on the same side of the peripheral visual area.

[0145] A display mode of the first information is determined according to the peripheral visual area.

[0146] In one embodiment, the display module 1050 is used to:

[0147] According to the user's sitting posture information, determine the field of view reference point of the user's corresponding mixed reality device.

[0148] Based on the field of view reference point, according to the first information, the display mode of the first information and the second information, the display of the vehicle's in-vehicle information is implemented in the mixed reality device.

[0149] In one embodiment, the vehicle information display device of the embodiment of the present disclosure further includes an adjustment module for:

[0150] When a color blindness display instruction is received, the vehicle information corresponding to the color blindness display is marked and displayed.

[0151] and / or

[0152] When it is determined that the ambient light in front of the vehicle changes color, the vehicle-mounted information that is consistent with the color change is changed in color and displayed.

[0153] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0154] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0155] Fig.11 FIG. 1 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Fig.11 As shown, the electronic device includes: a memory 1110 and a processor 1120, and the memory 1110 stores a computer program that can be run on the processor 1120. The number of memories 1110 and processors 1120 can be one or more. The memory 1110 can store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device executes the method provided by the above method embodiment. The electronic device may also include: a communication interface 1130, which is used to communicate with external devices and perform data exchange transmission.

[0156] If the memory 1110, the processor 1120 and the communication interface 1130 are implemented independently, the memory 1110, the processor 1120 and the communication interface 1130 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0157] Optionally, in a specific implementation, if the memory 1110, the processor 1120 and the communication interface 1130 are integrated on a chip, the memory 1110, the processor 1120 and the communication interface 1130 can communicate with each other through an internal interface.

[0158] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0159] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0160] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the process or function according to the embodiment of the present disclosure is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (for example: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (for example: infrared, Bluetooth, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by the computer, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. It is worth noting that the computer-readable storage medium mentioned in the present disclosure may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0161] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0162] In the description of the embodiments of the present disclosure, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0163] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B. "And / or" in this article is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0164] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0165] The above description is only an exemplary embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for displaying vehicle information, comprising: Determining, according to a state of the vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle; Determining a target visual area that the user is looking at in a field of view corresponding to the mixed reality device according to the user's head posture and the user's line of sight; Determining second information to be displayed by the mixed reality device according to the target visual area; Determining a display mode of the first information according to the target visual area; as well as The display of the vehicle's in-vehicle information is implemented in the mixed reality device according to the first information, the display method of the first information and the second information.

2. The method according to claim 1, wherein: Determining, according to a state of a vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle, including: According to the state of the vehicle, acquiring the environment information corresponding to the vehicle based on the image acquisition device; According to the state, obtaining vehicle information corresponding to the vehicle based on a data acquisition device, wherein the vehicle information includes at least one of driving information, road information, navigation information and multimedia information; The environmental information and the vehicle information are determined as first information to be displayed by a mixed reality device worn by a user in the vehicle.

3. The method according to claim 2, wherein: According to the state of the vehicle, the environment information corresponding to the vehicle is obtained based on the image acquisition device, including: When it is determined that the state of the vehicle is a driving state, acquiring external environment information of the vehicle based on a first image acquisition device of the vehicle; or When it is determined that the state of the vehicle is the start-up state, acquiring interior environment information of the vehicle and external environment information of the vehicle based on a second image acquisition device of the vehicle; The external environment information includes at least one of the front environment information, the side and rear environment information, and the rear environment information of the vehicle.

4. The method according to claim 1, wherein: Determining, according to the target visual area, second information to be displayed by the mixed reality device, including: In the case where it is determined that the target visual area is a monocular visual area outside the binocular visual area in the visual field, generating a warning message, wherein the warning message is used to prompt the user to look at the binocular visual area, and the warning message includes text information and / or icon information; The warning information is determined as the second information that the mixed reality device needs to display.

5. The method according to claim 1, wherein: Determining, according to the target visual area, a display mode of the first information, including: In the case where it is determined that the target visual area is a peripheral visual area within the binocular visual area in the visual field, determining target information corresponding to the peripheral visual area, wherein the first information includes the target information, and the target information includes environmental information displayed by an exterior rearview mirror on the same side of the peripheral visual area; A display mode of the first information is determined according to the peripheral visual area.

6. The method according to any one of claims 1 to 5, wherein: Implementing display of the vehicle-mounted information of the vehicle in the mixed reality device according to the first information, the display mode of the first information, and the second information, includes: Determining a field of view reference point of the user corresponding to the mixed reality device according to the sitting posture information of the user; Based on the field of view reference point, according to the first information, the display mode of the first information and the second information, the display of the vehicle's in-vehicle information is implemented in the mixed reality device.

7. The method according to any one of claims 1 to 5, wherein: After the display of the vehicle-mounted information of the vehicle is implemented in the mixed reality device according to the first information, the display mode of the first information and the second information, the method further includes: When receiving a color blindness display instruction, marking and displaying the vehicle information corresponding to the color blindness display; and / or When it is determined that the ambient light in front of the vehicle changes color, the vehicle-mounted information that is consistent with the color change is changed in color and displayed.

8. A vehicle information display system, comprising: An image acquisition device, used to acquire environmental information corresponding to the vehicle; A data collection device, used to collect vehicle information corresponding to the vehicle; A controller, electrically connected to the image acquisition device and the data acquisition device, and configured to execute the method according to any one of claims 1 to 7; as well as A mixed reality device is electrically connected to the controller and is used to display the vehicle's onboard information.

9. A vehicle information display device, comprising: A first determining module, configured to determine, according to a state of a vehicle, first information to be displayed on a mixed reality device worn by a user in the vehicle; A second determination module is used to determine a target visual area that the user is looking at in the field of view corresponding to the mixed reality device according to the head posture of the user and the user's line of sight; A third determination module, configured to determine, according to the target visual area, second information to be displayed by the mixed reality device; a fourth determining module, configured to determine a display mode of the first information according to the target visual area; as well as A display module is used to display the vehicle's in-vehicle information in the mixed reality device according to the first information, the display mode of the first information and the second information.

10. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images