Head-up display device, display control method, device and storage medium

By obtaining the reference line and stop line angle changes of the bend intersection, and adjusting the angle of the AR-HUD navigation guidance in real time, the problem that navigation guidance in the existing technology cannot change the posture in real time is solved, and the navigation accuracy and driver experience of the bend intersection are improved .

CN117863873BActive Publication Date: 2024-11-22JIANGSU NEW VISION AUTOMOTIVE ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410071496.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-17
Publication Date
2024-11-22
Estimated Expiration
2044-01-17

AI Technical Summary

Technical Problem

The existing AR-HUD cannot provide continuous and accurate signals at bend intersections, resulting in the inability to change the posture in real time, affecting the user experience.

Method used

By obtaining the reference line of the target bend intersection and the angle change of the stop line, the angle change of the navigation guide is controlled in real time so that the driver can intuitively feel the vehicle's exit from the bend intersection.

Benefits of technology

It achieves intuitive and accurate navigation and guidance at bend intersections, improving driving safety and experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117863873B_ABST
    Figure CN117863873B_ABST
Patent Text Reader

Abstract

The present disclosure provides a head-up display device, a display control method, a device and a storage medium, which belong to the field of assisted driving technology; the method includes: obtaining a baseline of a target turning intersection, the baseline is used to indicate the entrance of the target turning intersection; obtaining a first angle change between the exit stop line and the baseline during the turning process; and controlling the second angle change of the navigation guidance according to the first angle change. In this way, at the turning intersection, as the vehicle moves at the turning intersection, the angle of the navigation guidance changes continuously in real time, and the driver can intuitively feel the distance between the current vehicle and the exit of the turning intersection through the posture of the navigation guidance, thereby improving the driver's driving experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of assisted driving technology, and in particular to a head-up display device, a display control method, a device and a storage medium. Background Art

[0002] Augmented Reality-Head Up Display (AR-HUD) is an in-vehicle interactive system that integrates augmented reality technology, head-up display technology and multi-information fusion technology.

[0003] In the prior art, it is impossible to provide continuous and accurate signals to the AR-HUD at a turning intersection. Therefore, the navigation guidance cannot change its posture in real time (for example, when the vehicle has already driven out of the curve, the AR-HUD continues to display the turning animation), resulting in a poor user experience. Summary of the invention

[0004] The present invention provides a head-up display device, a display control method, a device and a storage medium, which can provide intuitive and accurate navigation guidance at turning intersections and improve driving safety.

[0005] The technical solution of the present disclosure is achieved as follows:

[0006] In a first aspect, the present disclosure provides a display control method, the method comprising:

[0007] Obtaining a baseline of a target turning intersection, where the baseline is used to indicate an entrance to the target turning intersection;

[0008] Acquire a first angle change between the stop line and the reference line during the turning process;

[0009] According to the first angle change, a second angle change of the navigation guidance is controlled.

[0010] In a second aspect, the present disclosure provides a display control device, the device comprising: an acquisition part and a control part;

[0011] An acquisition part is configured to acquire a baseline of a target turning intersection, where the baseline is used to indicate an entrance to the target turning intersection;

[0012] The acquisition part is further configured to acquire a first angle change with the reference line during the turning process when driving out of the stop line;

[0013] The control part is configured to control the change of the second angle of the navigation guidance according to the change of the first angle.

[0014] In a third aspect, the present disclosure provides a display control device, comprising: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the display control method as described in the first aspect.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the display control method as described in the first aspect.

[0016] In a fifth aspect, the present disclosure provides a head-up display device, the head-up display device comprising a display control unit and a display unit;

[0017] in,

[0018] A display control unit is configured to obtain a reference line of a target turning intersection, wherein the reference line is used to indicate an entrance to the target turning intersection;

[0019] and obtaining a first angle change between the vehicle exiting the stop line and the reference line during the turning process;

[0020] and, configured to control a change in a second angle of the navigation guide according to the change in the first angle;

[0021] The display unit is configured to project the navigation guidance onto the windshield of the vehicle for display based on the control of the display control unit.

[0022] In a sixth aspect, the present disclosure provides a vehicle, comprising the head-up display device described in the fifth aspect.

[0023] The present disclosure provides a head-up display device, a display control method, a device and a storage medium. The method includes: obtaining a baseline of a target turning intersection, the baseline is used to indicate the entrance of the target turning intersection; obtaining a first angle change between the exit stop line and the baseline during the turning process; and controlling a second angle change of the navigation guidance according to the first angle change. In this way, at the turning intersection, as the vehicle moves at the turning intersection, the angle of the navigation guidance changes continuously in real time, and the driver can intuitively feel the distance between the current vehicle and the exit of the turning intersection through the posture of the navigation guidance, thereby improving the driver's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of the composition of a vehicle-mounted system provided by the present invention.

[0025] Figure 2 An exemplary top view of a vehicle provided for the present disclosure.

[0026] Figure 3 An exemplary perspective view from the driver's seat of a vehicle is provided for the present disclosure.

[0027] Figure 4 A schematic diagram of the architecture of the head-up display device provided by the present disclosure.

[0028] Figure 5a One of the schematic diagrams of the navigation guidance provided by the present disclosure.

[0029] Figure 5b The second schematic diagram of the navigation guidance provided by the present invention.

[0030] Figure 6 The third schematic diagram of the navigation guidance provided by the present invention.

[0031] Figure 7 This is one of the flow charts of the display control method provided by the present disclosure.

[0032] Figure 8 A schematic diagram of entering a stop line and exiting a stop line provided by the present disclosure.

[0033] Fig. 9 This is one of the schematic diagrams of a frame of environment image provided by the present disclosure.

[0034] Fig.10 A schematic diagram of the angle change provided in the present disclosure.

[0035] Fig.11 The fourth schematic diagram of the navigation guidance provided by the present invention.

[0036] Fig.12 The second flowchart of the display control method provided by the present invention.

[0037] Fig.13 The fifth schematic diagram of the navigation guidance provided for the present invention.

[0038] Fig.14 Schematic diagram of the baseline provided for the present disclosure.

[0039] Fig.15 This is a third flow chart of the display control method provided by the present invention.

[0040] Fig.16 The second schematic diagram of a frame of environmental image provided by the present invention.

[0041] Fig.17 This is a fourth flow chart of the display control method provided by the present invention.

[0042] Fig.18 A schematic diagram of various coordinate points in a frame of environmental image provided by the present disclosure.

[0043] Fig.19 This is the fifth flow chart of the display control method provided by the present invention.

[0044] Fig. 20 This is the sixth flow chart of the display control method provided by the present invention.

[0045] Fig.21 A schematic diagram of arc length provided for the present disclosure.

[0046] Fig. 22 A schematic diagram of the distance to the exit of a turning intersection provided by the present invention.

[0047] Fig.23 A schematic diagram of the composition of a display control device provided by the present invention.

[0048] Fig.24 A schematic diagram of the structure of a display control device provided by the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the present disclosure to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of the present application.

[0050] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. The technical solution in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure.

[0051] See also Figure 1 , which shows an example of a vehicle-mounted system 100 that can be applied to the technical solution of the present disclosure. In some examples, the vehicle equipped with the system 100 can be an internal combustion engine vehicle using an engine as a power source, a hybrid vehicle using an engine and an electric motor as a power source, an electric vehicle using an electric motor as a power source, and other types of vehicles. In the subsequent content of this specification, the vehicle equipped with the vehicle-mounted system 100 is referred to as the present vehicle.

[0052] like Figure 1As shown, the vehicle-mounted system 100 includes: a navigation subsystem 110, an environment detection device group 120 for obtaining the vehicle's environment during the vehicle's driving, a vehicle driving state detection device group 130, a data processing unit 140, a display control unit 150, and a display unit 160. The above components or device groups are coupled together through a communication bus 12. In some examples, the communication bus 12 is used for connection and communication between the above components or device groups. It should be noted that Figure 1 Only a portion of the in-vehicle system 100 is shown, not all of the components of the in-vehicle system 100 .

[0053] exist Figure 1 In the navigation subsystem 110, a positioning device 111 and a map information storage device 112 are included. The positioning device 111 can locate the position of the vehicle based on the global positioning system (GPS), China's Beidou system, Russia's GLONASS system, Europe's Galileo system, Japan's Quasi-Zenith Satellite System (QZSS), India's Indian Regional Navigation Satellite System (IRNSS) and other positioning systems to obtain the position information of the vehicle. The map information storage device 112 stores map information, and can obtain a navigation path leading to the destination according to the position information obtained from the positioning device 111, and display the position information and the navigation path in the map application.

[0054] exist Figure 1 In the embodiment, the environment detection device group 120 may include an on-vehicle communication device 121, a radar 122, a laser rangefinder 123, and a camera 124. These devices can acquire environmental information representing the surrounding environment of the vehicle.

[0055] The vehicle-mounted communication device 121 can communicate wirelessly with one or more devices directly or via a communication network. These devices that can communicate with the vehicle-mounted communication device 121 can be other vehicles, roadside machines or roadside stations, or mobile terminal devices used by the occupants of the vehicle. In some examples, the vehicle-mounted communication device 121 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. In some examples, the vehicle-mounted communication device 121 can also communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the vehicle-mounted communication device 121 can also communicate directly with the device using an infrared link, Bluetooth or ZigBee. In some examples, the vehicle-mounted communication device 121 can also communicate with the device using other wireless protocols.

[0056] Radar 122 is used to sense objects in the surrounding environment of the vehicle, and can also be used to sense the speed and / or direction of these objects. In some examples, radar 122 can use electromagnetic waves or lasers as a medium to detect objects based on a time of flight (TOF) method or a phase-shift method, and detect the position of the detected object, the distance to the detected object, and the relative speed. In some examples, in order to be able to detect objects located in front of, behind, or to the side of the vehicle, radar 122 can be configured at an appropriate position outside the vehicle.

[0057] The laser rangefinder 123 may utilize laser light to sense objects in the environment in which the host vehicle is located. In some embodiments, the laser rangefinder 123 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0058] Camera 124 can be used to capture multiple images of the surrounding environment of the vehicle. Camera 122 can be a static camera or a video camera. In some examples, in order to obtain the external image of the vehicle, camera 122 can be located at an appropriate position outside the vehicle. For example, in order to obtain the image in front of the vehicle, camera 122 can be configured in the interior of the vehicle close to the front windshield. Alternatively, camera 122 can be configured around the front bumper or radiator grille. In some examples, in order to obtain the image behind the vehicle, camera 122 can be configured in the interior of the vehicle close to the rear window glass. Alternatively, camera 122 can be configured around the rear bumper, trunk or tailgate. In some examples, in order to obtain the image of the side of the vehicle, camera 122 can be configured in the interior of the vehicle close to at least one of the side windows. Alternatively, camera 122 can be configured around the side mirrors, fenders or doors.

[0059] exist Figure 1 In the embodiment, the vehicle driving state detection device group 130 may include: a steering angle sensor 131 for detecting the steering angle of the vehicle, a vehicle speed sensor 132 for detecting the driving speed of the vehicle, and an acceleration sensor 133 for detecting the acceleration applied to the vehicle. In some examples, as shown in the dotted box, an inertial sensor 134 for detecting the position and orientation change of the vehicle based on inertial acceleration may also be included. The inertial sensor 134 may be a combination of the acceleration sensor 133 and a gyroscope in a specific implementation process.

[0060] exist Figure 1 In the embodiment, the data processing unit 140 can be implemented as a computing system having a memory, a processor, an input / output interface, and a bus connecting these. In some examples, the data processing unit 140 causes the processor to execute multiple commands through program instructions stored in the memory to process the data obtained by the navigation subsystem 110, the environment detection device group 120, and the vehicle driving state detection device group 130. In some examples, the data processing unit 140 can also partially or completely control the driving of the vehicle based on the processed data.

[0061] exist Figure 1 As shown in the dotted box, the display control unit 150 and the display unit 160 can serve as the main body of the head-up display (HUD) device 170. The display control unit 150 can process the received data processed by the data processing unit 140, or after receiving the data obtained by the navigation subsystem 110, the environment detection device group 120 and the vehicle driving state detection device group 130, to obtain display information to be displayed, and project the display information onto the windshield of the vehicle through the display unit 160 for display.

[0062] Combination Figure 2An exemplary top view of the vehicle is shown and Figure 3 The exemplary perspective view from the driver's seat of the present vehicle is shown, and the present vehicle includes a windshield 204 located at the front of the vehicle. The driver and passengers in the passenger cabin 208 of the present vehicle can see the front of the present vehicle through the windshield 204.

[0063] exist Figure 3 In the embodiment of the present invention, the windshield 204 is visually located above the vehicle dashboard 206. The driver can turn the steering wheel 210 in the passenger cabin 208 to steer the vehicle, such as changing lanes, merging and parking the vehicle. In some embodiments, the steering wheel 210 can be retracted or omitted.

[0064] Head-up display device 170 (see Figure 4 ) projects display information 212 (eg, a virtual image) onto a portion of windshield 204 through one or more apertures (eg, aperture 216) in instrument panel 206. Figure 3 An example size of the display information 212 is shown, but the display information 212 may be presented over a larger or smaller area. The display information 212 may include various vehicle information, such as the current vehicle speed, the current gear of the vehicle transmission, the engine speed, the direction of the vehicle, the current infotainment system settings, and / or other vehicle information. The head-up display device 170 provides information to the vehicle driver without the driver having to look away from objects in front of the vehicle.

[0065] See also Figure 4 In the exemplary implementation architecture of the head-up display device 170 shown in FIG. 1 , the display control unit 150 generates a signal 412 based on the data processed by the data processing unit 140 or the data 420 transmitted by the navigation subsystem 110, the environment detection device group 120, and the vehicle driving state detection device group 130. The display unit 160 may include: a light source 161 and an optical path component 162. The light source 161 outputs light (e.g., a virtual image) based on the signal 412 from the display control unit 150 to display on the windshield 204. For example, the light source 161 may include one or more lasers and output red light, green light, and blue light.

[0066] The optical path component 162 can reflect the output of the light source 161 onto the windshield 204 through the hole 216. The viewer (e.g., the driver) can view the display information 212 in the display area where the display information 212 is projected onto the windshield 204. In some examples, the optical path component 162 may include one or more reflectors (plane mirrors) and concave mirrors (magnifying glasses). The output of the light source 161 is reflected back via the reflector and magnified by the concave mirror and then reflected to the windshield 204 to form a virtual image 40 that can be visually observed by the driver. The visual effect presented by the virtual image 40 is that the virtual image 40 is projected onto the projection surface 41 at a set distance in front of the vehicle, but the real environment remains visible through the projection surface 41. In some examples, the optical path component 162 can also be omitted, and the light source 161 can directly project the display information 212 onto the windshield 204 to form a virtual image 40 on the projection surface 41.

[0067] Combined with the above Figures 1 to 4 As shown, the display control unit 150 projects the navigation guidance onto the windshield 204 through the display unit 160 based on the vehicle's location information and navigation path provided by the navigation subsystem 110 before entering the turning intersection. The driver can more intuitively understand whether to turn left or right at the intersection ahead based on the navigation guidance. Specifically, when a left turn is about to be made, Figure 5a As shown, it is a navigation guide 51 displayed in the graphical user interface of the AR-HUD located on the windshield 204, and the arrow of the navigation guide 51 points to the left, indicating a left turn; Figure 5b As shown, the arrow of the navigation guide 51 points to the right, indicating a right turn.

[0068] As can be seen from Figure 5, at a turning intersection, the navigation guidance can clearly instruct the driver to turn left or right. The navigation guidance is prominently displayed on the windshield so that the driver can see the relevant information without lowering his head. While improving safety and convenience, it prompts the user to turn left or right to avoid missing the turning intersection.

[0069] Although the navigation guide 51 shown in FIG. 5 can prompt the driver to turn left or right at the turning intersection, the posture of the navigation guide does not change in real time during the turning process. Instead, after the turning is completed, the posture of the navigation guide in FIG. 5 is directly changed to Figure 6In the navigation guide 51 shown, even after the turn is completed, the navigation guide posture still instructs the driver to turn left or right. This is due to the change in posture of existing navigation guides. One is to rely on navigation signals, but at a turning intersection, continuous and accurate navigation signals are usually not provided; the other is to rely on vehicle computer signals, such as steering wheel angles, but changes in steering wheel angles cannot accurately reflect the turning process of the vehicle; therefore, at a turning intersection, the posture of existing navigation guides cannot be dynamically changed in real time, resulting in a poor driving experience for the driver.

[0070] It should be noted that the above Figures 5 and Figure 6 The navigation guide 51 is only an exemplary description and is not intended to be a limitation of the present disclosure. The navigation guide may also be represented in other forms, which are not specifically limited in the present disclosure.

[0071] Based on the above description, the present disclosure hopes to provide a solution in which the navigation guidance posture can change in real time with the movement of the vehicle at a turning intersection. Figure 7 As shown, an example of a display control method provided by the present disclosure is shown, which can be executed by the aforementioned head-up display device 170, and in particular, can be executed by the display control unit 150 in the aforementioned head-up display device 170. Figure 7 The method shown includes steps S701 to S703.

[0072] In step S701, a reference line of a target turning intersection is obtained, wherein the reference line is used to indicate the entrance of the target turning intersection.

[0073] Specifically, when it is determined that the distance between the vehicle and the target turning intersection is less than or equal to the preset distance, a continuous multi-frame environment image within the preset range of the vehicle is acquired at a preset frequency; when the entry stop line of the target frame environment image is at a preset position, the baseline is determined according to the entry stop line of the target frame environment image; wherein the multi-frame environment image includes the target frame environment image. In this way, the environment image is acquired only before entering the turning intersection, and the entry stop line is detected, which can reduce the data processing pressure of the vehicle computer.

[0074] When the navigation subsystem 110 is turned on, the distance between the current position of the vehicle and the target turning intersection to be entered can be determined in the navigation path. Specifically, when the navigation subsystem 110 determines that the distance between the current position of the vehicle and the turning intersection to be entered is less than or equal to a preset distance, the camera 124 is notified to collect the environment image within the preset range of the vehicle.

[0075] For each frame of the environmental image, detect whether each frame of the environmental image includes a stop line and whether the stop line is located at a preset position of the frame of the environmental image. If not, continue to detect the next frame of the environmental image; if so, record the frame of the environmental image as a target frame of the environmental image, and determine the baseline based on the stop line included in the target frame of the environmental image.

[0076] In step S702, a first angle change between the vehicle exiting the stop line and the reference line during the turning process is obtained.

[0077] The entry stop line is the stop line before the sidewalk before entering the turning intersection; if there is no stop line before the turning intersection, the edge of the sidewalk can also be used as the entry stop line. The exit stop line is the stop line after the next sidewalk after entering the turning intersection; if there is no stop line after the next sidewalk, the edge of the sidewalk can also be used as the exit stop line. Figure 8 As shown, taking the case where a vehicle is about to turn left as indicated by the dotted arrow as an example, the stop line before the sidewalk is the entry stop line 801, and the stop line after the next sidewalk is the exit stop line 803; if there is no entry stop line 801, the sidewalk edge line 803 shown by the dotted line is used as the entry stop line; if there is no exit stop line 803, the sidewalk edge line 804 shown by the dotted line is used as the exit stop line. The specific entry stop line and exit stop line can also be other marks, as long as they can mark the start and end of the turning intersection, and this disclosure does not limit them.

[0078] Specifically, the angle between the baseline and the exit stop line of the target frame environment image is determined as the reference angle; the angle between the exit stop line of the first environment image during the turning process and the baseline is determined as the target angle, and the change of the target angle relative to the reference angle is determined as the first angle change. The first environment image is the environment image after the target frame environment image.

[0079] After the target frame environment image, continue to acquire environment images, and for each frame environment image, determine the angle between the baseline and the exit stop line in the frame environment image. For each first frame environment image, keep the baseline unchanged, and as the vehicle enters the target turning intersection, the angle between the exit stop line in the first environment image and the baseline will become smaller and smaller.

[0080] For example, the preset position is the lower half of the image, that is, the entry stop line is located in the lower half of the image, which means that the required target frame environment image is detected. Fig. 9As shown, this is the acquired target frame environment image. The vehicle is about to turn left, and the entry stop line of the target turning intersection is the baseline 901 (if the entry stop line in the image is short and cannot form an intersection with the exit stop line, the entry stop line is extended to obtain the baseline), and the angle 903 between the exit stop line 902 of the target turning intersection and the baseline 901 is the reference angle.

[0081] Fig.10 The change of the target angle in the continuous multiple frames of environmental images compared with the reference angle is shown in an exemplary manner. Along the direction of the dashed arrow, the exit stop line (shown by a dashed line in the figure) in the continuous multiple frames of the first environmental image is shown. As the vehicle moves at the turning intersection, the target angle between the exit stop line and the baseline 901 becomes smaller and smaller. Therefore, the change of the angle between the exit stop line and the baseline can reflect the moving state of the vehicle at the target turning intersection.

[0082] In step S703, according to the first angle change, the second angle change of the navigation guidance is controlled.

[0083] In actual applications, at a turning intersection, the maximum change angle of navigation guidance is 90 degrees. Figure 5a or Figure 5b The navigation guide 51 in changes to Figure 6 In the navigation guide 51, the angle change is 90 degrees.

[0084] For example, Fig.11 As shown in the figure, it is an example of a left turn. As the vehicle enters the turning intersection and continues to move, it corresponds to Fig.10 The first angle in is getting smaller and smaller, Fig.11 The navigation guide 51 changes from the horizontal direction to the left along the direction of the dotted arrow until it is consistent with the forward direction of the vehicle ( Fig.11 The solid arrow direction is the driving direction of the vehicle), and the angle change process of the navigation guide 51 is shown by the dotted line in the figure. That is, the angle change of the navigation guide is synchronized with the vehicle's driving, and changes continuously as the vehicle enters the turning intersection. The angle change of the navigation guide intuitively reflects the driving state of the vehicle from entering the turning intersection to leaving the turning intersection. That is, at the turning intersection, the angle of the navigation guide changes in real time, which can more intuitively reflect the current driving state. It should be noted that Fig.11 Only four examples of angle changes of navigation guides are given. In actual applications, the preset frequency of acquiring external environment images is much higher than the frequency recognizable by the human eye. Therefore, the frequency of change of the displayed navigation guide is also higher than the frequency recognizable by the human eye. For example, if the angle of the navigation guide is changed at a frequency of 30 frames per second, the changes in the navigation guide seen by the driver are continuous animations.

[0085] According to the change of the first angle, the change of the second angle of the navigation guidance is controlled. Specifically, a first ratio of a preset change angle and a reference angle is determined; and the angle of the navigation guidance is controlled to be the product of the difference between the reference angle and the target angle and the first ratio.

[0086] In the present disclosure, the preset change angle is the maximum change angle of the navigation guidance at the turning intersection, the reference angle is the maximum angle between the baseline and the exit stop line in the collected environment image (the angle between the baseline and the exit stop line in the target frame environment image), and the display angle of the navigation guidance changes synchronously with the change of the angle in the collected environment image. For example: the reference angle is 60 degrees, the preset change angle is 90 degrees, the ratio of 90 to 60 is 1.5, the angle between the exit stop line and the baseline in the next frame environment image is 50 degrees, the difference between 60 and 50 is 10 degrees, and the product of 10 and 1.5 is 15. If it is a left turn, the navigation guidance is at Figure 5a The angle shown is offset 15 degrees clockwise. If it is a right turn, the navigation will be directed to Figure 5b The angle shown in the figure is offset 15 degrees counterclockwise; in the next frame of the environment image, the angle between the exit stop line and the baseline is 45 degrees, the difference between 60 and 45 is 15 degrees, and the product of 15 and 1.5 is 22.5. If it is a left turn, the navigation will be guided to Figure 5a The angle shown is offset 22.5 degrees clockwise. If it is a right turn, the navigation will be directed to Figure 5b The angle shown is offset counterclockwise by 22.5 degrees, and so on, until the target angle is 0 degrees in a certain frame of the environment image, then the difference is 60 degrees, and the degree of navigation guidance offset is 90 degrees, at which time the turning intersection ends. In this way, as the angle in the environment image changes, the angle of the navigation guidance also changes accordingly, and the navigation guidance can intuitively reflect the travel status of the vehicle at the turning intersection.

[0087] According to the principle of smaller distances and larger nearness, the closer to the exit stop line of the target turning intersection, the larger the displayed navigation guidance will be, thus further enhancing the driver's visual experience. Figure 7 ,like Fig.12 As shown, in some examples, after obtaining the first angle change between the exit stop line and the baseline during the turning process, the method further includes the following step S704.

[0088] In step S704, the display size of the navigation guide is controlled to change according to the first angle change.

[0089] For example, Fig.13 As shown in the figure, it is an example of a left turn. As the vehicle enters the turning intersection and continues to move, it corresponds to Fig.10 The first angle in is getting smaller and smaller, Fig.13The middle navigation guide 51 changes from horizontal to leftward along the direction of the dotted arrow and the display size becomes larger and larger until it is consistent with the forward direction of the vehicle and the displayed size is the largest. The change process of the navigation guide 51 is shown by dotted lines in the figure. Fig.13 The display size of the navigation guide shown in the figure increases as the vehicle approaches the exit stop line. In this way, after entering the turning intersection, the angle change of the navigation guide is consistent with the actual vehicle's traveling state at the turning intersection, and according to the principle of small at a distance and large at a near distance, the display size of the navigation guide is changed to achieve an augmented reality effect, further enhancing the driver's experience.

[0090] According to the change of the first angle, the change of the display size of the navigation guide is controlled. Specifically, a second ratio of a preset change multiple and a reference angle is determined; and the display size is controlled to expand the product of the difference between the reference angle and the target angle and the second ratio.

[0091] In order to make the driver feel that the exit of the turning intersection is getting closer and closer according to the navigation guidance displayed in the windshield 204, according to the principle of small distance and large near, the display size of the navigation guidance can be controlled after the vehicle enters the turning intersection, so that the driver feels more realistic. For example, the preset change multiple is 2 times, the reference angle is 60 degrees, the ratio is 1 / 30, the target angle in the next frame of the environment image is 50 degrees, the difference between 60 and 50 is 10 degrees, and the product of 10 and 1 / 30 is 1 / 3, then the navigation guidance is enlarged by 1 / 3 of the original navigation guidance, and so on, until in a certain frame of the environment image, the target angle is 0 degrees, then the difference is 60 degrees, and the navigation guidance is enlarged by 2 times of the original navigation guidance, and the turning intersection ends at this time. In this way, when the driver is at the turning intersection, the closer to the exit of the turning intersection, the size of the navigation guidance displayed by the AR-HUD is continuously enlarged, so that the user experience is better.

[0092] In order to keep the baseline in each frame unchanged, the resolution of the environmental image collected by the vehicle in the above steps S701 to S703 remains unchanged. A coordinate can be determined based on a pixel point in each frame image, thereby determining the two pixel points through which the baseline passes in the target frame environmental image. In any frame environmental image after the target frame environmental image, the baseline is determined based on the two pixel points. However, due to the large number of pixels and high precision, the stop line in the actual collected environmental image is wide, and the coordinates cannot be accurately determined based on the pixels. Fig.14 , is the enlarged baseline 901, each white dot represents a pixel, the lower left corner of the environment image is determined as the origin, and the left and downward direction is the positive direction. Since the baseline 901 has multiple values ​​of the vertical coordinate when the horizontal coordinate remains unchanged, a strategy needs to be adopted to determine the baseline, and two pixels are selected from multiple pixels to determine the baseline according to the two pixels in the next frame. This process consumes more computing power.

[0093] To simplify the determination of the baseline, refer to Fig.15 The above-mentioned step of obtaining a continuous multi-frame environment image within a preset range of the vehicle according to a preset frequency includes the following steps S1501 and S1502.

[0094] In step S1501, multiple frames of initial environment images are collected continuously within a preset range of the vehicle at a preset frequency.

[0095] In step S1502, each frame of the initial environment image is rasterized to obtain multiple frames of environment images.

[0096] The size of each grid is obtained by reducing the resolution of each frame of the initial environment image by a preset ratio.

[0097] The resolution of the environment image collected by the camera 124 in the vehicle is usually constant, so the size of each grid can be determined according to the resolution of the image. For example, if the resolution of the image is 1152*576 and the preset ratio is 1 / 100, the size of each grid is 11.52*5.76. Fig.16 , for Fig. 9 The image is obtained by rasterizing the image, the size of each grid is much larger than the size of a pixel, the reference line 901 is located in the grid, and each grid is used as a coordinate point, so it is easy to determine the reference line according to the coordinate point.

[0098] Specifically, refer to Fig.17 In the case where the stop line of the target frame environment image is located at a preset position, determining the baseline according to the stop line of the target frame environment image includes the following steps S1701 and S1702.

[0099] In step S1701, when the entry stop line of the target frame environment image is located at a preset position, the reference coordinates are determined according to the grid where the preset point of the entry stop line is located.

[0100] In step S1702, a line passing through the reference coordinates and parallel to the horizontal axis is determined as a reference line.

[0101] Specifically, Fig.18 As shown in FIG. 1 , the target frame environment image after acquisition and rasterization processing. Taking the lower left corner as the origin as an example, the base point coordinate 1801 is the midpoint of the entry stop line corresponding to the lane where the vehicle is located (the preset point of the entry stop line), and the line segment passing through the base point coordinate and parallel to the horizontal axis is determined as the reference line 1802. The reference line 1802 may completely coincide with the entry stop line, or there may be a small error, but the influence of the error on the angle can be ignored.

[0102] Correspondingly, for the rasterized environment image, see Fig.19 The above-mentioned step of determining the angle between the exit stop line and the baseline of the first environment image in the turning process as the target angle includes the following steps S1901 to S1903.

[0103] In step S1901, the coordinates of the stop point are determined according to the grid where the preset point of the exit stop line of the first environment image is located.

[0104] In the present disclosure, different preset points can be set for different lane numbers at the turning exit, but the selected preset point remains unchanged during a turning process. For example, for a single lane, when turning left, the intersection of the exit stop line and the outermost lane can be selected as the preset point, and for a two-lane lane, the intersection of the middle lane line and the exit stop line and the outermost lane can be selected as the preset point, or the intersection of the exit stop line and the outermost lane can be selected as the preset point, which is not specifically limited in the present disclosure. Fig.18 In the example of left turn, the intersection of the exit stop line and the outermost lane line is determined as the stop point coordinate 1803.

[0105] In step S1902, the coordinates of the intersection point are determined according to the grid where the intersection point of the exit stop line and the reference line of the first environment image is located.

[0106] If the exit stop line and the baseline do not intersect, extend the exit stop line and the baseline until the two lines intersect. Fig.18 The intersection point coordinates 1804 are shown in FIG.

[0107] In step S1903, the target angle is determined according to the reference coordinates, the intersection point coordinates and the stop point coordinates.

[0108] In the present disclosure, the target angle is determined according to the reference coordinates, the intersection coordinates and the stop point coordinates. Specifically, the vector dot product formula is used to derive the vector angle. For vector a and vector b, the vector dot product formula is: a*b=|a||b|cosθ, so the angle θ=acos(a*b) / (|a|*|b|). For example, the intersection coordinates, reference coordinates and stop point coordinates are: A(1,1), B(2,1), C(2,2), AC=(Cx-Ax, Cy-Ay)=(1,1); AB=(Bx-Ax, By-Ay)=(1,0), That is, the angle A is 45 degrees. In two dimensions, the amount of calculation required to calculate the angle through vectors is relatively small, and only one arc cosine operation is required, which can save the amount of calculation and reduce the requirements for the computing power of the vehicle computer.

[0109] In the present disclosure, in addition to indicating the distance to the exit of the turning intersection by the angle change of the navigation guidance, the distance to the exit can also be intuitively displayed on the windshield 204, specifically, as Fig. 20 As shown, it is achieved through the following steps 2001 to 2003.

[0110] In step 2001, the distance between the reference coordinates and the intersection coordinates is determined as the target radius.

[0111] In step 2002, in a target circle, a target arc length corresponding to a target included angle at the center angle is determined.

[0112] Among them, the center of the target circle is the intersection point coordinate, and the radius of the target circle is the target radius.

[0113] In step 2003 , the target arc length is displayed in a graphical user interface of an augmented reality head-up display system of the vehicle.

[0114] Since the distance of a turning intersection is usually not too long, in this disclosure, the driving trajectory of the turning intersection is approximately processed as an arc. Fig.21 As shown, the center of the target circle 210 is the intersection coordinate 1804, and the distance between the reference coordinate 1801 and the intersection coordinate 1804 is the target radius 2101. As the vehicle enters the turning intersection, the angle 803, that is, the central angle, becomes smaller and smaller, and the arc length corresponding to the central angle also becomes smaller and smaller. The arc length corresponds to the distance between the vehicle and the exit of the turning intersection.

[0115] Specifically, the calculation formula for the target arc length corresponding to the target angle is: (πr / 180)*θ, where r is the target radius and θ is the target angle. Fig. 22 As shown, the "50m" displayed in area 5 of the windshield 204 is the distance from the exit of the turning intersection. In this way, the driver can more intuitively understand the current distance from the exit of the turning intersection, which better assists the driver in driving.

[0116] In the present disclosure, in order to further reduce the data processing pressure of the vehicle computer, the method also includes the following step 2301.

[0117] In step 2301, when the vehicle completes turning, the vehicle is controlled to stop acquiring the environment image within a preset range.

[0118] Whether the vehicle has completed the turn can be determined based on the determined current angle, or based on the determined current target arc length. Specifically, when the target angle is less than or equal to the preset angle, it is determined that the vehicle has completed the turn; when the target angle is greater than the preset angle, it is determined that the vehicle has not completed the turn. Alternatively, when the target arc length is less than or equal to the preset arc length, it is determined that the vehicle has completed the turn; when the target arc length is greater than the preset arc length, it is determined that the vehicle has not completed the turn. After determining that the turn is completed, continue according to the original processing strategy for navigation guidance in AR-HUD. In this way, after the turn is completed, there is no need to continue to acquire environmental images and continue subsequent processing, which reduces the processing pressure of the vehicle computer.

[0119] Based on the same inventive concept as the above technical solution, see Fig.23 , which shows a display control device 230 provided by the present disclosure, the device 230 may be Figure 1 or Figure 4 The display control unit shown in the figure, the device 230 includes: an acquisition part 2301 and a control part 2302; wherein,

[0120] The acquisition part 2301 is configured to acquire a baseline of a target turning intersection, where the baseline is used to indicate an entrance of the target turning intersection;

[0121] The acquisition part 2301 is further configured to acquire a first angle change between the vehicle and the reference line during the turning process of exiting the stop line;

[0122] The control part 2302 is configured to control the second angle change of the navigation guidance according to the first angle change.

[0123] In some examples, the acquisition part 2301 is specifically configured to acquire multiple continuous frames of environmental images within a preset range of the vehicle at a preset frequency when it is determined that the distance between the vehicle and the target turning intersection is less than or equal to a preset distance; when the stop line of the target frame environmental image is at a preset position, determine the baseline according to the stop line of the target frame environmental image, and the multiple frames of environmental images include the target frame environmental image.

[0124] In some examples, the acquisition portion 2301 is specifically configured to determine the angle between the baseline and the exit stop line of the target frame environmental image as a reference angle; determine the angle between the exit stop line and the baseline of the first environmental image during a turning process as a target angle, the first environmental image being an environmental image after the target frame environmental image; and determine the change in the target angle relative to the reference angle as a first angle change.

[0125] In some examples, the acquisition part 2301 is specifically configured to collect multiple continuous frames of initial environmental images within a preset range of the vehicle at a preset frequency; each frame of the initial environmental image is rasterized to obtain multiple frames of environmental images, and the size of each grid is obtained by reducing the resolution of each frame of the initial environmental image by a preset ratio.

[0126] In some examples, the acquisition part 2301 is specifically configured to determine the reference coordinates according to the grid where the preset point of the stop line is located when the stop line of the target frame environment image is located at a preset position; and determine the line passing through the reference coordinates and parallel to the horizontal axis as the reference line.

[0127] In some examples, the acquisition part 2301 is specifically configured to determine the stop point coordinates based on the grid where the preset point of the exit stop line of the first environmental image is located; determine the intersection point coordinates based on the grid where the intersection of the exit stop line of the first environmental image and the baseline is located; determine the target angle based on the baseline coordinates, the intersection point coordinates and the stop point coordinates.

[0128] In some examples, the device also includes a display part and a determination part, the determination part is configured to determine the distance between the reference coordinates and the intersection coordinates as the target radius; in the target circle, determine the target arc length corresponding to the center angle of the circle as the target angle, the center of the target circle is the intersection coordinates, and the radius of the target circle is the target radius; the display part is configured to display the target arc length in the graphical user interface of the augmented reality-head-up display system of the vehicle.

[0129] In some examples, the determination part is also configured to, after determining the target angle based on the reference coordinates, the intersection point coordinates and the stop point coordinates, determine that the vehicle's turn is completed if the target angle is less than or equal to a preset angle; and determine that the vehicle's turn is not completed if the target angle is greater than the preset angle.

[0130] In some examples, the determination part is also configured to, after determining the target arc length corresponding to the target angle at the center angle of the target circle, determine that the vehicle has completed turning if the target arc length is less than or equal to the preset arc length; and determine that the vehicle has not completed turning if the target arc length is greater than the preset arc length.

[0131] In some examples, the control portion 2302 is further configured to control the change in the display size of the navigation guidance according to the first angle change after obtaining the first angle change between the exit stop line and the baseline during the turning process.

[0132] In the embodiment of the present application, each module can implement the display control method provided by the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0133] refer to Fig.24 , which shows a structural block diagram of a display control device 230 provided by an exemplary embodiment of the present disclosure. In some examples, the display control device 230 has a communication function and can access a wired network or a wireless network. In some examples, the display control device 230 can receive data based on the wired network or wireless network accessed. It can be understood that the display control device 230 undertakes the calculation and processing work of the technical solution of the present disclosure, and the present disclosure does not limit this.

[0134] like Fig.24 As shown, the display control device 230 in the present disclosure may include one or more of the following components: a processor 2410 and a memory 2420 .

[0135] Optionally, the processor 2410 uses various interfaces and lines to connect various parts of the entire computing device, and executes various functions of the computing device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 2420, and calling data stored in the memory 2420. Optionally, the processor 2410 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 2410 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above-mentioned baseband chip may not be integrated into the processor 2410, but may be implemented by a separate chip.

[0136] The memory 2420 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 2420 includes a non-transitory computer-readable storage medium. The memory 2420 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 2420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area may store data created according to the use of the computing device, etc.

[0137] In addition, those skilled in the art will appreciate that the structure of the computing device shown in the above figures does not constitute a limitation on the computing device, and the computing device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the computing device also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module, and other components, which will not be described in detail here.

[0138] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the display control method described in the above embodiments.

[0139] The present disclosure also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes to implement the display control method described in the above-mentioned embodiments.

[0140] The present disclosure also provides a head-up display device, which includes a display control unit and a display unit; the display control unit is configured to obtain a baseline of a target turning intersection, the baseline is used to indicate an entrance to the target turning intersection; and obtain a first angle change of the exit stop line with the baseline during the turning process; and is configured to control a second angle change of the navigation guidance according to the first angle change; the display unit is configured to project the navigation guidance onto the windshield of the vehicle for display based on the control of the display control unit.

[0141] The present disclosure also provides a vehicle, including the above-mentioned head-up display device, and the vehicle can implement each process of the display control method provided by the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0142] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0143] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0144] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A display control method, characterized in that: The method comprises: Acquire a baseline of a target turning intersection, wherein the baseline is used to indicate an entrance of the target turning intersection; Acquire a first angle change between the reference line and the exit stop line during the turning process; According to the first angle change, the second angle change of the navigation guidance is controlled.

2. The method according to claim 1, characterized in that: The step of obtaining the reference line of the target turning intersection includes: When it is determined that the distance between the vehicle and the target turning intersection is less than or equal to a preset distance, acquiring multiple frames of continuous environment images within a preset range of the vehicle at a preset frequency; When the entry stop line of the target frame environment image is located at a preset position, a baseline is determined according to the entry stop line of the target frame environment image, and the multiple frames of environment images include the target frame environment image.

3. The method according to claim 2, characterized in that The obtaining of a first angle change between the exit stop line and the reference line during the turning process includes: Determine the angle between the baseline and the exit stop line of the target frame environment image as a reference angle; Determine the angle between the exit stop line and the baseline of the first environment image in the turning process as the target angle, wherein the first environment image is an environment image after the target frame environment image; A change in the target angle relative to the reference angle is determined as the first angle change.

4. The method according to claim 3, characterized in that The step of obtaining a plurality of continuous frames of environment images within a preset range of the vehicle at a preset frequency includes: Collect multiple frames of initial environment images within a preset range of the vehicle at a preset frequency; Each frame of the initial environment image is subjected to rasterization processing to obtain the multiple frames of environment images, and the size of each grid is obtained by reducing the resolution of each frame of the initial environment image by a preset ratio.

5. The method according to claim 4, characterized in that The step of determining the reference line according to the stop line of the target frame environment image when the stop line of the target frame environment image is located at a preset position includes: When the entry stop line of the target frame environment image is located at a preset position, determining the reference coordinates according to the grid where the preset point of the entry stop line is located; A line passing through the reference coordinates and parallel to the horizontal axis is determined as the reference line.

6. The method according to claim 5, characterized in that The step of determining the angle between the exit stop line of the first environment image in the turning process and the reference line as the target angle includes: Determine the coordinates of the stop point according to the grid where the preset point of the exit stop line of the first environment image is located; Determine the coordinates of the intersection point according to the grid where the intersection point of the exit stop line of the first environment image and the baseline is located; The target angle is determined according to the reference coordinates, the intersection point coordinates and the stop point coordinates.

7. The method according to claim 6, characterized in that The method further comprises: Determine the distance between the reference coordinates and the intersection coordinates as a target radius; In a target circle, determine a target arc length whose central angle is the target included angle, the center of the target circle is the intersection coordinate, and the radius of the target circle is the target radius; The target arc length is displayed in a graphical user interface of an augmented reality head-up display system of the host vehicle.

8. The method according to claim 6, characterized in that After determining the target angle according to the reference coordinates, the intersection point coordinates and the stop point coordinates, the method further includes: When the target angle is less than or equal to a preset angle, determining that the vehicle has completed turning; When the target angle is greater than the preset angle, it is determined that the vehicle has not completed turning.

9. The method according to claim 7, characterized in that: After determining, in the target circle, that the center angle of the circle is the target arc length corresponding to the target angle, the method further includes: When the target arc length is less than or equal to the preset arc length, determining that the vehicle has completed turning; When the target arc length is greater than the preset arc length, it is determined that the vehicle has not completed turning.

10. The method according to any one of claims 1 to 9, characterized in that: After obtaining the first angle change between the exit stop line and the reference line during the turning process, the method further includes: According to the first angle change, the display size of the navigation guide is controlled to change.

11. A display control device, characterized in that: The device comprises: an acquisition part and a control part; The acquisition part is configured to acquire a reference line of a target turning intersection, wherein the reference line is used to indicate an entrance of the target turning intersection; The acquisition part is further configured to acquire a first angle change with respect to the reference line during a turning process when exiting the stop line; The control part is configured to control the second angle change of the navigation guidance according to the first angle change.

12. A display control device, characterized in that: The device comprises: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the display control method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the display control method according to any one of claims 1 to 10.

14. A head-up display device, characterized in that: The head-up display device includes a display control unit and a display unit; wherein, The display control unit is configured to obtain a reference line of a target turning intersection, wherein the reference line is used to indicate an entrance of the target turning intersection; and obtaining a first angle change between the exit stop line and the reference line during the turning process; and, configured to control a change in a second angle of navigation guidance according to the change in the first angle; The display unit is configured to project the navigation guidance onto a windshield of the vehicle for display based on the control of the display control unit.