Intelligent HUD display method and system and automobile

Abstract

The invention relates to the field of electronic vehicle-mounted technologies, in particular to an intelligent HUD display method and system and an automobile. The system comprises a voice recognition module and a driver fatigue detection module. The method comprises the steps of obtaining current state information of a driver; judging whether the driver is fatigue according to the current state information of the driver; if yes, continuously receiving driver language information, and determining a driver demand according to the language information; and determining an alarm and coping strategy according to the driver demand. According to the invention, a driver can control HUD display content through a voice instruction. By means of the mode, manual control of a driver can be omitted, and the driver can drive the automobile more safely. Meanwhile, the fatigue degree of the driver is monitored, and when it is detected that the driver is in a fatigue state, alarm information is displayed through the HUD to remind the driver, so that driving safety is guaranteed. The invention aims to improve the driving safety and reduce the risk of traffic accidents.

Description

Technical Field

[0001] The present invention relates to the field of electronic vehicle-mounted technology, and in particular to an intelligent HUD display method, system and automobile. Background Art

[0002] HUD technology (Head-up display) is a technology that projects key information into the driver's field of vision. It displays important driving data, navigation information, safety warnings and other information directly in the driver's field of vision, thus avoiding the driver's need to take their eyes off the road or navigation. This technology can not only improve the driver's attention and reaction ability, reduce the driver's visual fatigue, but also greatly improve the driver's driving safety.

[0003] Traditional HUD technology is mainly realized through projection optical systems, but with the development of computer vision technology, more and more intelligent HUD technologies are realized based on computer vision technology. These intelligent HUD technologies use on-board cameras to capture road conditions and then display relevant information in real time, such as vehicle speed, lane deviation, traffic signs, etc., so that drivers do not need to take their eyes away from the front field of view and can more effectively perceive road conditions and road traffic conditions.

[0004] In the prior art, the intelligent HUD system uses laser projection technology to directly project driving data onto the driver's field of vision, such as vehicle speed, fuel consumption, rear vehicle warnings, etc. Such intelligent HUD technologies have the following main disadvantages: 1. Information overload problem: Although HUD technology can display important information in the driver's field of vision, as the amount of information increases, the driver may not be able to process the information effectively.

[0005] 2. Display quality issues: Traditional projection HUD technology may be affected by external light, blind spots and other factors, resulting in poor display effects.

[0006] 3. Data accuracy issues: The accuracy of HUD technology is affected by many factors such as the quality of the sensor and the driving environment, and errors may occur. Summary of the invention

[0007] The present invention aims to at least improve one of the technical problems existing in the prior art. To this end, the present invention proposes an intelligent HUD display method, system and automobile.

[0008] The intelligent HUD display method according to the first aspect of the present invention is applied to an intelligent HUD display control system, wherein the system includes a speech recognition module and a driver fatigue detection module, and the method includes: Acquiring current state information of the driver, wherein the state information includes the driver's action and physiological state; Determine whether the driver is fatigued based on the driver's current status information; If yes, continue to receive the driver's language information and determine the driver's needs based on the language information; If it is a rest demand, the system triggers a Class I alarm and executes a first strategy; If the alarm is turned off, the system turns off the alarm without any reminder; If the language information still cannot determine the driver's needs within the first threshold time, the system triggers a Class II alarm and executes a second strategy.

[0009] In a possible implementation manner of the first aspect, the continuously receiving the driver's language information and determining the driver's demand according to the language information specifically includes: Collect the driver's voice information; Preprocessing the collected voice information, including denoising, filtering, and enhancement, to improve the accuracy of voice recognition; Convert the pre-processed speech into text using speech recognition technology; Parse and understand the identified text and extract keywords; Determine driver needs based on keywords to improve system accuracy and user experience.

[0010] In a possible implementation manner of the first aspect, the first strategy includes: HUD displays reminder information; The system lowers the volume and changes the music. Based on this reminder strategy, the driver can continue driving but should take a break for a while.

[0011] In a possible implementation manner of the first aspect, the second strategy includes: The HUD displays reminder information, where the reminder text is red, which can effectively display the information that the driver needs to pay attention to; Voice reminders to the driver to rest or park can reduce the frequency with which the driver needs to look at the HUD.

[0012] In a possible implementation manner of the first aspect, the method further includes: After executing the second strategy, the driver turns off the reminder strategy by voice inputting a keyword, wherein the keyword includes: turning off the alarm or canceling the alarm.

[0013] In a possible implementation manner of the first aspect, the speech recognition technology includes: The SLAM algorithm is used to determine the position and direction of the HUD. The acquired voice commands are aligned with the estimated operation surface of the HUD to achieve precise positioning. On this basis, the HUD system can display navigation information, vehicle-road fusion information, etc. at an appropriate position in front of the driver, which does not affect the driver's line of sight and provides clear and intuitive driving assistance information. Cognitive computing algorithms are used to analyze voice commands, including speech recognition and semantic understanding, to analyze the meaning of voice commands and user intent, and to generate corresponding commands and responses based on different contexts and needs.

[0014] In a possible implementation manner of the first aspect, the physiological state includes the driver's head tilt angle, eye movements, and blinking frequency.

[0015] In a possible implementation manner of the first aspect, the first threshold time ranges from 0 to 10 seconds.

[0016] According to the intelligent HUD display method of the embodiment of the present invention, the voice control function and fatigue recognition technology are combined through the HUD intelligent control system, which can help the driver drive the vehicle more safely. The advantages of the present invention are as follows: 1. Improve driving safety: Input control through voice commands eliminates the need for manual operation by the driver, thus reducing the difficulty of operation and improving driving safety. Driver fatigue triggers the HUD display alarm to remind the driver to take a rest in time to avoid fatigue driving, thus effectively preventing traffic accidents.

[0017] 2. Convenient operation: The system’s built-in voice control function allows the driver to adjust the HUD settings more conveniently, avoiding distractions due to inconvenient operation and causing traffic accidents.

[0018] 3. Reduce driver fatigue: Driver fatigue triggers the HUD display alarm, which uses fatigue recognition technology to analyze the driver's eye movement, head tilt angle, blinking frequency and other data to analyze the driver's fatigue state. It reminds the driver to take a rest in time, so that the driver can get enough rest and reduce the possibility of fatigue driving.

[0019] 4. Reduce traffic violations: Voice-controlled HUD helps reduce traffic violations caused by drivers operating on-board equipment with their hands. HUD can not only remind drivers to pay attention to driving rules such as speed, but also cooperate with the navigation system to remind drivers whether there are road signs such as violation monitoring ahead.

[0020] In summary, this invention allows the driver to control the HUD display content through voice commands. This method can eliminate the driver's manual control and allow the driver to drive the car more safely. At the same time, the present invention monitors the driver's fatigue level, and when it detects that the driver is in a fatigued state, the HUD displays an alarm message to remind the driver to ensure driving safety. The present invention aims to improve driving safety and reduce the risk of traffic accidents.

[0021] According to an embodiment of the second aspect of the present invention, an intelligent HUD display control system comprises: Microphone, used to collect driver's voice information; A speech recognition module, used for obtaining the driver's needs according to the speech information; Camera, used to obtain the driver's current status information; Driver fatigue detection module, used to detect whether the driver is fatigued; HUD control module, used to execute corresponding reminder strategies according to driver voice information and fatigue information; HUD, used to project and display reminder strategy information.

[0022] According to an embodiment of the third aspect of the present invention, a car is equipped with an intelligent HUD display control system for executing the intelligent HUD display method as described above.

[0023] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a flow chart of a smart HUD display method according to an embodiment of the present invention; Figure 2 4 is a schematic diagram of an intelligent HUD display control system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be an intermediate element at the same time.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0029] The terms "first", "second", "third", etc. in the specification and claims of the present application and the drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a series of steps or units are included, or optionally, steps or units not listed are included, or optionally, other steps or units inherent to these processes, methods, products or devices are included.

[0030] Only the part relevant to the present application is shown in the accompanying drawings, but not all of the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processing or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. When its operation is completed, the process can be terminated, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0031] The terms "component", "module", "system", "unit", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate through local and / or remote processes, for example, based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network. For example, the Internet interacts with other systems via signals).

[0032] Example 1 See also Figure 1 and Figure 2 As shown, this embodiment provides an intelligent HUD display method, which is applied to an intelligent HUD display control system, wherein the system includes a speech recognition module and a driver fatigue detection module, and the method includes: Step S100, obtaining the current state information of the driver, wherein the state information includes the driver's action and physiological state; Step S200, judging whether the driver is fatigued according to the driver's current state information; Step S300, if yes, continue to receive driver language information and determine the driver's needs based on the language information; Step S400: If the user needs to rest, the system triggers a Class I alarm and executes a first strategy, which includes: HUD displays reminder information; The system lowers the volume and changes the music. Based on this reminder strategy, the driver can continue driving, but should take a break for a while; Step S500, if the alarm is turned off, the system turns off the alarm without any reminder; Step S600: If the language information still cannot determine the driver's needs within the first threshold time, the system triggers a Class II alarm and executes a second strategy, which includes: Step S700, the HUD displays a reminder message, wherein the reminder message text is red, which can effectively display the information that the driver needs to pay attention to; Voice reminders to the driver to rest or park can reduce the frequency with which the driver needs to look at the HUD.

[0033] It should be noted that the continuously receiving the driver's language information and determining the driver's needs according to the language information specifically includes: Collect the driver's voice information; Preprocessing the collected voice information, including denoising, filtering, and enhancement, to improve the accuracy of voice recognition; Convert the pre-processed speech into text using speech recognition technology; Parse and understand the identified text and extract keywords; Determine driver needs based on keywords to improve system accuracy and user experience.

[0034] It should be noted that a smart HUD display method also includes: After executing the second strategy, the driver turns off the reminder strategy by voice inputting a keyword, wherein the keyword includes: turning off the alarm or canceling the alarm.

[0035] In this embodiment, the keywords may be "tired", "rest", "turn off alarm", etc., which are used to control the execution or cancellation of alarms and their corresponding strategies.

[0036] It should be noted that the speech recognition technology includes: The SLAM algorithm is used to determine the position and direction of the HUD. The acquired voice commands are aligned with the estimated operation surface of the HUD to achieve precise positioning. On this basis, the HUD system can display navigation information, vehicle-road fusion information, etc. at an appropriate position in front of the driver, which does not affect the driver's line of sight and provides clear and intuitive driving assistance information. Cognitive computing algorithms are used to analyze voice commands, including speech recognition and semantic understanding, to analyze the meaning of voice commands and user intent, and to generate corresponding commands and responses based on different contexts and needs.

[0037] It should be noted that the physiological state includes the driver's head tilt angle, eye movements, and blinking frequency.

[0038] It should be noted that the first threshold time ranges from 10 seconds.

[0039] It should be noted that in addition to voice control, the vehicle can also alert the driver by issuing sound prompts. For example, when the vehicle speed exceeds a certain limit, the vehicle can issue a sound alarm to alert the driver. This solution does not require the installation of a HUD device and has a low implementation cost.

[0040] It should be noted that in addition to the HUD, the in-vehicle display can also display information to provide the driver with vehicle status and reminders. This solution is suitable for those situations where fast and real-time information updates are not required, such as highway driving, and the driver can divert his attention to other aspects.

[0041] According to the intelligent HUD display method of the embodiment of the present invention, the voice control function and fatigue recognition technology are combined through the HUD intelligent control system, which can help the driver drive the vehicle more safely. The advantages of the present invention are as follows: 1. Improve driving safety: Input control through voice commands eliminates the need for manual operation by the driver, thus reducing the difficulty of operation and improving driving safety. Driver fatigue triggers the HUD display alarm to remind the driver to take a rest in time to avoid fatigue driving, thus effectively preventing traffic accidents.

[0042] 2. Convenient operation: The system’s built-in voice control function allows the driver to adjust the HUD settings more conveniently, avoiding distractions due to inconvenient operation and causing traffic accidents.

[0043] 3. Reduce driver fatigue: Driver fatigue triggers the HUD display alarm, which uses fatigue recognition technology to analyze the driver's eye movement, head tilt angle, blinking frequency and other data to analyze the driver's fatigue state. It reminds the driver to take a rest in time, so that the driver can get enough rest and reduce the possibility of fatigue driving.

[0044] 4. Reduce traffic violations: Voice-controlled HUD helps reduce traffic violations caused by drivers operating on-board equipment with their hands. HUD can not only remind drivers to pay attention to driving rules such as speed, but also cooperate with the navigation system to remind drivers whether there are road signs such as violation monitoring ahead.

[0045] In summary, this invention allows the driver to control the HUD display content through voice commands. This method can eliminate the driver's manual control and allow the driver to drive the car more safely. At the same time, the present invention monitors the driver's fatigue level, and when it detects that the driver is in a fatigued state, the HUD displays an alarm message to remind the driver to ensure driving safety. The present invention aims to improve driving safety and reduce the risk of traffic accidents.

[0046] Example 2 See also Figure 2 As shown, this embodiment provides an intelligent HUD display control system, wherein the system includes: Microphone, used to collect driver's voice information; A voice recognition module, used to obtain the driver's needs according to the voice information; it should be noted that the voice recognition module supports Chinese and English voice commands; Camera, used to obtain the driver's current status information; Driver fatigue detection module, used to detect whether the driver is fatigued; HUD control module, used to execute corresponding reminder strategies according to driver voice information and fatigue information; HUD, used to project and display reminder strategy information.

[0047] The speech recognition module drives the speech command through speech, and the HUD control module is responsible for managing the HUD and controlling the preprocessing of the speech command and controlling the display of the HUD. The driver fatigue monitoring module monitors the driver's status and sends CPU instructions to the control module when abnormal, requesting to adjust the HUD display in a preset way. At the same time, the speech recognition module can add voice queries through the control module to assist the driver in completing more reading operations.

[0048] It should be noted that the speech recognition module also includes the use of technologies similar to WebRTC to transmit audio streams in real time, ensuring low latency, low packet loss rate and high quality.

[0049] It should be noted that the speech recognition module also has a speech recognition API: using mature speech recognition API interfaces, such as Baidu AI, Alibaba Cloud Intelligence, etc., can reduce the difficulty of algorithm development and improve recognition accuracy and response speed.

[0050] It should be noted that the voice recognition module also includes the use of speech synthesis technology to convert the HUD's response to the command into audio for broadcasting, so as to provide feedback to the user.

[0051] Example 3 This embodiment provides an intelligent HUD display method based on Embodiment 1, which is applied to an intelligent HUD display control system, wherein the system includes a touch screen recognition module and a driver fatigue detection module, and the method includes: Acquiring current state information of the driver, wherein the state information includes the driver's action and physiological state; Determine whether the driver is fatigued based on the driver's current status information; If so, receiving the driver's touch screen input information, and determining the driver's needs based on the touch screen input; If the demand is for rest, the system triggers a Class I alarm and executes a first strategy, which includes: HUD displays reminder information; The system lowers the volume and changes the music. Based on this reminder strategy, the driver can continue driving, but should take a break for a while; If the alarm is turned off, the system turns off the alarm without any reminder; If no touch screen input information is received within the first threshold time, the driver's demand cannot be determined, and the system triggers a Class II alarm and executes a second strategy, which includes: The HUD displays reminder information, where the reminder text is red, which can effectively display the information that the driver needs to pay attention to; Voice reminders to the driver to rest or park can reduce the frequency with which the driver needs to look at the HUD.

[0052] It should be noted that the first threshold time is 5s. It is worth mentioning that the first threshold time can be directly customized by the user to improve the user experience.

[0053] The difference between this embodiment and embodiment 1 is that the touch screen operation can be operated by fingers, thereby realizing the display and control of vehicle information. Unlike voice control, the touch screen can input data more accurately, and the touch screen is also easier to use than voice control.

[0054] Example 4 This embodiment provides an intelligent HUD display method based on Embodiment 1, which is applied to a handheld intelligent HUD display control system, which is convenient for the driver to carry with him. The system includes a speech recognition module and a driver fatigue detection module, and the method includes: Acquiring current state information of the driver, wherein the state information includes the driver's action and physiological state; Determine whether the driver is fatigued based on the driver's current status information; If yes, continue to receive the driver's language information and determine the driver's needs based on the language information; If the demand is for rest, the system triggers a Class I alarm and executes a first strategy, which includes: HUD displays reminder information; The system lowers the volume and changes the music. Based on this reminder strategy, the driver can continue driving, but should take a break for a while; If the alarm is turned off, the system turns off the alarm without any reminder; If the language information still cannot determine the driver's needs within the first threshold time, the system triggers a Class II alarm and executes a second strategy, which includes: The HUD displays reminder information, where the reminder text is red, which can effectively display the information that the driver needs to pay attention to; Voice reminders to the driver to rest or park can reduce the frequency with which the driver needs to look at the HUD.

[0055] It should be noted that the first threshold time is 8s. It is worth mentioning that the first threshold time can be directly customized by the user to improve the user experience.

[0056] The difference between this embodiment and embodiment 1 is that the touch screen operation can be operated by fingers, thereby realizing the display and control of vehicle information. Unlike voice control, the touch screen can input data more accurately, and the touch screen is also easier to use than voice control.

[0057] This embodiment provides a car, wherein the car is equipped with an intelligent HUD display control system for executing the intelligent HUD display method as described above.

[0058] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation to the invention.

[0059] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0060] Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present embodiment application. The appearance of this phrase in various positions in the specification is not necessarily the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It can be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0061] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An intelligent HUD display method, applied to an intelligent HUD display control system, characterized in that: The system includes a speech recognition module and a driver fatigue detection module, and the method includes: Acquiring current state information of the driver, wherein the state information includes the driver's action and physiological state; Determine whether the driver is fatigued based on the driver's current status information; If yes, continue to receive the driver's language information and determine the driver's needs based on the language information; If it is a rest demand, the system triggers a Class I alarm and executes a first strategy; If the alarm is turned off, the system turns off the alarm without any reminder; If the language information still cannot determine the driver's needs within the first threshold time, the system triggers a Class II alarm and executes a second strategy.

2. The intelligent HUD display method according to claim 1, characterized in that: The continuously receiving the driver's language information and determining the driver's demand according to the language information specifically includes: Collect the driver's voice information; Preprocessing the collected voice information, wherein the preprocessing includes denoising, filtering, and enhancement; Convert the pre-processed speech into text using speech recognition technology; Parse and understand the identified text and extract keywords; Identify driver needs based on keywords.

3. The intelligent HUD display method according to claim 1, characterized in that: The first strategy includes: HUD displays reminder information; The system lowers the volume and changes the music.

4. The intelligent HUD display method according to claim 1, characterized in that: The second strategy includes: The HUD displays reminder information, where the reminder text is red; Voice reminds the driver to take a break or stop.

5. The intelligent HUD display method according to claim 1, characterized in that: The method further comprises, After executing the second strategy, the driver turns off the reminder strategy by voice inputting a keyword, wherein the keyword includes: turning off the alarm or canceling the alarm.

6. The intelligent HUD display method according to claim 1, characterized in that: The speech recognition technology includes: The SLAM algorithm is used to determine the position and direction of the HUD, and the acquired voice commands are aligned with the estimated operation direction of the HUD to achieve precise positioning; Cognitive computing algorithms are used to analyze voice commands, including speech recognition and semantic understanding, to analyze the meaning of voice commands and user intent, and to generate corresponding commands and responses based on different contexts and needs.

7. The intelligent HUD display method according to claim 1, characterized in that: The physiological state includes the driver's head tilt angle, eye movements, and blinking frequency.

8. The intelligent HUD display method according to claim 1, characterized in that: The first threshold time ranges from 0 to 10 seconds.

9. An intelligent HUD display control system, characterized in that: The system comprises: Microphone, used to collect driver's voice information; A speech recognition module, used for obtaining the driver's needs according to the speech information; Camera, used to obtain the driver's current status information; Driver fatigue detection module, used to detect whether the driver is fatigued; HUD control module, used to execute corresponding reminder strategies according to driver voice information and fatigue information; HUD, used to project and display reminder strategy information.

10. An automobile, characterized in that: The automobile is equipped with an intelligent HUD display control system for executing the intelligent HUD display method according to any one of claims 1 to 8.

Images