A 1:1 AR HUD Real-Scene Projection Driving Assistance System and Method Based on Infrared Thermal Imaging

By using an infrared thermal imaging camera and image processing technology, complete infrared real-scene images are directly acquired and projected, solving the problems of insufficient field of view and algorithm error in low visibility environments in existing technologies. This enables high-precision, low-latency driving assistance, improving driving safety and system efficiency.

CN123113154APending Publication Date: 2026-11-17张平林
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610999351.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2026-05-04
Filing Date
2026-07-07
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing vehicle-mounted visual camera equipment cannot clearly identify road, obstacles, pedestrians and lane information in low visibility environments, resulting in blind spots. Furthermore, existing AR HUD solutions cannot achieve a complete 1:1 virtual-real scene overlay, leading to low driving safety, high algorithm consumption and high response latency.

Method used

The system uses an infrared thermal imaging camera to capture complete infrared real-scene images, performs noise reduction and geometric calibration through an image processing unit, and projects them onto the windshield at a 1:1 scale using an AR HUD display unit. The images are then overlaid on the real road scene, and brightness and contrast are adjusted using a light sensor to achieve real-time panoramic projection.

Benefits of technology

It achieves high-precision, blind-spot-free driving vision enhancement in low-visibility environments, eliminating the risk of missed detections and false detections, improving driving safety, simplifying the system architecture, reducing computing power consumption, and adapting to multiple scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113154A_ABST
    Figure CN123113154A_ABST
Patent Text Reader

Abstract

This invention discloses a 1:1 AR HUD real-scene projection driving assistance system and method based on infrared thermal imaging, belonging to the field of intelligent driving vehicle display technology. The system includes an infrared thermal imaging camera, an image processing unit, an AR HUD display unit, and a light sensor; the method acquires a complete infrared real-scene image of the front of the vehicle, preprocesses it, and performs dynamic geometric calibration, then projects it onto the vehicle's windshield in real time at a 1:1 scale, overlaying it with the real road scene. This invention completely abandons the technical logic of traditional AI object recognition and environmental visibility judgment, directly projecting the real-scene road view across the entire area, fundamentally solving the pain points of existing technology algorithms such as missed detections, blind spots, image misalignment, and high computational consumption; through matching the camera and AR HUD field of view and real-time compensation of vehicle dynamic data, it achieves accurate alignment of virtual and real scenes, adapting to various low-visibility driving scenarios such as nighttime, heavy fog, and heavy rain, significantly improving driving safety in low-visibility environments. The system architecture is simple, highly reliable, and has strong mass production feasibility.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of intelligent driving and in-vehicle AR display technology, specifically to a 1:1 AR HUD real-scene projection driving assistance system and method based on infrared thermal imaging. Background Technology

[0002] Low visibility driving scenarios such as nighttime, heavy fog, heavy rain, and sandstorms are high-risk environments for vehicle accidents. Conventional in-vehicle visual camera equipment is severely affected by ambient light and fog, making it impossible to clearly identify the road ahead, obstacles, pedestrians, and lane information, greatly reducing driving safety.

[0003] Currently, both in-vehicle night vision assistance and AR HUD driver assistance solutions in the industry have inherent technical flaws: mainstream solutions generally adopt a technical approach of "image acquisition + AI object recognition + virtual icon warning," which can only mark and alert identified targets such as vehicles and pedestrians, but cannot present complete real-world road information. These solutions are highly dependent on the accuracy of the recognition algorithm, making them prone to missed detections and false detections. Furthermore, they cannot identify non-standard targets such as potholes, scattered debris, and road edges, resulting in numerous blind spots and limited driving assistance safety.

[0004] Meanwhile, existing solutions lack the ability to overlay virtual and real scenes in a complete 1:1 ratio. The virtual warning icons have poor matching degree with the real road scene in terms of scale and location, which can easily lead to drivers misjudging distance and location. Moreover, most solutions rely on complex recognition algorithms and visibility judgment logic, which consume a lot of computing power and have high response latency, making them unsuitable for the real-time assistance needs of high-speed driving.

[0005] Given the many shortcomings of existing technologies, there is an urgent need for a driving assistance solution that can directly achieve a 1:1 accurate projection of the complete infrared real scene without the need for object recognition or visibility judgment, so as to completely solve the industry pain points of insufficient driving vision, algorithm missed detection, and image misalignment in low visibility environments.

[0006] This application claims priority to Chinese Patent Application No. 2026106019146, filed on May 4, 2026. The technical features involved in this application, such as the geometric calibration method, dynamic compensation algorithm, illumination sensor adjustment, and AR HUD field-of-view matching, are clearly described in the aforementioned priority application. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a 1:1 AR HUD real-scene projection driving assistance system and method based on infrared thermal imaging. It abandons the traditional AI recognition and warning technology route, eliminates the need for object recognition and visibility judgment, and directly collects and projects the complete infrared real scene to achieve blind-spot-free, high-precision, and highly reliable driving vision enhancement in low-visibility environments.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a 1:1 AR HUD real-scene projection driving assistance system based on infrared thermal imaging, comprising: An infrared thermal imaging camera, mounted at the front of the vehicle, is used to capture a complete infrared real-view image of the vehicle's direction of travel. An image processing unit, electrically connected to the infrared thermal imaging camera, is used to receive and process the infrared real-scene image and output a complete grayscale infrared real-scene image. The AR HUD display unit is electrically connected to the image processing unit and is used to project the processed complete infrared real-scene image onto the vehicle's windshield in real time at a 1:1 scale, and overlay it with the real road scene.

[0009] Furthermore, the projection field of view of the AR HUD display unit is the same as the shooting field of view of the infrared thermal imaging camera.

[0010] Furthermore, the image processing unit includes an image preprocessing module, a geometric calibration module, and a projection control module; The image preprocessing module is used to denoise and correct distortion in the acquired infrared images; The geometric calibration module is used to perform dynamic geometric calibration on the image based on the camera installation parameters and the vehicle's motion state, so that the objects in the image correspond one-to-one with the objects in the real scene in terms of position and scale. The projection control module is used to control the projection parameters of the AR HUD display unit.

[0011] Furthermore, the geometric calibration module dynamically compensates the image based on the vehicle's real-time speed, steering angle, and body posture data to eliminate image offset caused by vehicle movement.

[0012] Furthermore, the system also includes a light sensor electrically connected to the image processing unit for detecting ambient light intensity; the projection control module can automatically adjust the projection brightness and contrast of the AR HUD display unit according to the ambient light intensity.

[0013] Furthermore, the infrared thermal imaging camera is an uncooled infrared detector with a resolution of not less than 640×512 and a NETD of less than 50mK.

[0014] Secondly, the present invention provides a 1:1 AR HUD real-scene projection driving assistance method based on infrared thermal imaging, comprising the following steps: S1: Acquire real-time complete infrared real-scene images of the vehicle's forward direction using an infrared thermal imaging camera; S2: Denoise and distortion correction are performed on the acquired infrared real-world images; S3: Based on the camera's installation parameters and the vehicle's movement status, perform geometric calibration on the image to ensure that the objects in the image correspond one-to-one with the objects in the real scene in terms of position and scale. S4: Send the calibrated complete infrared real-scene image to the AR HUD display unit; S5: The AR HUD display unit projects images onto the vehicle's windshield in real time at a 1:1 scale, overlaying them with the real road scene.

[0015] Furthermore, the geometric calibration described in step S3 specifically includes: Calculate the real-world coordinates of each pixel in the image based on the camera's installation height, pitch angle, and horizontal angle. Based on the calculation results, a perspective transformation is performed on the image to keep the straight lines in the image parallel to the straight lines in the real world. Based on the vehicle's real-time speed, steering angle, and body posture data, the image is dynamically compensated to eliminate image offset caused by vehicle movement.

[0016] Furthermore, the execution order of steps S2 and S3 can be interchanged.

[0017] Furthermore, the method also includes the following steps: detecting ambient light intensity using a light sensor; and automatically adjusting the projection brightness and contrast of the AR HUD display unit based on the detected ambient light intensity.

[0018] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the 1:1 AR HUD real-scene projection driving assistance method based on infrared thermal imaging as described in any of the preceding claims.

[0019] This invention completely abandons the object recognition and visibility judgment technology approach, directly projecting a complete infrared real-world image at a 1:1 scale. This eliminates the risk of missed detections and false detections at the source, and is adaptable to various unknown and unconventional road obstacle scenarios, significantly improving driving safety. The solution fully presents all road condition information, including road edges, potholes, and debris, achieving true full-view enhancement. Through field-of-view matching and real-time compensation of vehicle dynamic data, it ensures accurate and error-free superposition of the projected image with the real road, eliminating image misalignment and distance misjudgment. The system architecture is simple, with low computing power consumption, achieving low-latency projection without the need for high-performance domain controllers, making it highly feasible for mass production. Simultaneously, relying on a light sensor to adaptively adjust brightness and contrast, it seamlessly adapts to all environmental scenarios such as darkness, dense fog, rain, and snow, combining practicality and reliability. Attached Figure Description

[0020] Figure 1 This is a schematic block diagram of the 1:1 AR HUD real-scene projection driving assistance system based on infrared thermal imaging of the present invention.

[0021] Figure 2 This is a flowchart of the 1:1 AR HUD real-scene projection driving assistance method based on infrared thermal imaging according to the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to specific embodiments.

[0023] This invention discloses a 1:1 AR HUD real-scene projection driving assistance system based on infrared thermal imaging. The core of the system consists of an infrared thermal imaging camera, an image processing unit, an AR HUD display unit, and a light sensor. It completely abandons the object recognition and environmental visibility judgment logic in traditional driving assistance solutions, and takes the direct projection of the complete real scene as the core innovation point, adapting to the low visibility driving assistance needs of all scenarios.

[0024] The infrared thermal imaging camera uses an automotive-grade uncooled infrared detector with a resolution of no less than 640×512 and a NETD of less than 50mK. It boasts ultra-high thermal sensitivity, capable of penetrating fog, darkness, and dust to accurately capture a complete infrared view of the road ahead, eliminating blind spots. The camera is fixedly mounted at the front of the vehicle, ensuring a perfect match between the field of view and the vehicle's path.

[0025] The image processing unit is the core processing module of the system, integrating an image preprocessing module, a geometric calibration module, and a projection control module. The image preprocessing module performs noise reduction and distortion correction on the raw infrared image, removing image distortion caused by lens distortion and environmental noise. The geometric calibration module combines static parameters such as the fixed installation height, pitch angle, and horizontal angle of the camera with dynamic driving data such as real-time vehicle speed, steering angle, and vehicle posture to complete image perspective transformation and dynamic compensation, completely eliminating image offset and misalignment caused by vehicle movement, steering, and bumps, ensuring that the image matches the real road at a 1:1 scale and with precise positional correspondence.

[0026] The projection field of view of the AR HUD display unit is completely consistent with the field of view of the infrared thermal imaging camera. It receives the complete grayscale infrared real-scene image output by the image processing unit and projects it onto the vehicle's windshield in real time at a 1:1 scale, seamlessly overlaying it with the real road scene. The driver does not need to look down at the central control screen; he can look straight ahead and simultaneously observe the real road conditions and the enhanced infrared real-scene, completely eliminating blind spots.

[0027] The light sensor collects real-time light intensity data of the driving environment and transmits it synchronously to the image processing unit. The projection control module adaptively adjusts the projection brightness and contrast of the AR HUD to achieve image adaptation in all lighting scenarios, such as strong light during the day, low light at night, and cloudy or foggy days, avoiding the problems of overly bright and dazzling images or overly dark images that are difficult to see.

[0028] The specific execution process of the corresponding driver assistance method is as follows: First, an infrared thermal imaging camera acquires a complete infrared real-scene image of the vehicle's forward direction in real time. Then, the image undergoes noise reduction and distortion correction preprocessing. Next, geometric calibration is performed by combining the camera installation parameters with the vehicle's dynamic driving data to achieve precise matching of the image's proportion and position with the real scene. Finally, the calibrated complete real-scene image is overlaid and projected 1:1 using an AR HUD. The entire process requires no object recognition or environmental visibility judgment, directly outputting a complete real-scene view of the road, thus fundamentally avoiding the risks of algorithmic missed detections and false detections.

[0029] It should be noted that although the above embodiments illustrate the execution of S2 (denoising and distortion correction) followed by S3 (geometric calibration) as an example, those skilled in the art will understand that the execution order of S2 and S3 can be interchanged according to actual needs. That is, geometric calibration can be performed first, followed by denoising and distortion correction. Such interchanges can also achieve the technical effects of the present invention and fall within the scope of protection of the present invention.

Claims

1. An infrared thermography based 1:1 AR HUD real scene projection driving assistance system, characterized in that, include: An infrared thermal imaging camera, mounted at the front of the vehicle, is used to capture a complete infrared real-view image of the vehicle's direction of travel. An image processing unit, electrically connected to the infrared thermal imaging camera, is used to receive and process the infrared real-scene image and output a complete grayscale infrared real-scene image. The AR HUD display unit is electrically connected to the image processing unit and is used to project the processed complete infrared real-scene image onto the vehicle's windshield in real time at a 1:1 scale, and overlay it with the real road scene.

2. The system of claim 1, wherein, The projection field of view of the AR HUD display unit is the same as the shooting field of view of the infrared thermal imaging camera.

3. The system of claim 1, wherein, The image processing unit includes an image preprocessing module, a geometric calibration module, and a projection control module; The image preprocessing module is used to denoise and correct distortion in the acquired infrared images; The geometric calibration module is used to perform dynamic geometric calibration on the image based on the camera installation parameters and the vehicle's motion state, so that the objects in the image correspond one-to-one with the objects in the real scene in terms of position and scale. The projection control module is used to control the projection parameters of the AR HUD display unit.

4. The system of claim 3, wherein, The geometric calibration module dynamically compensates the image based on the vehicle's real-time speed, steering angle, and body posture data to eliminate image offset caused by vehicle movement.

5. The system of claim 1, wherein, It also includes a light sensor, which is electrically connected to the image processing unit, for detecting ambient light intensity; the projection control module can automatically adjust the projection brightness and contrast of the ARHUD display unit according to the ambient light intensity.

6. The system of claim 1, wherein, The infrared thermal imaging camera is an uncooled infrared detector with a resolution of not less than 640×512 and a NETD of less than 50mK. 7.A 1:1 AR HUD real scene projection driving assistance method based on infrared thermal imaging, characterized in that, Includes the following steps: S1: Acquire real-time complete infrared real-scene images of the vehicle's forward direction using an infrared thermal imaging camera; S2: Denoise and distortion correction are performed on the acquired infrared real-world images; S3: Based on the camera's installation parameters and the vehicle's movement status, perform geometric calibration on the image to ensure that the objects in the image correspond one-to-one with the objects in the real scene in terms of position and scale. S4: Send the calibrated complete infrared real-scene image to the AR HUD display unit; S5: The AR HUD display unit projects images onto the vehicle's windshield in real time at a 1:1 scale, overlaying them with the real road scene.

8. The method of claim 7, wherein, The geometric calibration described in step S3 specifically includes: Calculate the real-world coordinates of each pixel in the image based on the camera's installation height, pitch angle, and horizontal angle. Based on the calculation results, a perspective transformation is performed on the image to keep the straight lines in the image parallel to the straight lines in the real world. Based on the vehicle's real-time speed, steering angle, and body posture data, the image is dynamically compensated to eliminate image offset caused by vehicle movement.

9. The method of claim 7, wherein, The execution order of steps S2 and S3 can be interchanged.

10. The method of claim 7, wherein, It also includes the following steps: The ambient light intensity is detected using a light sensor; The AR HUD display unit automatically adjusts its projection brightness and contrast based on the detected ambient light intensity.

11. A computer readable storage medium having stored thereon a computer program, characterized in that When the computer program is executed by a processor, it implements the method of any one of claims 7-10.