Detection method and detection device of head-up display, head-up display system and vehicle
By acquiring and comparing the coordinates of the real and virtual calibration targets in the head-up display projected image, the system automatically detects head-up display projection offset, solving the problem that drivers find it difficult to detect projection gear shifts and improving driving safety and experience.
Patent Information
- Application Number
- CN202511296769.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-11
- Publication Date
- 2026-01-06
AI Technical Summary
Existing head-up displays may experience projection shifts due to vehicle vibrations or other reasons during use, which may be difficult for drivers to notice, affecting driving safety and experience.
By acquiring the real environmental information of the image projected by the head-up display, the coordinates of the real calibration target are extracted, a virtual calibration target is generated, and the coordinates of the virtual and real calibration targets are compared to determine whether there is an offset. If there is an offset, an alarm signal is issued.
It enables automatic detection of head-up display projection offset, reducing safety hazards caused by drivers not being aware of the offset, and improving driving safety and user experience.
Smart Images
Figure CN121268554A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of head-up display technology, and in particular to a head-up display detection method, detection device, head-up display system, and vehicle. Background Technology
[0002] Head-up displays (HUDs) improve driving safety by projecting key driving information directly in front of the driver's line of sight, effectively reducing the number of times the driver needs to look down at the instrument panel. Augmented Reality Head-up Displays (AR-HUDs), in particular, provide a more intelligent and immersive driving experience by rendering virtual content that matches the actual road conditions in real time, such as lane markings, pedestrian and vehicle collision warnings. However, over time, the head-up display of an AR-HUD system may become loose due to vehicle vibrations, causing the projection to shift. This shift can cause the virtual content projected by the AR-HUD to no longer accurately match the actual environment, thus affecting the driver's visual experience and driving safety.
[0003] Currently, the accuracy of the projection gear of a head-up display mainly depends on the driver's subjective judgment. However, in daily driving, drivers may gradually become accustomed to the projected effect after the gear shift, making it difficult for them to notice that the gear has shifted, thus increasing safety hazards. Summary of the Invention
[0004] To address the aforementioned technical problems, embodiments of the present invention provide a detection method, detection device, head-up display system, and vehicle for detecting whether a head-up display has projection offset.
[0005] The technical solutions adopted by the embodiments of the present invention to solve their technical problems are as follows: A method for detecting a head-up display (HUD) includes: acquiring real-world environmental information of an image projected by the HUD; extracting a real calibration target and its first coordinates from the real-world environmental information; generating a virtual calibration target based on the real calibration target and controlling the HUD to project the virtual calibration target; acquiring the virtual calibration target projected by the HUD and obtaining its second coordinates; determining whether the virtual calibration target is offset relative to the real calibration target based on the first coordinates and the second coordinates; and if an offset exists, determining that the image projected by the HUD has a deviation.
[0006] Optionally, the step of determining whether the virtual calibration target is offset relative to the real calibration target based on the first coordinate and the second coordinate further includes: converting the first coordinate of the real calibration target into a first transformed coordinate in a preset coordinate system according to a preset first transformation model; converting the second coordinate of the virtual calibration target into a second transformed coordinate in a preset coordinate system according to a preset second transformation model; and determining whether the virtual calibration target is offset relative to the real calibration target based on the first transformed coordinate and the second transformed coordinate.
[0007] Optionally, both the virtual and real proofreading targets are several matching points. The step of determining whether the virtual proofreading target is offset relative to the real proofreading target based on the first and second transformation coordinates further includes: calculating the distance values of the first and second transformation coordinates of each matching point; calculating the average value of the distance values of all matching points; determining whether the average value is greater than a preset threshold; if the average value is greater than the preset threshold, then determining that the virtual proofreading target is offset relative to the real proofreading target; otherwise, determining that the virtual proofreading target is not offset relative to the real proofreading target.
[0008] Optionally, both the virtual and real calibration targets are matching lines. The step of determining whether the virtual calibration target is offset relative to the real calibration target based on the first and second transformation coordinates further includes: calculating the vertical distance from each point of the real calibration target to the matching line of the virtual calibration target; calculating the average value of the vertical distances from all points of the real calibration target to the matching line of the virtual calibration target; determining whether the average value is greater than a preset threshold; if the average value is greater than the preset threshold, then determining that the virtual calibration target is offset relative to the real calibration target; otherwise, determining that the virtual calibration target is not offset relative to the real calibration target.
[0009] Optionally, the method further includes: if it is determined that there is a deviation in the image projected by the head-up display, then determining the offset based on the difference between the average value and a preset threshold.
[0010] Optionally, the method further includes: issuing an alarm signal if it is determined that there is a deviation in the image projected by the head-up display.
[0011] Optionally, the alarm signal is at least one of a projected virtual image signal, an audio signal, and a light signal.
[0012] The technical solutions adopted by the embodiments of the present invention to solve their technical problems are as follows: A detection device for a head-up display (HUD) system includes an acquisition module, an extraction module, a control module, a collection module, a judgment module, and a determination module. The acquisition module acquires the real environment of the image projected by the HUD. The extraction module extracts a real calibration target from the real environment and the first coordinates of the real calibration target. The control module generates a virtual calibration target based on the real calibration target and controls the HUD to project the virtual calibration target. The collection module collects the virtual calibration target projected by the HUD and acquires the second coordinates of the virtual calibration target. The judgment module determines whether the virtual calibration target is offset relative to the real calibration target based on the first and second coordinates. The determination module determines that if an offset exists, the image projected by the HUD has a deviation.
[0013] The technical solutions adopted by the embodiments of the present invention to solve their technical problems are as follows: A head-up display system includes a head-up display, a three-dimensional ranging sensing module, an image acquisition module, at least one processor, and a memory. The head-up display is used to project images. The three-dimensional ranging sensing module is used to acquire real-world environmental information of the projected images from the head-up display. The image acquisition module is used to acquire images projected by the head-up display. The at least one processor is connected to the three-dimensional ranging module, the image acquisition module, and the head-up display, respectively. The memory is communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the method described in any one of claims 1-7.
[0014] The beneficial effects of this application's embodiments are as follows: The head-up display (HUD) detection method provided in this application includes: acquiring real environmental information of the HUD-projected image; extracting a real calibration target from the real environmental information, and the first coordinates of the real calibration target; generating a virtual calibration target based on the real calibration target, and controlling the HUD to project the virtual calibration target; acquiring the virtual calibration target projected by the HUD, and acquiring the second coordinates of the virtual calibration target; determining whether the virtual calibration target is offset relative to the real calibration target based on the first coordinates and the second coordinates; if an offset exists, determining that the HUD-projected image has a deviation. Through the above detection method, it is possible to automatically detect whether the HUD projection is offset, effectively reducing safety hazards caused by the driver's failure to perceive the offset, and enhancing driving safety and user experience. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the drawings without creative effort.
[0016] Figure 1 This is a schematic diagram of an application environment for a head-up display system; Figure 2 This is a schematic flowchart of a head-up display detection method provided in an embodiment of this application; Figure 3 yes Figure 2 A schematic diagram of a sub-process of step S50 in the detection method shown; Figure 4 yes Figure 3 A schematic diagram of a sub-process of step S53 in the detection method shown; Figure 5 yes Figure 3 A schematic diagram of another sub-process of step S53 in the detection method shown; Figure 6 This is a schematic diagram of the structure of a detection device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a head-up display system provided in an embodiment of this application; Figure 8 This is a schematic diagram of an application environment for a head-up display system provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the head-up display provided in the embodiments of this application. Detailed Implementation
[0017] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as being "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as being "connected" to another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this specification are for illustrative purposes only.
[0018] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.
[0019] In a head-up display system, please refer to Figure 1 The head-up display 1 generates image light and projects it onto a display area 21 on the windshield 2. The windshield 2 reflects the image light into the eye box area 3. After the human eye receives the reflected light, a virtual image 4 is formed on the imaging plane corresponding to the display area 21. The virtual image 4 may include a realistic virtual image based on the real environment 5, such as lane markings. This realistic virtual image corresponds to and fits the real environment 5 to provide visual guidance to the driver and improve driving safety. When the head-up display 1 becomes loose during long-term use, it shifts at an angle, causing an angular and / or displacement deviation between the projected image light and the display area 21. This results in a shift in the formation position of the virtual image 4, and the realistic virtual image is no longer perfectly aligned with the real environment 5, reducing driving safety and affecting the driving experience.
[0020] To address the aforementioned technical issues, this application provides a head-up display (HUD) detection method, detection device, HUD system, and vehicle for detecting whether the HUD has projection offset, thereby improving driving safety and driving experience.
[0021] The following provides explanations for some of the terms used in this application. It should be noted that these explanations are for the convenience of those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.
[0022] 1. World Coordinate System The world coordinate system is a fixed and unchanging reference coordinate system established based on the real environment, used to describe the absolute position of objects in three-dimensional space.
[0023] 2. Eye Box Coordinate System The eyebox coordinate system refers to a coordinate system established with the center of the eyebox area as the origin. It is used to describe the position information of the human eye relative to the human eye when observing virtual or real scenes.
[0024] 3. Physical camera coordinate system The physical camera coordinate system refers to a coordinate system established with the physical camera as the origin.
[0025] 4. First image coordinate system The first image coordinate system refers to a two-dimensional coordinate system based on the image plane of the physical camera. The origin of the coordinate system is the intersection of the optical axis center of the physical camera and the imaging plane. It is introduced to describe the relationship from the physical camera coordinate system to the first pixel coordinate system during the imaging process.
[0026] 5. First pixel coordinate system The first pixel coordinate system refers to the pixel coordinate system of the image captured by the physical camera on the imaging plane. The first pixel coordinate system corresponds to the first image coordinate system.
[0027] 6. Virtual Camera The virtual camera corresponds to the center position of the eye box. The virtual camera refers to a camera model simulated based on the center position of the eye box and preset parameters, used to simulate the imaging effect observed by the human eye at the center of the eye box.
[0028] 7. Virtual Camera Coordinate System The virtual camera coordinate system refers to a coordinate system established with the virtual camera as the origin. The virtual camera coordinate system is equivalent to the eye box coordinate system.
[0029] 8. Second image coordinate system The second image coordinate system refers to a two-dimensional coordinate system based on the image plane of the virtual camera. The origin of the coordinate system is the intersection of the center of the optical axis of the virtual camera and the imaging plane. It is introduced to describe the relationship between the virtual camera coordinate system and the second pixel coordinate system during the imaging process.
[0030] 9. Second pixel coordinate system The second pixel coordinate system refers to the pixel coordinate system of the image obtained by acquiring the imaging plane through the virtual camera. The second pixel coordinate system corresponds to the second image coordinate system.
[0031] Firstly, embodiments of this application provide a method for detecting a head-up display (HUD). Please refer to [link to relevant documentation]. Figure 2 The detection method includes the following steps: Step S10: Obtain the real-world information of the image projected by the head-up display.
[0032] Specifically, the 3D ranging sensing module acquires real-world environmental information from the image projected by the head-up display. This real-world environmental information is a point cloud or depth map. The 3D ranging sensing module can be a LiDAR, millimeter-wave radar, binocular camera, TOF camera, etc. It can also be formed by combining multiple sensors, such as an RGB camera combined with LiDAR, an RGB camera combined with a depth camera, or an RGB camera combined with a binocular camera. The real-world environment refers to the surrounding environment outside the vehicle that can be observed through the windshield from the viewpoint.
[0033] Step S20: Extract the real calibration target and the first coordinates of the real calibration target from the real environment information.
[0034] Specifically, computer vision algorithms, such as object detection and semantic segmentation, or deep learning models, such as YOLO (You Only Look Once) and Mask R-CNN (Mask Region-based Convolutional Neural Network), are used to identify and extract matching points and / or matching lines from point clouds or depth maps. These matching points and lines represent the real-world calibration targets. Matching points correspond to environmental intersections in the real environment, such as corners or bends. Matching lines correspond to environmental edge lines in the real environment, such as vertical lines at the intersection of building walls or horizontal lines at the edge of road shoulders. The first coordinate of the real-world calibration target refers to the coordinates of the matching points and / or matching lines based on the world coordinate system.
[0035] In some embodiments, the extracted real calibration target is further subjected to fitting optimization processing to further improve the extraction accuracy and obtain the optimized first coordinates.
[0036] Step S30: Generate virtual proofing target information based on the real proofing target, and control the head-up display to project the virtual proofing target.
[0037] Specifically, based on the relative positional relationship between the 3D ranging sensing module and the center position of the eye box, the first coordinate of the real calibration target is transformed to the eye box coordinate system to obtain the position information of the virtual calibration target. Based on the position information of the virtual calibration target, combined with preset display parameters, rendering processing is performed to obtain the rendered virtual calibration target information.
[0038] Transforming the first coordinate of the real calibration target to the eyebox coordinate system is equivalent to transforming the first coordinate of the real calibration target from the world coordinate system to the virtual camera coordinate system, as shown in the following expression: , , Among them, P W To accurately calibrate the first coordinate of the target in the world coordinate system, P C To accurately calibrate the target's first camera coordinates in the virtual camera coordinate system, [R A | t A ] represents the extrinsic parameter matrix of the virtual camera, R A Let t be the extrinsic rotation matrix of the virtual camera. A Let R be the extrinsic translation matrix of the virtual camera. A | t A [] is a 3*4 matrix used to describe the rigid body transformation from the world coordinate system to the virtual camera coordinate system.
[0039] Preset display parameters refer to the parameters that the head-up display (HUD) is pre-set to control the display effect of the virtual calibration target. These include display attribute parameters such as the virtual calibration target's color, size, shape, transparency, and brightness, as well as layout parameters such as the virtual calibration target's display position and display hierarchy within the HUD display interface. Virtual calibration target information refers to a data set containing parameters such as the virtual calibration target's position, display attributes, and layout, used to determine the imaging effect of the virtual calibration target.
[0040] In some embodiments, during rendering, matching points and / or matching points are marked to distinguish the virtual calibration target from other rendered content and the real environment observed through the windshield, facilitating subsequent acquisition of the projected virtual calibration target. In some examples, the marking process is highlighting.
[0041] In some embodiments, the head-up display includes a control module and an optical component. The control module controls the optical component to generate and output image light of the virtual calibration target based on the rendered virtual calibration target information. The image light is projected onto the display area of the windshield to form a corresponding virtual calibration target.
[0042] Step S40: Acquire the virtual calibration target projected by the head-up display and obtain the second coordinates of the virtual calibration target.
[0043] Specifically, a physical camera is used to capture images of the display area, extract virtual calibration targets from the images, and determine the second coordinates of the virtual calibration targets in the first pixel coordinate system. The physical camera is positioned outside the eye box area.
[0044] In some embodiments, due to factors such as the lens structure and manufacturing process of the physical camera, light undergoes non-ideal refraction when passing through the lens, resulting in lens distortion in the image. Furthermore, the curvature and material properties of the windshield may introduce additional optical path distortion, causing non-linear deformation of the virtual image on the imaging plane. To improve detection accuracy, obtaining the second coordinates of the virtual calibration target also includes distortion correction processing of the second coordinates.
[0045] In some embodiments, the distortion correction process for the second coordinate includes lens distortion correction and optical path distortion correction.
[0046] In some embodiments, lens distortion correction processing is performed on the second coordinates, including: establishing a blank image; mapping all pixels of the blank image to the imaging plane of the physical camera coordinate system to obtain the normalized coordinates of the pixels of the blank image; obtaining the corresponding distortion coordinates of the pixels of the blank image through iterative calculation and interpolation algorithms based on lens distortion parameters and distortion models; converting the distortion coordinates into distorted pixel coordinates and establishing a correspondence between the pixel coordinates of the blank image and the distorted pixel coordinates; determining the pixel of the blank image corresponding to the second coordinate based on the correspondence, and the corresponding coordinate of the pixel is the second coordinate after lens distortion correction.
[0047] In some embodiments, optical path distortion correction processing is performed on the second coordinates, including: obtaining optical path distortion parameters corresponding to the physical camera position; and performing optical path distortion correction processing on the second coordinates based on the optical path distortion parameters.
[0048] In some embodiments, before obtaining the optical path distortion parameters corresponding to the physical camera position, a correspondence between different camera positions or eyepiece positions and optical path distortion parameters is first established. This correspondence can be obtained through pre-shipment calibration or pre-test calibration. Therefore, based on the correspondence between different camera positions or eyepiece positions and optical path distortion parameters, the optical path distortion parameters of the physical camera position can be determined. The calibration method is existing and will not be elaborated upon here.
[0049] In some embodiments, the second coordinates are subjected to optical path distortion correction processing based on optical path distortion parameters. Specifically, the optical path distortion parameters are distortion vectors. The second coordinates are inversely translated based on the distortion vectors to obtain the second coordinates after optical path distortion correction processing.
[0050] Step S50: Determine whether the virtual calibration target is offset relative to the real calibration target based on the first coordinate and the second coordinate.
[0051] Specifically, please refer to the following: Figure 3 , Figure 3 yes Figure 2 A sub-process of step S50, which involves determining whether the virtual calibration target is offset relative to the real calibration target based on the first and second coordinates, further includes: Step S51: Based on the preset first transformation model, convert the first coordinates of the actual calibration target into the first transformed coordinates under the preset coordinate system.
[0052] In some embodiments, the preset coordinate system refers to the second pixel coordinate system. The imaging plane refers to the plane on which the virtual image corresponding to the image light projected by the head-up display is reflected by the windshield and observed by the human eye. The distance from the optical center of the virtual camera to the imaging plane is equal to the focal length of the virtual camera.
[0053] In some embodiments, the preset first conversion model includes the following process: First, obtain the first camera coordinates P based on the first coordinate transformation. C First camera coordinates P C To accurately calibrate the target's coordinates in the virtual camera coordinate system. In some examples, the first camera coordinates P... C It is obtained by extracting the coordinate transformation results from step S30. In some other examples, the first camera coordinates P C It was obtained by recalculating based on the transformation formula from the world coordinate system to the virtual camera coordinate system.
[0054] Next, the real calibration target is projected from the virtual camera coordinate system onto the imaging plane, and the first camera coordinates are transformed from the virtual camera coordinate system to the second image coordinate system to obtain the first image coordinates, as shown in the following expression: .
[0055] Among them, P norm The first image coordinates are the coordinates of the actual calibration target in the second image coordinate system.
[0056] Then, the first image coordinates are transformed from the second image coordinate system to the second pixel coordinate system to obtain the first pixel coordinates, as shown in the following expression: , , Wherein, the first pixel coordinate P is the first transformation coordinate, K A For the intrinsic parameter matrix of the virtual camera, (f x , f y ) is the scale factor of the virtual camera's focal length f in pixels, (c x , c y The principal point coordinates are the pixel coordinates of the intersection of the optical axis of the virtual camera and the imaging plane. The intrinsic parameter matrix K... A It is a 3x3 matrix used to describe the transformation from the virtual camera coordinate system to the corresponding pixel coordinate system.
[0057] It should be noted that in the detection method of this application, the positions of the eye box range, the three-dimensional ranging sensing module, the physical camera and the virtual camera are fixed, and the intrinsic and extrinsic parameters of the physical camera and the virtual camera are obtained through camera calibration.
[0058] Step S52: Based on the preset second transformation model, convert the second coordinates of the virtual calibration target into the second transformation coordinates under the preset coordinate system.
[0059] The preset second conversion model is: Perform homography transformation to convert the second coordinates of the virtual calibration target from the first pixel coordinate system to the second pixel coordinate system, as shown in the following expression: , , in, The second coordinate, K represents the second transformation coordinates, which are the coordinates of the virtual calibration target in the second pixel coordinate system. B R is the intrinsic parameter matrix of the physical camera. B Let t be the external parameter rotation matrix of the physical camera. B Let be the extrinsic translation matrix of the physical camera. [R] B | t B [] represents the extrinsic parameter matrix of the physical camera, used to describe the rigid body transformation from the world coordinate system to the physical camera coordinate system. n Let d be the unit normal vector of the imaging plane, and d be the distance from the imaging plane to the origin of the world coordinate system.
[0060] In some embodiments, the intrinsic parameters of the physical camera are the same as those of the virtual camera.
[0061] Step S53: Determine whether the virtual calibration target is offset relative to the real calibration target based on the first transformation coordinate and the second transformation coordinate.
[0062] In some embodiments, both the virtual proofreading target and the real proofreading target are several matching points; please refer to the following: Figure 4 , Figure 4 for Figure 3 A sub-process of step S53. The step of determining whether the virtual calibration target is offset relative to the real calibration target based on the first and second transformation coordinates further includes: Step S5311: Calculate the distance between the first transformed coordinates and the second transformed coordinates of each matching point.
[0063] , in, The Euclidean distance between the first and second transformed coordinates of each matching point.
[0064] Step S5312: Calculate the average distance value of all matching points.
[0065] The average distance between N pairs of matching points is .
[0066] Step S5313: Determine whether the average value is greater than the preset threshold.
[0067] The preset threshold is The preset threshold refers to the critical distance value that can ensure driving safety and experience under the requirements of projection accuracy in actual driving scenarios. The preset threshold is determined based on factors such as different vehicle models, windshield characteristics, and driver visual sensitivity.
[0068] Step S5314: If the average value is greater than the preset threshold, then determine the offset of the virtual proofreading target relative to the real proofreading target.
[0069] .
[0070] Step S5315: Otherwise, determine that the virtual proofreading target is not offset relative to the real proofreading target.
[0071] In other embodiments, both the virtual proofreading target and the real proofreading target are matching lines; please refer to [link to relevant documentation]. Figure 5 , Figure 5 yes Figure 3 Another sub-process of step S53. The step of determining whether the virtual calibration target is offset relative to the real calibration target based on the first and second transformation coordinates further includes: Step S5321: Calculate the perpendicular distance from each point of the real calibration target to the matching line of the virtual calibration target.
[0072] True calibration target corresponding to matching line l A Matching line l A Point P on i =( u i， v i Virtual calibration target corresponding to matching line l B , .
[0073] Point P i To the matching line l B The distance is .
[0074] Step S5322: Calculate the average vertical distance from all points of the real calibration target to the matching line of the virtual calibration target.
[0075] The average vertical distance from all points of the real calibration target to the matching line of the virtual calibration target is .
[0076] Step S4323: Determine whether the average value is greater than the preset threshold.
[0077] The preset threshold is .
[0078] Step S5324: If the average value is greater than the preset threshold, then determine the offset of the virtual proofreading target relative to the real proofreading target.
[0079] .
[0080] Step S5325: Otherwise, determine that the virtual proofing target is not offset relative to the real proofing target.
[0081] In some embodiments, it is understood that the position of the virtual calibration target on the imaging plane is slightly offset from the position of the real calibration target on the imaging plane. When the real calibration target is the matching line, the real calibration target and the virtual calibration target are matched by nearest neighbor matching to determine the matching line.
[0082] In some other embodiments, the virtual proofreading target and the real proofreading target include both matching points and matching lines, then the execution... Figure 4 and Figure 5 The steps in the process.
[0083] Step S60: If an offset exists, it is determined that there is a deviation in the image projected by the head-up display.
[0084] If it is determined that the virtual calibration target is offset relative to the real calibration target, it means that the head-up display failed to accurately align the virtual calibration target with the real calibration target in the real environment during the projection process, and there is a deviation in the image projected by the head-up display.
[0085] Step S70: If there is no offset, then it is determined that there is no deviation in the image projected by the head-up display.
[0086] When it is determined that the virtual calibration target is not offset relative to the real calibration target, it indicates that the head-up display can accurately align the virtual calibration target with the real calibration target in the real environment during the projection process. In other words, there is no deviation in the image projected by the head-up display. At this time, it can be considered that the projection accuracy of the head-up display in the current detection scenario meets the requirements.
[0087] In some embodiments, please refer to Figure 2 The detection method also includes: Step S80: If it is determined that there is a deviation in the image projected by the head-up display, an alarm signal is issued.
[0088] Specifically, the warning signal is at least one of the following: a projected virtual image signal, an audible signal, and a visual signal. The projected virtual image signal is a virtual image projected onto the windshield via the head-up display, such as a virtual image of a red warning sign. The audible signal is an alarm sound with a specific frequency and rhythm emitted through speakers inside the vehicle, such as a rapid beeping sound. The visual signal is the flashing or color change of a specific indicator light on the vehicle's instrument panel, such as an indicator light changing from green to red and flashing continuously. By issuing warning signals, the driver is alerted to a projection misalignment problem on the head-up display, enabling the driver to take timely countermeasures and improving driving safety and experience.
[0089] In some embodiments, please refer to Figure 2 The detection method also includes: Step S90: If it is determined that there is a deviation in the image projected by the head-up display, the offset is determined based on the difference between the average value and the preset threshold.
[0090] Specifically, offset refers to the distance between the actual area projected onto the windshield by the head-up display and the preset display area, or the difference between the actual angle of the head-up display and the preset installation angle. A correspondence between the difference and the offset can be established in advance, and the corresponding offset can be obtained based on this correspondence. By determining the specific offset, drivers or vehicle maintenance personnel can more accurately understand the degree of head-up display offset, providing a quantitative basis for subsequent adjustments or repairs.
[0091] In some embodiments, the warning signal includes offset information so that the driver can intuitively understand the offset status. For example, in the projected virtual image signal, in addition to displaying a virtual image of a red warning sign, the offset value can also be presented in numerical form next to the warning sign; in the sound signal, the magnitude of the offset can be conveyed through different combinations of sound frequencies or rhythms; and the light signal can indicate the offset by changing the flashing frequency or color intensity of the indicator light according to different ranges of the offset.
[0092] Understandably, in order to meet the needs of drivers of different heights and driving postures, vehicles usually have multiple preset eye box ranges of different heights in head-up display systems, and different eye box ranges of different heights correspond to different head-up display projection levels.
[0093] In some embodiments, before step S10, i.e., before acquiring the real-world environmental information of the head-up display (HUD) projected image, the detection method further includes acquiring the position information of the currently set eye box range. This enables real-time detection of whether the HUD has projection offset, improving the accuracy and adaptability of the detection method. Specifically, in some examples, the position information of the driver's eyes is detected using an infrared camera or other eye-tracking device to match the corresponding eye box range and the HUD projection level. In other examples, the current HUD projection level is manually selected by the driver, and the position information of the current eye box range is obtained based on the correspondence between the projection level and the eye box range position.
[0094] In this embodiment of the application, the head-up display detection method described above can detect whether the head-up display has a projection offset. This not only allows the driver to be aware of the abnormal state of the head-up display in a timely manner and take corresponding measures, which is beneficial to improving driving safety and driving experience, but also eliminates the need to move the vehicle to a specific location to detect the projection offset of the head-up display manually or with other equipment, which helps to reduce usage costs and enhance user experience.
[0095] Secondly, based on the above content and the same concept, please refer to... Figure 6 This application provides a detection device 100, which is used to implement the functions of the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments.
[0096] like Figure 6 As shown, the detection device 100 includes an acquisition module 101, an extraction module 102, a control module 103, a collection module 104, a judgment module 105, and a determination module 106. The acquisition module 101 acquires the real environment of the image projected by the head-up display. The extraction module 102 extracts the real calibration target from the real environment, and the first coordinates of the real calibration target. The control module 103 generates a virtual calibration target based on the real calibration target and controls the head-up display to project the virtual calibration target. The collection module 104 collects the virtual calibration target projected by the head-up display and acquires the second coordinates of the virtual calibration target. The judgment module 105 determines whether the virtual calibration target is offset relative to the real calibration target based on the first and second coordinates. The determination module 106 determines that if an offset exists, the image projected by the head-up display has a deviation.
[0097] For a more detailed description of the above-mentioned acquisition module 101, extraction module 102, control module 103, data collection module 104, judgment module 105, and determination module 106, please refer to [the relevant documentation / reference]. Figures 2 to 5 The relevant descriptions in the method embodiments shown are directly obtained and will not be repeated here.
[0098] In this embodiment of the application, the detection device 100 described above detects whether the head-up display has a projection offset, which helps to improve driving safety and reduce usage costs.
[0099] Thirdly, this application provides a head-up display system 200, please refer to... Figures 7 to 9 The head-up display system 200 includes a head-up display 1, a three-dimensional ranging sensing module 6, an image acquisition module 7, at least one processor 8, and a memory 9.
[0100] In some embodiments, the three-dimensional ranging sensing module 6 is used to acquire real-world environmental information from the image projected by the head-up display 1. This real-world environmental information is a point cloud or depth map. The three-dimensional ranging sensing module 6 can be a lidar, millimeter-wave radar, binocular camera, or TOF camera, etc. The three-dimensional ranging sensing module can also be formed by combining multiple sensors, such as an RGB camera combined with lidar, an RGB camera combined with a depth camera, or an RGB camera combined with a binocular camera. In some examples, the three-dimensional ranging sensing module 6 is located on the outside side of the windshield 2.
[0101] In some embodiments, please refer to Figure 9 The head-up display 1 is used to project images. The head-up display 1 includes a control unit 11 and an optical component 12. The control unit 11 is connected to the optical component 12. The control unit 11 is used to control the optical component 12 to generate and output the image light of the virtual calibration target according to the rendered virtual calibration target information. The image light is projected onto the display area 21 of the windshield 2 to form the corresponding virtual calibration target.
[0102] In some embodiments, the optical component 12 includes a light source and an imaging lens group, the light source being used to generate and emit image light, and the imaging lens group being used to project the image onto the display area 21 of the windshield 2.
[0103] In some embodiments, please refer to Figure 8 The image acquisition module 7 is used to acquire images projected by the head-up display 1. The image acquisition module 7 includes a physical camera 71, which is positioned outside the eye-box area 3, such as above the vehicle's dashboard or below the sun visor, reducing the risk of the physical camera 71 interfering with the driver's vision. In some examples, the physical camera 71 is positioned adjacent to the eye-box area 3.
[0104] In some embodiments, the image acquisition module 7 further includes a virtual camera, which corresponds to the position of the eye box. The virtual camera refers to a camera model simulated based on the center position of the eye box and preset parameters, used to simulate the imaging effect observed by the human eye at the center of the eye box. The camera model is stored in the control unit 11 or the memory 9.
[0105] In some embodiments, at least one processor 8 is connected to the three-dimensional ranging module, the image acquisition module 7, and the head-up display 1, respectively.
[0106] In some embodiments, the memory 9 is communicatively connected to at least one processor 8, wherein the memory 9 stores instructions executable by at least one processor 8, the instructions being executed by at least one processor 8 to enable at least one processor 8 to perform the detection method described above.
[0107] Fourthly, embodiments of this application provide a vehicle that includes the head-up display system 200 described above.
[0108] It should be noted that while preferred embodiments of this application are provided in the specification and accompanying drawings, this application can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are not intended to impose additional limitations on the content of this application; their purpose is to provide a more thorough and comprehensive understanding of the disclosure of this application. Furthermore, the above-described technical features can be combined with each other to form various embodiments not listed above, all of which are considered to be within the scope of this application's specification. Moreover, those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. A detection method of a head-up display, characterized by, The method comprises: acquiring real environment information of the head-up display projection image; extracting a real alignment target from the real environment information, and a first coordinate of the real alignment target; generating a virtual alignment target according to the real alignment target, and controlling the head-up display to project the virtual alignment target; capturing the virtual alignment target projected by the head-up display, and acquiring a second coordinate of the virtual alignment target; determining whether the virtual alignment target is offset relative to the real alignment target according to the first coordinate and the second coordinate; if there is an offset, determining that the head-up display projection image has a deviation.
2. The method of claim 1, wherein the step of determining whether the virtual alignment target is offset relative to the real alignment target according to the first coordinate and the second coordinate further comprises: converting the first coordinate of the real alignment target into a first converted coordinate in a preset coordinate system according to a preset first conversion model; converting the second coordinate of the virtual alignment target into a second converted coordinate in the preset coordinate system according to a preset second conversion model; determining whether the virtual alignment target is offset relative to the real alignment target according to the first converted coordinate and the second converted coordinate.
3. The method of claim 2, wherein the virtual alignment target and the real alignment target are both a plurality of matching points, and the step of determining whether the virtual alignment target is offset relative to the real alignment target according to the first converted coordinate and the second converted coordinate further comprises: calculating a distance value of the first converted coordinate and the second converted coordinate of each matching point; calculating an average value of the distance values of all the matching points; determining whether the average value is greater than a preset threshold value; if the average value is greater than the preset threshold value, determining that the virtual alignment target is offset relative to the real alignment target; otherwise, determining that the virtual alignment target is not offset relative to the real alignment target.
4. The method of claim 2, wherein the virtual alignment target and the real alignment target are both a matching line, and the step of determining whether the virtual alignment target is offset relative to the real alignment target according to the first converted coordinate and the second converted coordinate further comprises: calculating a perpendicular distance from each point of the real alignment target to the matching line of the virtual alignment target; calculating an average value of the perpendicular distances from all the points of the real alignment target to the matching line of the virtual alignment target; determining whether the average value is greater than a preset threshold value; if the average value is greater than the preset threshold value, determining that the virtual alignment target is offset relative to the real alignment target; otherwise, determining that the virtual alignment target is not offset relative to the real alignment target. The method further comprises: if it is determined that the head-up display projection image has a deviation, determining an offset amount according to a difference between the average value and the preset threshold value. The method further comprises: if it is determined that the head-up display projection image has a deviation, issuing an alarm signal. The alarm signal is at least one of a projected virtual image signal, a sound signal, and a light signal.
5. The method according to claim 3 or 4, characterized in that , The method comprises: 6. The method of claim 1, wherein , 7. The method of claim 6, wherein , 8. A detection device applied to a head-up display system, characterized in that, An acquisition module is configured to acquire a real environment in which the head-up display projects an image. An extraction module is configured to extract a real alignment target from the real environment, and a first coordinate of the real alignment target. A control module is configured to generate a virtual alignment target according to the real alignment target, and control the head-up display to project the virtual alignment target. A collection module is configured to collect the virtual alignment target projected by the head-up display, and acquire a second coordinate of the virtual alignment target. A judgment module is configured to determine whether the virtual alignment target is offset relative to the real alignment target according to the first coordinate and the second coordinate. A determination module is configured to determine that the head-up display projects an image with deviation if there is an offset.
9. A heads-up display system characterized by, The head-up display system comprises: a head-up display configured to project an image; a three-dimensional distance measurement sensor module configured to acquire real environment information of the head-up display projecting the image; an image collection module configured to collect the image projected by the head-up display; at least one processor connected with the three-dimensional distance measurement sensor module, the image collection module and the head-up display respectively; a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A vehicle characterized by comprising: The head-up display system comprises the head-up display system of claim 9.