Data display method and device based on ARHUD, equipment and medium

By calculating the cumulative delay of the target signal in the transmission link and the vehicle and target object status parameters, dynamically adjusting the target object information on ARHUD, solving the problem of inaccurate target object position information on ARHUD, and improving the accuracy of display and Real-time enhances driving safety.

CN119087675AActive Publication Date: 2024-12-06CHONGQING CHANGAN AUTOMOBILE CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411186004.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

In the prior art, the target object position information displayed on ARHUD is inaccurate, mainly due to the low accuracy of signal transmission delay information collection, which makes it difficult for the target object information to remain consistent with the real world in the environment of rapid changes in the vehicle.

Method used

By calculating the cumulative delay of the target signal in the transmission link, and combining the state parameters of the vehicle and the target object, the target state parameters of the target object relative to the vehicle are calculated, and the target object information display on the ARHUD is dynamically adjusted to improve the display accuracy .

Benefits of technology

It effectively improves the accuracy and real-time display of target information on ARHUD, enhances driving safety and user experience, and ensures that target information always matches the actual environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119087675A_ABST
    Figure CN119087675A_ABST
Patent Text Reader

Abstract

The invention relates to an ARHUD-based data display method, device, equipment and medium, and the method comprises the steps: when a target signal is transmitted in a target transmission link, calculating the cumulative time delay of the target signal transmitted to each module, the target signal being obtained through the signal collection of a target object by a collection module, and the cumulative time delay being the cumulative time delay of the target signal transmitted to each module; the target transmission link is a link from the acquisition module to the ARHUD. If the accumulated time delay of the target signal in any module exceeds the preset time delay, a first state parameter of the vehicle is calculated according to the target time delay, the driving data of the vehicle and the wheelbase, and the state parameter comprises a position change parameter and a course angle change parameter. Acquiring a second state parameter of the target object, and calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter. And displaying the target object in the ARHUD according to the target state parameter. According to the invention, the accuracy of displaying the target object information in the ARHUD is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a data display method, device, equipment and medium based on ARHUD. Background Art

[0002] Augmented Reality Head Up Display (ARHUD) is a device that combines augmented reality technology with head-up display. In the in-vehicle display system, it can help the driver to observe the road environment ahead while also quickly observing the functional status of navigation, warning, intelligent driving, etc., reducing the frequency of the driver looking down at the instrument and increasing driving safety. However, due to changes in the vehicle itself and the external environment information, it is easy to cause the target objects displayed on the ARHUD to be inaccurate. Therefore, in order to ensure that the target object information displayed on the ARHUD is consistent with the real world, it is very important to dynamically adjust the display information of the target objects on the ARHUD.

[0003] At present, the method for adjusting the target object information displayed on the ARHUD is mainly to adjust the delay of the motion trajectory information collected by the image collector. The specific process is firstly to simultaneously collect the actual motion trajectory of the flat plate with a fixed-pitch pattern array (i.e., the calibration plate) and the framing screen of the image collector loaded on the vehicle itself through the terminal device, and then process the actual motion trajectory of the calibration plate and the framing screen to obtain the absolute delay of the image collector, and then make a prediction based on the absolute delay of the image collector to adjust the target object information displayed on the ARHUD.

[0004] However, the data collected by the prior art for transmission delay information is single and of low accuracy, and there is still the problem of inaccurate position information of the target object displayed on the ARHUD. Summary of the invention

[0005] The purpose of the present invention is to provide a data display method based on ARHUD to solve the problem that the accuracy of signal transmission delay information collection in the prior art is low and the position information of the target object displayed on the ARHUD is inaccurate.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A data display method based on ARHUD, comprising:

[0008] When a target signal is transmitted in a target transmission link, the cumulative delay of the target signal to be transmitted to each module is calculated, wherein the target signal is obtained by collecting signals of a target object by an acquisition module, and the target transmission link is a link from the acquisition module to the ARHUD;

[0009] If the accumulated delay of the target signal in any module exceeds the preset delay, the first state parameter of the vehicle is calculated according to the accumulated delay, the driving data of the vehicle and the wheelbase, and the state parameter includes a position change parameter and a heading angle change parameter;

[0010] Acquiring a second state parameter of the target object;

[0011] Calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter;

[0012] The target object is displayed in the ARHUD according to the target state parameters.

[0013] According to the above technical means, the target state parameters of the target object relative to the vehicle are calculated by combining the vehicle's state parameters and the target object's state parameters, and finally the target object information displayed on the ARHUD is dynamically adjusted. Since the vehicle's state parameters and the target object's state parameters are considered at the same time, the signal transmission delay information obtained is more comprehensive and accurate, and the position information of the target object displayed on the ARHUD can be effectively adjusted according to the current environment.

[0014] Further, the calculating the cumulative delay of the target signal being transmitted to each module includes:

[0015] For each module in the target transmission link, the accumulated delay of the target signal in the module is calculated according to the accumulated delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0016] According to the above technical means, by accurately calculating the accumulated delay of the target signal in the module based on the accumulated delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay, the delay of the entire target signal in the transmission link can be effectively managed and controlled to avoid display lags or errors caused by delay accumulation.

[0017] Furthermore, the method further comprises:

[0018] The processing delay of the target signal in the module is calculated according to the signal input timestamp and the signal output timestamp of the target signal in the module.

[0019] Further, for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0020] If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0021] According to the above technical means, by setting the first preset delay and the second preset delay to compensate and adjust the module that cannot calculate the processing delay, the delay of the entire signal transmission link can be effectively managed and controlled to avoid display lag or error caused by delay accumulation.

[0022] Further, the obtaining of the second state parameter of the target object includes:

[0023] If the target object is a static target object, collecting the second state parameter of the target object;

[0024] If the target object is a dynamic target object, the absolute speed, absolute acceleration, steering angular speed and steering angular acceleration of the target object are collected;

[0025] The second state parameter of the target object is calculated according to the accumulated delay, the absolute speed of the target object, the absolute acceleration, the steering angular speed, and the steering angular acceleration.

[0026] According to the above technical means, for static targets, it is only necessary to collect their second state parameters; for dynamic targets, it is necessary to collect and calculate their second state parameters, especially in the presence of delays, to perform accurate state prediction and compensation, which can significantly improve the accuracy and real-time display of the target objects in the ARHUD system, thereby enhancing driving safety and user experience.

[0027] Further, calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter includes:

[0028] According to the first state parameter and the second state parameter, a coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

[0029] According to the above technical means, the positions of dynamic and static targets can be accurately converted to the new vehicle coordinate system, ensuring that the target information displayed on the ARHUD is still accurate after the vehicle moves and rotates, thus improving the reliability and real-time performance of the system.

[0030] Further, after calculating the target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter, the method further includes:

[0031] According to the map positioning information, obtaining the position of the vehicle and the discrete slope information of the area in front of the vehicle;

[0032] Determining a first height of the target object relative to the ground according to the position of the vehicle and discrete slope information of an area in front of the vehicle;

[0033] Determining a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter;

[0034] Correspondingly, displaying the target object in the ARHUD according to the target state parameter includes:

[0035] The target object is displayed in the ARHUD according to the target state parameter and the second height.

[0036] According to the above technical means, by integrating road slope information and vehicle pitch angle, the system can accurately correct the display position of the target object in ARHUD and reduce the visual errors caused by terrain changes or vehicle posture changes.

[0037] Furthermore, the method further comprises:

[0038] Clear the accumulated delay of the module.

[0039] According to the above technical means, it is ensured that the subsequent transmission of new target signals is not affected by historical delays.

[0040] Furthermore, the method further comprises:

[0041] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0042] Calculating the position difference between the driver's observation point and the observation origin according to the coordinates of the observation point and the coordinates of a pre-calibrated observation origin;

[0043] The target state parameter is updated according to the position difference to generate an updated target state parameter.

[0044] According to the above-mentioned technical means, the target state parameters can be dynamically adjusted. ARHUD can ensure that the target object information always remains within the driver's field of view, regardless of how the position of his head or eyes changes. This not only improves the adaptability and comfort of the system, but also enhances the driver's interactive experience with the ARHUD system.

[0045] Furthermore, the method further comprises:

[0046] When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in the ARHUD.

[0047] The above-mentioned technical means can help drivers better understand the status of the vehicle and its auxiliary functions during driving, thereby improving driving safety, enhancing perception of the surrounding environment, and providing a better driving experience.

[0048] A data display device based on ARHUD, applied to a vehicle, comprising:

[0049] A first calculation module is used to calculate the cumulative delay of the target signal to each module when the target signal is transmitted in the target transmission link, the target signal is obtained by collecting signals of the target object by the collection module, and the target transmission link is a link from the collection module to the ARHUD;

[0050] A second calculation module is used to calculate the first state parameter of the vehicle according to the target delay, the driving data of the vehicle and the wheelbase if the cumulative delay of the target signal in any module exceeds the preset delay, the state parameter includes a position change parameter and a heading angle change parameter;

[0051] An acquisition module, used for acquiring a second state parameter of the target object;

[0052] A third calculation module: used for calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter;

[0053] The display module is used to display the target object in the ARHUD according to the target state parameters.

[0054] Furthermore, the first calculation module is specifically used to:

[0055] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0056] Furthermore, the first computing module is further used for:

[0057] The processing delay of the target signal in the module is calculated according to the signal input timestamp and signal output timestamp of the target signal in the module.

[0058] Furthermore, the first computing module is further used for:

[0059] For each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0060] If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0061] Furthermore, the acquisition module is specifically used for:

[0062] If the target object is a static target object, collecting a second state parameter of the target object;

[0063] If the target is a dynamic target, the absolute speed, absolute acceleration, steering angular speed and steering angular acceleration of the target are collected;

[0064] A second state parameter of the target object is calculated according to the accumulated delay, the absolute speed, the absolute acceleration, the steering angular speed, and the steering angular acceleration of the target object.

[0065] Furthermore, the third computing module is specifically used for:

[0066] According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

[0067] Furthermore, the ARHUD-based data display device 140 further includes a processing device, which, after calculating the target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter, is further used to:

[0068] According to the map positioning information, the discrete slope information of the vehicle position and the area in front of the vehicle is obtained;

[0069] Determining a first height of the target object relative to the ground according to the position of the vehicle and discrete slope information of the area in front of the vehicle;

[0070] Determining a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter;

[0071] Accordingly, the display module is specifically used for:

[0072] The target object is displayed in the ARHUD according to the target state parameters and the second altitude.

[0073] Furthermore, the processing module is also used to clear the accumulated delay of the module.

[0074] Furthermore, the processing module is also used for:

[0075] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0076] According to the coordinates of the observation point and the coordinates of the pre-calibrated observation origin, the position difference between the driver's observation point and the observation origin is calculated;

[0077] The target state parameters are updated according to the position difference to generate updated target state parameters.

[0078] Furthermore, the display module is also used for:

[0079] When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in ARHUD.

[0080] A vehicle comprises: a processor, a memory and computer-executable instructions stored in the memory and executable on the processor, wherein the processor is used to implement the above-mentioned ARHUD-based data display method when executing the computer-executable instructions.

[0081] A computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned ARHUD-based data display method when executed by a processor.

[0082] A computer program product includes a computer program, and when the computer program is executed by a processor, it is used to implement the above-mentioned ARHUD-based data display method.

[0083] Beneficial effects of the present invention:

[0084] (1) When the target signal is transmitted in the target transmission link, the cumulative delay generated in each module is related to the accuracy of the target object's display content on the ARHUD, and under the cumulative delay, the vehicle's own motion trajectory will also affect the accuracy of the target object's display content on the ARHUD. Therefore, this technical solution calculates the cumulative delay of the target signal exceeding the preset delay in any module, and calculates the state parameters of the vehicle and the target object on this basis. Since the target state parameters finally formed take into account the changes in data information of multiple modules, the accuracy of the target object's display content on the ARHUD is effectively improved.

[0085] (2) By considering the state parameters of the vehicle and the target object under cumulative delay, the relative error between the vehicle and the target object under different driving conditions is effectively eliminated. In addition, the acquisition module monitors the driver's eye position in real time to ensure that the target object information displayed on the ARHUD always remains within the driver's field of view. This not only improves the system's adaptability and comfort, but also enhances the driver's interactive experience with the ARHUD system. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1A schematic diagram of the structure of the ARHUD system provided by an embodiment of the present invention;

[0087] Figure 2 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ;

[0088] Figure 3 A diagram of a cumulative delay transmission system of a target transmission link provided by an embodiment of the present invention;

[0089] Figure 4 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;

[0090] Figure 5 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;

[0091] Figure 6 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;

[0092] Figure 7 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;

[0093] Figure 8 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ;

[0094] Fig. 9 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;

[0095] Fig.10 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;

[0096] Fig.11 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;

[0097] Fig.12 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;

[0098] Fig.13 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 6 ;

[0099] Fig.14 A schematic diagram of the structure of a data display device based on ARHUD provided in an embodiment of the present invention;

[0100] Fig.15 A schematic structural diagram of a vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0101] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0102] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and corresponding operation entrances shall be provided for users to choose to authorize or refuse.

[0103] Before introducing the present invention, the application background of the present invention is first explained.

[0104] With the rapid development of science and technology, the application of ARHUD in vehicle display systems is becoming more and more widespread. ARHUD can help drivers observe the road environment ahead while also quickly observing the status of navigation, warning, intelligent driving and other functions, helping drivers to focus more on the road ahead, reducing the frequency of drivers looking down at the instrument, and increasing driving safety.

[0105] However, in actual applications, due to the rapid driving of the vehicle, environmental information will change rapidly when passing through ramps, curves, or turning, changing lanes, turning around, etc. At the same time, when the relative speed of the vehicle itself and the target object is greatly different, the relative position of the vehicle itself and various targets will change rapidly. These changes will make it difficult to match the content displayed by ARHUD with the real scene due to signal transmission delays. In particular, due to cost and other reasons, the scene fusion function of ARHUD in many vehicles is not based on dedicated sensors, but relies on sensor information such as cameras used by intelligent driving functions or fusion target information, and the signal processing and transmission links are lengthy. At this time, the accumulated delays of various sensors and processors are large, and the target positions, lane lines, etc. marked on the ARHUD will be more inaccurate.

[0106] At present, the method for adjusting the target information displayed on the ARHUD is mainly to adjust the target information displayed on the ARHUD through the absolute delay of the image collector. The specific process is firstly to simultaneously collect the actual motion trajectory of the flat plate with a fixed-pitch pattern array (i.e., the calibration plate) and the framing screen of the image collector mounted on the vehicle itself through the terminal device, and establish the relationship between the actual motion trajectory of the target and the image acquisition; then, the image processing technology is used to simultaneously extract the actual motion trajectory of the calibration plate and the target motion trajectory of the calibration plate in the framing screen; then, on the same time axis, the time difference between the actual motion trajectory and the target motion trajectory is used as the absolute delay of the image collector; finally, the calculated absolute delay is used to predict the motion trajectory captured by the image collector, and the motion state of the target object within the delay time is obtained, and the display content on the ARHUD is adjusted accordingly to ensure that the displayed target information is synchronized with the actual environment.

[0107] However, since the performance of the image collector may be affected by environmental factors such as light and weather, errors in delay calculation will occur; and the absolute delay measurement based on static conditions cannot maintain accuracy when the environment and system parameters change. In particular, when the delay has dynamic characteristics, static measurement will lead to inaccurate displayed information. On the other hand, when the corresponding hardware or algorithm in the vehicle changes, the entire process needs to be recalibrated, which increases the complexity and time cost of system maintenance.

[0108] In summary, when the prior art collects and processes transmission delay information, there is a problem that the collected data is single and has low accuracy, which leads to inaccurate position information of the target object displayed on the ARHUD.

[0109] Based on the above technical problems, the present invention provides a data display method based on ARHUD. Considering that the cumulative delays of signals in different acquisition modules are inconsistent, when the target signal is transmitted from the acquisition module to the ARHUD, the cumulative delay of the target signal to each module is calculated, and when the cumulative delay of the target signal in any module exceeds the preset delay, considering that the surrounding environment and the vehicle itself will change rapidly, the first state parameter of the vehicle and the second state parameter of the target object are obtained respectively, which can effectively compensate for the display error caused by the delay, and improve the accuracy and reliability of the ARHUD display information. Finally, the target state parameters of the target object relative to the vehicle are calculated and displayed according to the first state parameter and the second state parameter to realize the dynamic adjustment of the target object information displayed on the ARHUD, thereby solving the problem that the transmission delay information data is single due to the use of only the image sensor to measure the delay, and the position information of the target object displayed on the ARHUD is inaccurate due to the inability to measure the dynamic target object.

[0110] The technical solution of the present invention is described in detail below through specific embodiments.

[0111] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0112] Figure 1 A schematic diagram of the structure of the ARHUD system provided in an embodiment of the present invention; Figure 1 As shown, the ARHUD system includes an environmental sensing module, a sensor information fusion module, a vehicle status information module, an eye tracking sensor module, an ARHUD computing module, an ARHUD rendering module and an ARHUD display screen.

[0113] Among them, first, the road and the target are collected and detected by the environmental sensing module (such as camera, millimeter wave radar, ultrasonic radar, laser radar, etc.) to form sensor information, and the sensor information fusion module fuses the sensor information; then, the vehicle status information module is used to collect the vehicle's (that is, the vehicle's) orientation, posture, steering, speed, wheelbase, steering transmission ratio and other information; then, the eye tracking sensor module (for example, driver monitoring system) is used to detect the driver's observation point; then, the information of the above-mentioned sensor information fusion module, vehicle status information module and eye tracking sensor module is input into the ARHUD calculation module for summary calculation; finally, all the information is input into the ARHUD rendering module for rendering, and displayed on the ARHUD display screen, and finally the driver can see the display content and the real scene of the outside world through the ARHUD display screen.

[0114] It should be noted that, since the present invention takes into account the changes in the state parameters of the target object and the vehicle under the cumulative delay of the target signal, the ARHUD calculation module also includes a delay compensation module, a slope compensation module, a function labeling calculation module, a driver observation point compensation module, and a distortion calibration and correction calculation module. The slope compensation module includes map positioning information, such as slope data of a discrete distance in front of the vehicle, slope data at 0m, slope data at 5m, etc. The function labeling calculation module contains labeling information, such as dangerous targets, line crossing prompts, navigation arrows and other information, with the aim of providing the driver with guidance or warning information that fits the real scene. On the other hand, the ARHUD system may further include an ARHUD mechanical adjustment device to assist in adjusting the reflection angle or image projection angle of the ARHUD display screen to increase the adjustment range of its visual display.

[0115] Figure 2 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ,like Figure 2 As shown, the data display method based on ARHUD may include the following steps:

[0116] S21. When the target signal is transmitted in the target transmission link, the accumulated delay of the target signal being transmitted to each module is calculated.

[0117] Among them, the target signal is obtained by collecting signals from the target object through the acquisition module (i.e., environmental sensing module, such as camera, millimeter-wave radar, ultrasonic radar, lidar, etc.). The target transmission link is the link from the acquisition module to the ARHUD. The target object refers to the object or feature that needs to be displayed in the ARHUD system, such as pedestrians, road signs and signals, passing vehicles, and obstacles.

[0118] It can be understood that when the target object information is collected by the environmental sensing module and then converted into a target signal and transmitted in the target transmission link, it will pass through multiple modules (such as the sensor information fusion module, etc.), and because different acquisition modules may have different processing speeds and delays, by calculating the cumulative delay of each module, these delay differences can be identified and processed to ensure that the data of all modules can be aligned in time, thereby ultimately ensuring that consistent display information is provided on the ARHUD system, thereby improving the accuracy of the target object information display.

[0119] S22: If the accumulated delay of the target signal in any module exceeds the preset delay, the first state parameter of the vehicle is calculated according to the accumulated delay, the driving data and the wheelbase of the vehicle.

[0120] The state parameters include position change parameters and heading angle change parameters.

[0121] It should be understood that since the vehicle is driving at a high speed, if the cumulative delay of the target signal in any module exceeds the preset delay, it means that the accuracy of the target object displayed in ARHUD is low at this time, and the position of the target object displayed in ARHUD needs to be updated. In order to ensure that the position of the target object displayed in ARHUD is consistent with the actual scene, it is necessary to calculate the state change of the vehicle and the target object within the cumulative delay respectively, and adjust the display position of the target object in ARHUD according to the state change of the vehicle and the target object. First, it is necessary to obtain the state change of the vehicle within the cumulative delay, that is, the first state parameter.

[0122] Exemplarily, first use the vehicle's driving data recorded in the vehicle status information module, including the steering wheel angle δ, vehicle speed v, and steering gear ratio R, and then obtain the cumulative delay Δt of the target signal in the module and the vehicle wheelbase L; then, for the convenience of calculation, take the center of the vehicle's front as the origin, the positive direction of the x-axis is along the vehicle body, the positive direction of the y-axis is perpendicular to the x-axis toward the left side of the vehicle body, and the positive direction of the z-axis is perpendicular to the xOy plane upward. The steering wheel angle and heading angle are both counterclockwise as positive directions; finally, calculate the front wheel angle, instantaneous steering radius, instantaneous steering angular velocity, and the heading angle of the vehicle within the cumulative delay time period Δt. The specific calculation process is:

[0123] 1) Calculate the front wheel angle: Calculate the front wheel angle θ through the steering wheel angle and steering gear ratio. The calculation formula is:

[0124]

[0125] Where θ is the front wheel angle; δ is the steering wheel angle; and R is the vehicle steering ratio.

[0126] 2) Calculate the instantaneous turning radius: Calculate the instantaneous turning radius R by the front wheel turning angle and vehicle wheelbase t The calculation formula is:

[0127]

[0128] Among them, R t is the instantaneous turning radius; L is the vehicle wheelbase; θ is the front wheel turning angle.

[0129] 3) Calculate the instantaneous steering angular velocity: Calculate the instantaneous steering angular velocity ω by the vehicle speed and the instantaneous steering radius. The calculation formula is:

[0130]

[0131] Where, ω is the instantaneous steering angular velocity; v is the vehicle speed; R t is the instantaneous turning radius.

[0132] 4) Calculate the change in the vehicle's heading angle within the cumulative delay period Δt, and the calculation formula is:

[0133] Δφ=ω×Δt

[0134] Among them, Δφ is the change in the vehicle's heading angle within Δt; ω is the instantaneous steering angular velocity; and Δt is the accumulated delay.

[0135] It should be noted that when the steering wheel angle δ is 0 or its absolute value is less than a very small value, such as 10 -6 , and the turning radius R t Infinite, the steering angular velocity ω is zero, and the vehicle's motion can be simplified to linear motion. The position change of the vehicle at this time is:

[0136] Position change in the X direction: Δx = v × Δt

[0137] Among them, v is the vehicle speed; Δt is the cumulative delay, and Δx is the distance the vehicle travels forward.

[0138] Position change in Y direction: Δy = 0

[0139] Wherein, Δy is the distance perpendicular to the x-axis toward the left side of the vehicle body.

[0140] The steering wheel angle δ is not 0 or its absolute value is greater than a very small value, such as 10 -6 , the vehicle moves along an arc, and the position change of the vehicle at this time is:

[0141] Position change in the X direction: Δx = R t ×sin(Δφ)

[0142] Where Δx is the distance the vehicle travels forward; R t is the instantaneous turning radius; Δφ is the change in the vehicle’s heading angle within Δt.

[0143] Position change in the Y direction: Δy = R t ×(1-cos(Δφ))

[0144] It is understandable that if the cumulative delay of the target signal in any module exceeds the preset delay, it means that the cumulative delay may subsequently affect the display accuracy of the target object in ARHUD. Therefore, it is necessary to calculate the first state parameters of the vehicle based on the cumulative delay, the vehicle's driving data and the wheelbase, so that the target object can be subsequently converted to the vehicle's coordinate system and the display position of the target object can be updated, thereby improving the display accuracy.

[0145] S23. Obtain a second state parameter of the target object.

[0146] Among them, targets can be divided into static targets and dynamic targets according to their state relative to the ground. Static targets can include lane lines, curbs, etc., while dynamic targets can include vehicles, people, animals, etc.

[0147] In a possible implementation, for targets in different states, their second state parameters are also different, specifically:

[0148] If the target object is a static target object, the second state parameter of the target object is determined to be 0; if the target object is a dynamic target object, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object are collected, and the second state parameter of the target object is calculated respectively by uniformly accelerated motion and uniformly accelerated steering according to the accumulated delay, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object.

[0149] It should be noted that, since the static target is stationary relative to the ground, there is no position change, so the second state parameter of the target is determined to be 0; while for the dynamic target, since it is dynamic relative to the ground, it needs to compensate for its own position change, so it is necessary to further calculate the second state parameter of the target.

[0150] Optionally, for dynamic targets, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the dynamic target from the sensor information fusion module can be used, and the position change parameters and heading angle change parameters of the dynamic target can be calculated based on the accumulated delay. In addition, if there are input quantities that cannot be obtained, the calculation is simplified by treating them as zero.

[0151] In this way, the foundation is laid for the subsequent coordinate transformation of the target object, which can significantly improve the accuracy and real-time display of the target object in the ARHUD system, thereby enhancing driving safety and user experience.

[0152] S24. Calculate a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter.

[0153] In a possible implementation, the coordinate system of the second state parameter may be converted into a coordinate system corresponding to the vehicle according to the first state parameter and the second state parameter, so as to generate a target state parameter of the target object relative to the vehicle.

[0154] Since the first state parameters (Δx, Δy, Δφ) of the vehicle are obtained in S24, the coordinate system is transformed in such a way that the coordinate system of the target object on the vehicle is translated along the vector (Δx, Δy, 0) and rotated counterclockwise around the z-axis by Δφ.

[0155] It can be understood that, by obtaining the first state parameter of the vehicle and the second state parameter corresponding to the target object through the above steps, the coordinate system of the target object can be converted to the coordinate system of the vehicle. This method can accurately display the real-time position and dynamic changes of the target object on the ARHUD by accurately calculating the target state parameters of the target object relative to the vehicle, so that the driver can intuitively understand the position and movement trend of the target object.

[0156] S25. Display the target object in the ARHUD according to the target state parameters.

[0157] In actual applications, after determining the target state parameters, the target object can be moved based on the current position of the target object according to the target state parameters, the target position of the target object in the ARHUD is determined, and the target object is updated to the target position in the ARHUD.

[0158] It can be understood that the target object is displayed in ARHUD according to the target state parameters, ensuring that regardless of whether the vehicle is in a straight line or circular motion, whether the target object is static or dynamic, ARHUD can provide accurate target object content and enhance driving safety and user experience.

[0159] The data display method based on ARHUD provided by the embodiment of the present invention calculates the cumulative delay of the target signal transmitted to each module when the target signal is transmitted in the target transmission link; then, if the cumulative delay of the target signal in any module exceeds the preset delay, the first state parameter of the vehicle is calculated according to the target delay, the driving data of the vehicle and the wheelbase; then, the second state parameter of the target object is obtained; then, according to the first state parameter and the second state parameter, the target state parameter of the target object relative to the vehicle is calculated; finally, according to the target state parameter, the target object is displayed in ARHUD. The display error caused by the delay can be effectively compensated to realize the dynamic adjustment of the target object information displayed on the ARHUD. While improving the accuracy of the target object information displayed by the ARHUD, it also enhances the system's adaptability to complex driving environments, providing users with a safer and more reliable driving experience.

[0160] Furthermore, in S21, the specific process of calculating the accumulated delay of the target signal transmitted to each module may be:

[0161] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0162] In a possible implementation, since some modules in the system may not be able to calculate their processing delays, it is necessary to use preset delay values ​​for compensation and adjustment, specifically including: for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, then the first preset delay corresponding to the module is a positive number greater than 0; if the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, then the second preset delay corresponding to the module is a positive number greater than 0.

[0163] For example, assume that there is a target transmission link consisting of the following five modules: module A, module B, module C, module D, and module E. In this link, module B cannot calculate the processing delay of the target signal inside it, module D is the second to last module, and module E cannot calculate the processing delay of the target signal inside it.

[0164] Then, since module B cannot calculate the processing delay of the target signal, a first preset delay must be set for module B, for example, 10ms; and, since module D is the second-to-last module and module E cannot calculate the processing delay of the target signal within it, a second preset delay must also be set for module D, for example, 15ms, to avoid the accumulation of system errors caused by unmeasurable delays.

[0165] Optionally, the processing delay of the target signal in the module can be calculated as follows:

[0166] The processing delay of the target signal in the module is calculated according to the signal input timestamp and signal output timestamp of the target signal in the module.

[0167] It should be understood that the clock inside the module will record the signal input timestamp when the signal arrives, and correspondingly, it will record the signal output timestamp when the signal is output. By calculating the difference between the signal input timestamp and the signal output timestamp, the processing delay of the target signal in the module can be obtained.

[0168] Optionally, in order to ensure that the subsequent transmission of a new target signal is not affected by the historical delay, after calculating the accumulated delay of the target signal in the module, the accumulated delay of the module needs to be cleared.

[0169] Specifically, the cumulative delay calculation formula of the target transmission link is:

[0170] T d =T d1 +(T o -T i )+T DB +T DA

[0171] Among them, Td is the cumulative delay of the target signal in the current module; T d1 is the cumulative delay of the target signal in the previous module; T o Output timestamp for the signal; T i Enter the timestamp for the signal; T DB is the first preset delay; T DA This is the second preset delay.

[0172] It should be noted that, when the transmission link has transmission performance test data, the average of the transmission delays is taken as the cumulative delay of the transmission link; when there is no performance test data for the transmission link, if there are performance index requirements, half of the maximum delay performance requirement can be taken as the cumulative delay; and when there is performance test data for a longer transmission link that includes the transmission link:

[0173] In each transmission process, the test delay of the longer transmission link and the test delay of the calculable part of the longer transmission link are obtained. After that, the test delays of the calculable part of the link in multiple transmission processes are accumulated to obtain the accumulated test delay, and the delay difference between the maximum test delay and the minimum test delay of the longer transmission link in each transmission process is calculated.

[0174] 1) If the ratio of the accumulated test delay to the delay difference is greater than the set ratio, the difference between the test delay of the longer transmission link and the calculable test delay is calculated during each transmission process, and the average of the difference is taken as the accumulated delay of the transmission link;

[0175] 2) If the ratio of the accumulated test delay to the delay difference is less than or equal to the set ratio, the average of the test delays of the longer transmission link is used as the accumulated delay of the transmission link.

[0176] Among them, the first preset delay is generally performed in the ARHUD calculation module to integrate all information for overall calculation. Of course, due to considerations such as computing power, memory resources, and computing convenience, some or all of the calculations can be performed in other modules.

[0177] It can be understood that for each module, by accurately calculating the cumulative delay of the target signal in the module based on the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay, the delay of the entire target signal in the transmission link can be effectively managed and controlled to avoid display lags or errors caused by delay accumulation.

[0178] Figure 3 The cumulative delay transmission system diagram of the target transmission link provided by the embodiment of the present invention. Figure 3As shown, the cumulative delay of signal transmission to each module is achieved by using a system with a unified clock. The system can create a signal package for the signal, which includes four attributes: the cumulative delay of the target signal in the previous module, the signal input timestamp, the signal output timestamp, and the cumulative delay. However, if the signal itself is a set of data values ​​that are input and output at the same time, then the group of data also has only one signal input timestamp, signal output timestamp, and cumulative delay.

[0179] When a signal is input to the module, the signal input timestamp is updated first, and the signal output timestamp is updated when the signal outputs the module, and the cumulative delay is updated. When the ARHUD calculation module receives information from the sensor information fusion module, the vehicle status information module, and the eye tracking sensor module, the dynamic real scene has changed when the ARHUD is rendered because of the transmission and calculation processing delays of each link. If the delay of the ARHUD calculation module and the rendering module is large, and the real scene changes quickly, the dynamic real scene changes significantly during its processing. At this time, the impact of the delay cannot be ignored. Therefore, the second preset delay must be used for appropriate compensation to eliminate or reduce the impact of the delay.

[0180] Figure 4 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;like Figure 4 As shown, the specific steps of the compensation process for the processing delay in the cumulative delay in the present invention are as follows:

[0181] Step 41: Determine whether the current module performs delay compensation.

[0182] If yes, execute step 42; if no, end.

[0183] It should be understood that delay compensation refers to updating the position of the target in the ARHUD when the cumulative delay exceeds the preset delay. Delay compensation is generally performed in the ARHUD calculation module to integrate all information for overall calculation. Of course, due to considerations such as computing power, memory resources, and computing convenience, some or all of the calculations can be performed in other modules.

[0184] Step 42: Determine the accumulated delay of the target signal.

[0185] Step 43: When the accumulated delay exceeds the preset delay, determine the target state parameters of the target object relative to the vehicle.

[0186] Step 44: Display the target object in the ARHUD according to the target state parameters.

[0187] Step 45: Clear the accumulated delay of the current module.

[0188] Step 46: Input the accumulated delay of the current module to the next module.

[0189] It should be understood that the specific implementation methods in each step are already described in Figure 3 The corresponding embodiments are described in detail, and the embodiments of the present application will not be described in detail here.

[0190] Figure 5 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;like Figure 5 As shown, the specific steps of the process of calculating the position change of the dynamic target in the present invention are as follows:

[0191] Step 51: Determine whether the accumulated delay of the target signal in the current module exceeds the preset delay.

[0192] If yes, execute step 52; if no, execute step 54.

[0193] Step 52: Determine the target state parameters of the target object relative to the vehicle.

[0194] Step 53: Determine the target position of the target object in the ARHUD according to the target state parameters of the target object relative to the vehicle.

[0195] Step 54: Output the target position of the target object.

[0196] It should be understood that if the cumulative delay of the target signal in the current module does not exceed the preset delay, the target position of the target object can be directly determined by the target signal value.

[0197] Figure 6 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;like Figure 6 As shown, the specific steps of the process of calculating the target state parameters of the target object relative to the vehicle in the present invention are as follows:

[0198] Step 61: Determine whether the accumulated delay of the target signal in the current module exceeds the preset delay.

[0199] If yes, execute step 62 ; if no, execute step 54 .

[0200] Step 62: Calculate the first state parameter of the vehicle according to the accumulated delay, the driving data of the vehicle and the wheelbase.

[0201] Step 63: If the target object is a static target object, the second state parameter of the target object is collected; if the target object is a dynamic target object, the absolute speed, absolute acceleration, steering angular speed and steering angular acceleration of the target object are collected, and the second state parameter of the target object is calculated according to the accumulated delay.

[0202] Step 64: According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and the target state parameter of the target object relative to the vehicle is output.

[0203] Furthermore, since the sensor information fusion module can only output the state information of the target object, but cannot output or accurately output the height information of the target object, this will also cause the display content of the target object on the ARHUD to be inconsistent with the actual scene, so the relative height of the target object needs to be calculated.

[0204] Figure 7 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;like Figure 7 As shown, the ARHUD-based data display method may also include:

[0205] S71. Obtain the position of the vehicle and discrete slope information of the area in front of the vehicle according to the map positioning information.

[0206] Among them, map positioning information can use high-precision map data or be obtained in combination with the global positioning system of the vehicle body; discrete slope information refers to a series of discontinuous slope values.

[0207] It can be understood that obtaining discrete slope information of the road ahead of the vehicle through high-precision maps and the vehicle body's global positioning system can enable the system to perceive road changes in advance so as to make necessary visual corrections later.

[0208] S72: Determine a first height of the target object relative to the ground according to the position of the vehicle and discrete slope information of the area in front of the vehicle.

[0209] It is understandable that if the target object is located in the slope area in front of the vehicle, its height relative to the ground will be affected by the slope. By using the discrete slope information and the position information of the vehicle, the first height of the target object relative to the ground can be determined. This helps the system understand the relative position of the target object in the terrain change and lays the foundation for subsequent display correction.

[0210] S73. Determine a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter.

[0211] Among them, the pitch angle of the vehicle can be obtained through the vehicle status information module, that is, the pitch angle of the vehicle (that is, the angle of the vehicle's front and rear tilt) is measured in real time through vehicle-mounted sensors (such as gyroscopes, accelerometers, etc.).

[0212] It can be understood that by considering the pitch angle of the vehicle, the display deviation caused by the change of the vehicle posture can be compensated. On the other hand, by combining the pitch angle of the vehicle, the first height of the target object, and the target state parameters, the second height of the target object relative to the vehicle is calculated, which can ensure that the position of the target object displayed in the ARHUD is consistent with its actual position relative to the vehicle, thereby improving the driver's perception accuracy.

[0213] It should be noted that, in order to facilitate understanding of the calculation process of the first height of the target object relative to the ground and the second height of the target object relative to the vehicle, the present invention Fig. 9 , Fig.10 A specific scenario diagram is used for illustration, and the embodiments of the present invention are not described in detail here.

[0214] S74. Display the target object in the ARHUD according to the target state parameter and the second altitude.

[0215] It can be understood that the display position of the target object in ARHUD is adjusted based on the target state parameter and the second height. This enables the system to effectively respond to changes in road slope and vehicle posture, ensuring that the displayed information in ARHUD always matches the actual environment. This not only improves the driver's ability to perceive the environment, but also significantly improves driving safety in complex road conditions.

[0216] In one possible implementation, in order to enhance driving safety and driving experience, it is also necessary to add annotation information to the ARHUD so that the driver can perceive the surrounding environment more comprehensively. The specific implementation process is as follows:

[0217] When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in ARHUD.

[0218] Among them, the target warning function refers to an important function in ARHUD, which is used to detect and warn the driver in real time about potential dangerous targets or obstacles in front, on the side or in the surrounding area of ​​the vehicle; the annotation information refers to various auxiliary information or prompts presented to the driver in a visual way in ARHUD. The annotation information can be superimposed on the display content of ARHUD in the form of graphics, colors, text or symbols.

[0219] For example, text messages such as "Speed ​​limit 30 km / h" and "Pedestrians ahead" may be displayed in the ARHUD as a prompt.

[0220] It should be noted that the calculation of the orientation of the annotation information can be processed based on the accumulated delay and the calculation of the second height of the vehicle. For example, when the lane departure warning function is enabled when the vehicle crosses the line without turning on the turn signal, a red warning color can be superimposed on the original lane line position for early warning. If there is a function to mark dangerous targets, they will be marked after calculating the second height of the vehicle. More complex function marking can be calculated based on the second state parameters of dynamic targets and static targets, and then processed uniformly with all target information later.

[0221] Since the driver's observation point will also change due to seat adjustment or sitting posture changes, the target information displayed on the ARHUD will be inconsistent with the real scene, making it difficult for users to match the target information marked on the ARHUD with the real scene, and unable to achieve timely navigation, warning and other functions, which ultimately leads to users' distrust of the target information marked on the ARHUD and the frequency of using ARHUD technology. Therefore, in order to ensure that the target information displayed on the ARHUD is consistent with the real world, it is crucial to dynamically adjust the display information of the target on the ARHUD.

[0222] In a possible implementation, since the driver may adjust his sitting posture or seat position during driving, which may cause the driver's observation point to change, the target state parameters need to be further updated according to the driver's observation point before being displayed. The specific update process is as follows:

[0223] Firstly, the coordinates of the driver's observation point are collected through the eye tracking sensor module; then, the position difference between the driver's observation point and the observation origin is calculated according to the coordinates of the observation point and the coordinates of the pre-calibrated observation origin; finally, the target state parameters are updated according to the position difference to generate updated target state parameters.

[0224] The coordinates of the pre-calibrated observation origin are the coordinates of the driver's observation point in the standard sitting posture.

[0225] It is understandable that due to the adjustment of the driver's sitting posture or seat position, the driver's current observation point is offset relative to the standard observation point, so it is necessary to collect the coordinates of the driver's observation point in real time through the eye tracking sensor module, and then calculate the position difference between the driver's observation point and the observation origin, and finally adjust the target state parameters in the ARHUD. By dynamically adjusting the target state parameters, the ARHUD can accurately display the target object regardless of how the position of the head or eyes changes.

[0226] Figure 8 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ,like Figure 8As shown in the figure, the road slope may cause the position of the target displayed on the ARHUD to deviate from the actual position. When the vehicle is driving on a sloped road, due to the inclination of the vehicle body, that is, the pitch angle of the vehicle body, the target displayed on the ARHUD will visually appear to be vertically offset. If the vehicle is going uphill, the actual position of the vehicle may be higher than the position of the virtual image marked on the ARHUD; correspondingly, when going downhill, the actual position of the vehicle may be lower than the position of the virtual image marked on the ARHUD.

[0227] Fig. 9 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 .like Fig. 9 As shown in the figure, the discrete slope information of the vehicle road is shown. Since the height and pitch angle changes of the target on the slope are generally difficult to detect accurately by sensors, they can be better calculated through the road surface information. Therefore, the map positioning information can be used to obtain the vehicle position and the discrete slope information of the road ahead, such as (0, S0), (x1, S1), ..., (x n , S n ), where (0, S0) indicates that the slope corresponding to the vehicle coordinate system position is S0. Generally, the slope refers to the ratio of the rise (or fall) of the road to its horizontal distance, usually expressed as a percentage or angle. The slope can be positive (uphill) or negative (downhill). For ease of explanation, S is assumed here. i (i is 0 to n) is the angle in radians; i The road surface height at is denoted as h i .

[0228] Because we only need to obtain the relative height of the target object to the current vehicle, we can set h0 = 0; then use the linear method to calculate the height of each position point. For each i from 1 to n, the specific calculation formula is:

[0229] h i =h i-1 +(x i -x i-1 )*tan(S i -1)

[0230] Among them, h i For each position point i, the relative height of the target to the current vehicle; x i are different positions in the discrete slope; S is the slope.

[0231] After that, find the interval [x i-1 ,x i ], and then linear interpolation is used to calculate the slope. The specific calculation formula is:

[0232] Sx =S i-1 +(xx i-1 ) / (x i -x i-1 )*(S i -S i-1 )

[0233] Among them, S x is the slope at position x; i is the different positions in the discrete slope; S i is the position slope at each position point i.

[0234] Finally, the first height of the target object relative to the ground at a given position x can be calculated. The specific calculation formula is:

[0235]

[0236] Among them, h x is the first height of the target relative to the ground at a given position x; S i The position slope at each position point i; x i are different positions in the discrete slope.

[0237] It should be noted that the above calculation process is a calculation method adopted by the present invention, and linear interpolation or other similar mathematical algorithms may be used. In the actual calculation process, different calculation methods may be selected according to system performance, and the present invention does not make specific restrictions here. In addition, for more complex roads, when the vehicle position and the discrete slope information (x i , S i ) is too sparse, you can also use methods such as cubic spline interpolation to process it in advance to reduce (x i , S i ) between the two.

[0238] Furthermore, since the vehicle is on a slope, it is also necessary to calculate the second height of the target object relative to the vehicle based on the pitch angle of the vehicle, the first height of the target object and the target state parameter.

[0239] Fig.10 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 .like Fig.10 As shown in the figure, the distance between the target object and the current vehicle in the slope direction is shown in the top plan view. It can be seen from the figure that the distance between the target object and the current vehicle in the slope direction should be the distance between point A and point C. Because the vehicle-mounted ARHUD only displays the target in front of the vehicle, in practical applications, the longitudinal distance line segment X of the target object relative to the vehicle can be simplified to replace it, that is, the first height of the target object relative to the ground.

[0240] Fig.11 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 . Among them, (a) is the principle diagram of the height and pitch angle of the target object relative to the vehicle on the ramp, and (b) is the schematic diagram of the height and pitch angle of the target object relative to the vehicle on the ramp. Among them, line segment AD is the second height of the target object relative to the vehicle.

[0241] The specific calculation process of the second height of the target object relative to the vehicle is as follows: first, the pitch angle β of the target object relative to the vehicle is calculated; then, the second height of the target object relative to the vehicle can be calculated. The specific calculation formula is:

[0242] β=(S x -α)=(S x -S0)

[0243] Where β is the pitch angle of the target object relative to the vehicle; S x is the slope at the x position; α is the pitch angle of the vehicle, and α=S0.

[0244] AD=(h x -x*tan(α))*cos(α)=h x *cos(α)-x*sin(α)=h x *cos(S0)-x*sin(S0)

[0245] Where AD is the second height of the target relative to the vehicle; h x is the first height of the target object relative to the ground at a given position x; α and S0 are both the pitch angles of the vehicle.

[0246] Fig.12 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 .exist Fig.12In the figure, it is assumed that the coordinates of the driver's pre-calibrated observation point in the eye tracking sensor module coordinate system are (x0, y0, z0. At this time, the ARHUD display content seen by the driver can match and fit the actual static target. When the driver's observation point moves relatively by (dx, dy, dz), in order to keep the display content fit with the real scene, the position point of the virtual image of the target should meet the following conditions: the vector from the driver's observation point to the position point of the virtual image of the target should be the same as the vector from the observation origin to the initial position point of the virtual image of the target. Therefore, it is necessary to calculate the position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of the pre-calibrated observation origin, and then accumulate the position difference with the original target state parameters to display as the new target position to be rendered. It should be noted that some eye tracking sensor modules output the positions of the left and right eyes respectively, and the average position can be calculated as the observation point position.

[0247] Fig.13 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 6 .like Fig.13 As shown in the figure, the complete process of the data display method based on ARHUD is shown, and its specific steps include:

[0248] Step 131: During the transmission of the target signal, the position of the target object in the ARHUD is corrected according to the accumulated delay of the target signal.

[0249] Step 132: Correct the position of the target object in the ARHUD according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter.

[0250] Step 133: Correct the position of the target object in the ARHUD according to the coordinates of the observation point and the coordinates of the pre-calibrated observation origin.

[0251] Step 134: When it is determined that the vehicle triggers the target warning function according to the current position of the target object in the ARHUD, the annotation information corresponding to the target warning function is displayed in the ARHUD.

[0252] It should be understood that the specific implementation methods in each step have been specifically described in the corresponding embodiments above, and the embodiments of the present application will not be described in detail here.

[0253] It should be understood that the embodiments of the present application do not limit the execution order of the various steps.

[0254] The following are embodiments of the device of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.

[0255] Fig.14 This is a schematic diagram of the structure of the data display device based on ARHUD provided by an embodiment of the present invention. Fig.14 As shown, the ARHUD-based data display device 140 is applied to a vehicle, and includes:

[0256] A first calculation module 141 is used to calculate the cumulative delay of the target signal to each module when the target signal is transmitted in the target transmission link, the target signal is obtained by the acquisition module performing signal acquisition on the target object, and the target transmission link is a link from the acquisition module to the ARHUD;

[0257] The second calculation module 142 is used to calculate the first state parameter of the vehicle according to the target delay, the driving data of the vehicle and the wheelbase if the cumulative delay of the target signal in any module exceeds the preset delay, the state parameter includes the position change parameter and the heading angle change parameter;

[0258] An acquisition module 143, used to acquire a second state parameter of the target object;

[0259] The third calculation module 144 is used to calculate the target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter;

[0260] The display module 145 is used to display the target object in the ARHUD according to the target state parameters.

[0261] Furthermore, the first calculation module 141 is specifically configured to:

[0262] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0263] Furthermore, the first calculation module 141 is further configured to:

[0264] The processing delay of the target signal in the module is calculated according to the signal input timestamp and signal output timestamp of the target signal in the module.

[0265] Furthermore, the first calculation module 141 is further configured to:

[0266] For each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0267] If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0268] Furthermore, the acquisition module 143 is specifically used for:

[0269] If the target object is a static target object, collecting a second state parameter of the target object;

[0270] If the target is a dynamic target, the absolute speed, absolute acceleration, steering angular speed and steering angular acceleration of the target are collected;

[0271] A second state parameter of the target object is calculated according to the accumulated delay, the absolute speed, the absolute acceleration, the steering angular speed, and the steering angular acceleration of the target object.

[0272] Furthermore, the third calculation module 144 is specifically used for:

[0273] According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

[0274] Furthermore, the ARHUD-based data display device 140 further includes a processing device, which, after calculating the target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter, is further used to:

[0275] According to the map positioning information, the discrete slope information of the vehicle position and the area in front of the vehicle is obtained;

[0276] Determining a first height of the target object relative to the ground according to the position of the vehicle and discrete slope information of the area in front of the vehicle;

[0277] Determining a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter;

[0278] Accordingly, the display module 145 is specifically used for:

[0279] The target object is displayed in the ARHUD according to the target state parameters and the second altitude.

[0280] Furthermore, the processing module is also used to clear the accumulated delay of the module.

[0281] Furthermore, the processing module is also used to:

[0282] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0283] According to the coordinates of the observation point and the coordinates of the pre-calibrated observation origin, the position difference between the driver's observation point and the observation origin is calculated;

[0284] The target state parameters are updated according to the position difference to generate updated target state parameters.

[0285] Furthermore, the display module 145 is also used for:

[0286] When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in ARHUD.

[0287] The ARHUD-based data display device provided in an embodiment of the present invention can be used to execute the ARHUD-based data display method in any of the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0288] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, all or part of them can be integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements. They can also be implemented in the form of hardware. Some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0289] Fig.15 Schematic diagram of the structure of a vehicle provided by an embodiment of the present invention. Fig.15 As shown, the vehicle 150 may include: a processor 151, a memory 152, and computer execution instructions stored in the memory 152 and executable on the processor 151. When the processor 151 executes the computer execution instructions, the ARHUD-based data display method provided in any of the aforementioned embodiments is implemented.

[0290] Optionally, the above-mentioned components of the vehicle 150 may be connected via a system bus.

[0291] The memory 152 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0292] Optionally, the vehicle 150 may also include a communication interface for interacting with other devices.

[0293] It should be understood that the processor 151 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0294] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0295] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The above-mentioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed. The above-mentioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0296] The vehicle provided in the embodiment of the present invention can be used to execute the ARHUD-based data display method provided in any of the above-mentioned method embodiments, and its implementation principles and technical effects are similar and will not be repeated here.

[0297] An embodiment of the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the computer executes the above-mentioned ARHUD-based data display method.

[0298] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0299] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0300] An embodiment of the present invention also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the above-mentioned ARHUD-based data display method can be implemented.

[0301] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data display method based on ARHUD, It is characterized in that Applied to a vehicle, the method comprises: When a target signal is transmitted in a target transmission link, calculating the cumulative delay of the target signal to be transmitted to each module, wherein the target signal is obtained by collecting signals of a target object by an acquisition module, and the target transmission link is a link from the acquisition module to an augmented reality head-up display (ARHUD); If the accumulated delay of the target signal in any module exceeds the preset delay, the first state parameter of the vehicle is calculated according to the accumulated delay, the driving data of the vehicle and the wheelbase, and the state parameter includes a position change parameter and a heading angle change parameter; Acquiring a second state parameter of the target object; Calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter; The target object is displayed in the ARHUD according to the target state parameters.

2. The method according to claim 1, It is characterized in that The calculating the cumulative delay of transmitting the target signal to each module includes: For each module in the target transmission link, the accumulated delay of the target signal in the module is calculated according to the accumulated delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

3. The method according to claim 2, It is characterized in that The method further comprises: The processing delay of the target signal in the module is calculated according to the signal input timestamp and the signal output timestamp of the target signal in the module.

4. The method according to claim 2, It is characterized in that For each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0; If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

5. The method according to any one of claims 1 to 4, It is characterized in that The obtaining of the second state parameter of the target object comprises: If the target object is a static target object, collecting the second state parameter of the target object; If the target object is a dynamic target object, the absolute speed, absolute acceleration, steering angular speed and steering angular acceleration of the target object are collected; The second state parameter of the target object is calculated according to the accumulated delay, the absolute speed of the target object, the absolute acceleration, the steering angular speed, and the steering angular acceleration.

6. The method according to any one of claims 1 to 4, It is characterized in that The step of calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter includes: According to the first state parameter and the second state parameter, a coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

7. The method according to any one of claims 1 to 4, It is characterized in that After calculating the target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter, the method further includes: Acquiring the position of the vehicle and discrete slope information of the area in front of the vehicle according to the map positioning information; Determining a first height of the target object relative to the ground according to the position of the vehicle and discrete slope information of an area in front of the vehicle; Determining a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter; Correspondingly, displaying the target object in the ARHUD according to the target state parameter includes: The target object is displayed in the ARHUD according to the target state parameter and the second height.

8. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: Clear the accumulated delay of the module.

9. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: The coordinates of the driver's observation point are collected through the eye tracking sensor module; Calculating the position difference between the driver's observation point and the observation origin according to the coordinates of the observation point and the coordinates of a pre-calibrated observation origin; The target state parameter is updated according to the position difference to generate an updated target state parameter.

10. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in the ARHUD.

11. A data display device based on ARHUD, It is characterized in that Applied to vehicles, including: A first calculation module is used to calculate the cumulative delay of the target signal to each module when the target signal is transmitted in the target transmission link, wherein the target signal is obtained by collecting signals of the target object by the collection module, and the target transmission link is a link from the collection module to the ARHUD; a second calculation module, configured to calculate a first state parameter of the vehicle according to the target delay, the driving data of the vehicle and the wheelbase if the cumulative delay of the target signal in any module exceeds a preset delay, the state parameter including a position change parameter and a heading angle change parameter; An acquisition module, used for acquiring a second state parameter of the target object; A third calculation module: used for calculating a target state parameter of the target object relative to the vehicle according to the first state parameter and the second state parameter; A display module is used to display the target object in the ARHUD according to the target state parameters.

12. A vehicle, include: A processor, a memory, and a computer-executable instruction stored in the memory and executable on the processor, wherein the processor is used to implement the ARHUD-based data display method as described in any one of claims 1 to 10 when executing the computer-executable instruction.

13. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the ARHUD-based data display method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, it is used to implement the ARHUD-based data display method according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Time delay control method and system for tracking autonomous driving path of intelligent vehicle

    CN113050651A

  • Vehicle head-up display control method and device and vehicle

    CN115534672A

  • Reticle and target based calibration for augmented reality head-up displays

    CN115685548A

  • Head-up display system, guidance information generation method, apparatus, medium, and program

    CN116859591A

  • Three-phase display method and system based on ARHUD and vehicle

    CN117698416A