ARHUD-based data display method, device, equipment and medium

Abstract

The application relates to an ARHUD-based data display method, device, equipment and medium, which comprises the following steps: when a target signal is transmitted in a target transmission link, the cumulative delay of the target signal transmitted to each module is calculated, the target signal is obtained by signal acquisition of a target object by an acquisition module, and the target transmission link is a link between the acquisition module and an ARHUD. If the cumulative delay of the target signal at any module exceeds a preset delay, a first state parameter of a vehicle is calculated according to the cumulative delay, driving data of the vehicle and a wheelbase, the state parameter comprises a position change parameter and a heading angle change parameter. A second state parameter of the target object is acquired, and a target state parameter of the target object relative to the vehicle is calculated according to the first state parameter and the second state parameter. The target object is displayed in the ARHUD according to the target state parameter. The application improves the accuracy of displaying target object information in the ARHUD.

Description

Technical Field

[0001] The present invention relates to the field of automotive technology, and in particular to an ARHUD-based data display method, device, equipment, and medium. Background Art

[0002] An augmented reality head-up display (ARHUD) is a device that combines augmented reality technology with a head-up display. In an in-vehicle display system, it helps drivers quickly observe the status of navigation, warnings, intelligent driving, and other functions while observing the road ahead. This reduces the frequency of drivers looking down at the instrument panel and increases driving safety. However, due to changes in the vehicle itself and the external environment, the objects displayed on the ARHUD can easily be inaccurate. Therefore, in order to ensure that the object information displayed on the ARHUD is consistent with the real world, it is crucial to dynamically adjust the display information of the objects on the ARHUD.

[0003] At present, the method for adjusting the target object information displayed on the ARHUD is mainly to adjust the delay of the motion trajectory information collected by the image collector. The specific process is to first use the terminal device to simultaneously collect the actual motion trajectory of the flat plate with a fixed-pitch pattern array (i.e., the calibration plate) and the viewfinder of the image collector mounted on the vehicle itself, and then process the actual motion trajectory of the calibration plate and the viewfinder to obtain the absolute delay of the image collector, and then make a prediction based on the absolute delay of the image collector to adjust the target object information displayed on the ARHUD.

[0004] However, the data collected by the existing technology for transmission delay information is single and has low accuracy, and there is still the problem of inaccurate position information of the target object displayed on the ARHUD. Summary of the Invention

[0005] The purpose of the present invention is to provide a data display method based on ARHUD to solve the problem in the prior art that the accuracy of signal transmission delay information collection is low and the position information of the target object displayed on the ARHUD is inaccurate.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A data display method based on ARHUD, comprising:

[0008] When a target signal is transmitted in a target transmission link, calculating the cumulative delay of the target signal to each module, wherein the target signal is obtained by collecting signals from a target object by an acquisition module, and the target transmission link is a link from the acquisition module to the ARHUD;

[0009] If the cumulative delay of the target signal in any module exceeds a preset delay, calculating the first state parameter of the vehicle according to the cumulative delay, the driving data of the vehicle, and the wheelbase, the state parameter including a position change parameter and a heading angle change parameter;

[0010] Acquiring a second state parameter of the target object;

[0011] Calculating a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter;

[0012] The target object is displayed in the ARHUD according to the target state parameters.

[0013] According to the above technical means, by combining the vehicle's state parameters and the target object's state parameters, the target state parameters of the target object relative to the vehicle are calculated, and finally the target object information displayed on the ARHUD is dynamically adjusted. Since the vehicle's state parameters and the target object's state parameters are taken into consideration at the same time, the signal transmission delay information obtained is more comprehensive and accurate, and can effectively adjust the position information of the target object displayed on the ARHUD according to the current environment.

[0014] Furthermore, the calculating of the cumulative delay of the target signal transmitted to each module includes:

[0015] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0016] According to the above technical means, by accurately calculating the cumulative delay of the target signal in the module based on the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay, the delay of the entire target signal in the transmission link can be effectively managed and controlled, avoiding display lag or errors caused by delay accumulation.

[0017] Furthermore, the method further comprises:

[0018] The processing delay of the target signal in the module is calculated according to the signal input timestamp and the signal output timestamp of the target signal in the module.

[0019] Furthermore, for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0020] If subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0021] According to the above technical means, by setting the first preset delay and the second preset delay to compensate and adjust the module that cannot calculate the processing delay, the delay of the entire signal transmission link can be effectively managed and controlled to avoid display lag or error caused by delay accumulation.

[0022] Furthermore, obtaining the second state parameter of the target object includes:

[0023] If the target object is a static target object, collecting the second state parameter of the target object;

[0024] If the target object is a dynamic target object, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object are collected;

[0025] The second state parameter of the target object is calculated according to the accumulated delay, the absolute velocity of the target object, the absolute acceleration, the steering angular velocity, and the steering angular acceleration.

[0026] According to the above technical means, for static targets, only their second state parameters need to be collected; for dynamic targets, their second state parameters need to be collected and calculated, especially in the presence of delays, to perform accurate state prediction and compensation, which can significantly improve the accuracy and real-time display of the target objects in the ARHUD system, thereby enhancing driving safety and user experience.

[0027] Furthermore, calculating a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter includes:

[0028] According to the first state parameter and the second state parameter, a coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

[0029] The above technical means can accurately and uniformly transform the orientation of dynamic and static targets into the new vehicle coordinate system. This ensures that the target information displayed on the ARHUD remains accurate even after the vehicle moves and rotates, improving the reliability and real-time performance of the system.

[0030] Furthermore, after calculating the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter, the method further includes:

[0031] Obtaining the position of the vehicle and discrete slope information of the area in front of the vehicle based on the map positioning information;

[0032] determining a first height of the target object relative to the ground based on the position of the vehicle and discrete slope information of an area in front of the vehicle;

[0033] determining a second height of the target relative to the vehicle based on the pitch angle of the vehicle, the first height of the target, and the target state parameter;

[0034] Accordingly, displaying the target object in the ARHUD according to the target state parameter includes:

[0035] The target object is displayed in the ARHUD according to the target state parameter and the second height.

[0036] Based on the above technical means, by integrating road slope information and vehicle pitch angle, the system can accurately correct the display position of the target object in the ARHUD, reducing visual errors caused by terrain changes or vehicle posture changes.

[0037] Furthermore, the method further comprises:

[0038] Clear the accumulated delay of the module.

[0039] According to the above technical means, it is ensured that the subsequent transmission of new target signals is not affected by historical delays.

[0040] Furthermore, the method further comprises:

[0041] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0042] Calculating a position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of a pre-calibrated observation origin;

[0043] The target state parameter is updated according to the position difference to generate an updated target state parameter.

[0044] According to the above technical means, the target state parameters can be dynamically adjusted, and ARHUD can ensure that the target object information always remains in the driver's field of view, regardless of how the position of his head or eyes changes. This not only improves the adaptability and comfort of the system, but also enhances the driver's interactive experience with the ARHUD system.

[0045] Furthermore, the method further comprises:

[0046] When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in the ARHUD.

[0047] The above-mentioned technical means can help drivers better understand the status of the vehicle and its auxiliary functions during driving, thereby improving driving safety, enhancing perception of the surrounding environment, and providing a better driving experience.

[0048] A data display device based on ARHUD, applied to a vehicle, comprising:

[0049] A first calculation module is configured to calculate the cumulative delay of the target signal from being transmitted to each module when the target signal is transmitted in the target transmission link, wherein the target signal is obtained by collecting signals from the target object by the collection module, and the target transmission link is the link between the collection module and the ARHUD;

[0050] a second calculation module, configured to calculate a first state parameter of the vehicle based on the accumulated delay, the vehicle's driving data, and the wheelbase, if the accumulated delay of the target signal in any module exceeds a preset delay. The state parameter includes a position change parameter and a heading angle change parameter.

[0051] An acquisition module, configured to acquire a second state parameter of the target object;

[0052] A third calculation module is used to calculate the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter;

[0053] The display module is used to display the target object in the ARHUD according to the target state parameters.

[0054] Furthermore, the first calculation module is specifically configured to:

[0055] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0056] Furthermore, the first calculation module is further configured to:

[0057] The processing delay of the target signal in the module is calculated based on the signal input timestamp and signal output timestamp of the target signal in the module.

[0058] Furthermore, the first calculation module is further configured to:

[0059] For each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0060] If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0061] Furthermore, the acquisition module is specifically used to:

[0062] If the target object is a static target object, collecting a second state parameter of the target object;

[0063] If the target is a dynamic target, the absolute velocity, absolute acceleration, steering angular velocity, and steering angular acceleration of the target are collected;

[0064] A second state parameter of the target object is calculated based on the accumulated delay, the absolute speed, the absolute acceleration, the steering angular speed, and the steering angular acceleration of the target object.

[0065] Furthermore, the third calculation module is specifically configured to:

[0066] According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle to generate a target state parameter of the target object relative to the vehicle.

[0067] Furthermore, the ARHUD-based data display device 140 further includes a processing device, which, after calculating the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter, is further configured to:

[0068] Obtain the vehicle's location and the discrete slope information of the area in front of the vehicle based on the map positioning information;

[0069] determining a first height of the target object relative to the ground based on the position of the vehicle and discrete slope information of an area in front of the vehicle;

[0070] determining a second height of the target object relative to the vehicle based on the pitch angle of the vehicle, the first height of the target object, and the target state parameter;

[0071] Accordingly, the display module is specifically used for:

[0072] The target object is displayed in the ARHUD according to the target state parameters and the second altitude.

[0073] Furthermore, the processing module is also used to clear the accumulated delay of the module.

[0074] Furthermore, the processing module is further configured to:

[0075] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0076] Calculate the position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of the pre-calibrated observation origin;

[0077] The target state parameters are updated according to the position difference to generate updated target state parameters.

[0078] Furthermore, the display module is also used for:

[0079] When the vehicle triggers the target warning function, the marking information corresponding to the target warning function is displayed in the ARHUD.

[0080] A vehicle includes: a processor, a memory, and computer-executable instructions stored in the memory and executable by the processor, wherein the processor is used to implement the above-mentioned ARHUD-based data display method when executing the computer-executable instructions.

[0081] A computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned ARHUD-based data display method when executed by a processor.

[0082] A computer program product includes a computer program, which is used to implement the above-mentioned ARHUD-based data display method when executed by a processor.

[0083] Beneficial effects of the present invention:

[0084] (1) When the target signal is transmitted in the target transmission link, the cumulative delay generated in each module is related to the accuracy of the target object's display content on the ARHUD, and under the cumulative delay, the vehicle's own motion trajectory will also affect the accuracy of the target object's display content on the ARHUD. Therefore, this technical solution calculates the cumulative delay of the target signal exceeding the preset delay in any module, and calculates the state parameters of the vehicle and the target object on this basis. Since the target state parameters finally formed take into account the data information changes of multiple modules, the accuracy of the target object's display content on the ARHUD is effectively improved.

[0085] The accuracy of the content displayed on ARHUD.

[0086] (2) By considering the state parameters of the vehicle and the target object under the cumulative delay, the relative error between the vehicle and the target object under different driving conditions is effectively eliminated. In addition, the acquisition module monitors the driver's eye position in real time, ensuring that the target object information displayed on the ARHUD always remains within the driver's field of view. This not only improves the system's adaptability and comfort, but also enhances the driver's interactive experience with the ARHUD system. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A schematic diagram of the structure of the ARHUD system provided in an embodiment of the present invention;

[0088] Figure 2 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ;

[0089] Figure 3 A diagram of a transmission system for a target transmission link with cumulative delay provided by an embodiment of the present invention;

[0090] Figure 4 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;

[0091] Figure 5 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;

[0092] Figure 6 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;

[0093] Figure 7 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;

[0094] Figure 8 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ;

[0095] Figure 9 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;

[0096] Figure 10 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;

[0097] Figure 11 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;

[0098] Figure 12 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;

[0099] Figure 13 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 6;

[0100] Figure 14 A schematic structural diagram of a data display device based on ARHUD provided in an embodiment of the present invention;

[0101] Figure 15 A schematic structural diagram of a vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0102] The following describes the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are intended only to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0103] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and corresponding operation entrances must be provided for users to choose to authorize or refuse.

[0104] Before introducing the present invention, the application background of the present invention is first explained.

[0105] With the rapid development of technology, the application of ARHUD in in-vehicle display systems is becoming increasingly widespread. ARHUD can help drivers observe the road ahead while also quickly checking the status of navigation, warnings, intelligent driving, and other functions. This helps drivers focus more on the road ahead, reduces the frequency of looking down at the instrument panel, and improves driving safety.

[0106] However, in actual applications, due to the rapid driving of the vehicle, environmental information will change rapidly when passing through slopes, curves, or turning, changing lanes, turning around, etc. At the same time, when the relative speed of the vehicle itself and the target object is greatly different, the relative position of the vehicle itself and various targets will change rapidly. These changes will make it difficult to match the content displayed by the ARHUD with the real scene due to signal transmission delays. In particular, due to cost and other reasons, the scene fusion function of the ARHUD in many vehicles is not based on dedicated sensors, but relies on sensor information such as cameras used for intelligent driving functions or fused target information, and the signal processing and transmission links are lengthy. At this time, the accumulated delays of various sensors and processors are large, and the target position, lane lines, etc. marked on the ARHUD will be even more inaccurate.

[0107] At present, the method for adjusting the target information displayed on the ARHUD is mainly to adjust the target information displayed on the ARHUD through the absolute delay of the image collector. The specific process is to first use the terminal device to simultaneously collect the actual motion trajectory of the flat plate with a fixed-pitch pattern array (i.e., the calibration plate) and the viewfinder screen of the image collector mounted on the vehicle itself, and establish the relationship between the actual motion trajectory of the target and the image acquisition; then, use image processing technology to simultaneously extract the actual motion trajectory of the calibration plate and the target motion trajectory of the calibration plate in the viewfinder; then, on the same time axis, use the time difference between the actual motion trajectory and the target motion trajectory as the absolute delay of the image collector; finally, use the calculated absolute delay to predict the motion trajectory captured by the image collector, obtain the motion state of the target object within the delay time, and adjust the display content on the ARHUD accordingly to ensure that the displayed target information is synchronized with the actual environment.

[0108] However, since the performance of the image acquisition device may be affected by environmental factors such as lighting and weather, these factors can lead to errors in delay calculation. Furthermore, static absolute delay measurement cannot maintain accuracy when environmental and system parameters change. Especially when delays are dynamically changing, static measurement can lead to inaccurate displayed information. Furthermore, changes to the corresponding hardware or algorithms in the vehicle require full recalibration, which increases the complexity and time cost of system maintenance.

[0109] In summary, when collecting and processing transmission delay information, the existing technology has the problem that the collected data is single and has low accuracy, which leads to inaccurate position information of the target object displayed on the ARHUD.

[0110] Based on the above technical problems, the present invention provides a data display method based on ARHUD. Taking into account the inconsistency of the cumulative delays of signals in different acquisition modules, when the target signal is transmitted from the acquisition module to the ARHUD, the cumulative delay of the target signal transmitted to each module is calculated, and when the cumulative delay of the target signal in any module exceeds the preset delay, taking into account that the surrounding environment and the vehicle itself will change rapidly, the first state parameter of the vehicle and the second state parameter of the target object are obtained respectively, which can effectively compensate for the display error caused by the delay and improve the accuracy and reliability of the ARHUD display information. Finally, the target state parameter of the target object relative to the vehicle is calculated and displayed based on the first state parameter and the second state parameter to realize the dynamic adjustment of the target object information displayed on the ARHUD, thereby solving the problem that the transmission delay information data is single due to the use of only the image sensor to measure the delay, and the position information of the target object displayed on the ARHUD is inaccurate due to the inability to measure the dynamic target object.

[0111] The technical solution of the present invention is described in detail below through specific embodiments.

[0112] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0113] Figure 1 A schematic diagram of the structure of the ARHUD system provided in an embodiment of the present invention; Figure 1 As shown in the figure, the ARHUD system includes an environmental sensing module, a sensor information fusion module, a vehicle status information module, an eye tracking sensor module, an ARHUD calculation module, an ARHUD rendering module and an ARHUD display screen.

[0114] Among them, first, the road and target objects are collected and detected through the environmental sensing module (such as camera, millimeter wave radar, ultrasonic radar, lidar, etc.) to form sensor information, and the sensor information fusion module fuses the sensor information; then, the vehicle status information module is used to collect the vehicle's (that is, the vehicle's) orientation, posture, steering, speed, wheelbase, steering transmission ratio and other information; then, the eye tracking sensor module (for example, driver monitoring system) is used to detect the driver's observation point; then, the information of the above-mentioned sensor information fusion module, vehicle status information module and eye tracking sensor module is input into the ARHUD calculation module for summary calculation; finally, all the information is input into the ARHUD rendering module for rendering, and displayed on the ARHUD display screen. Finally, the driver can see the display content and the real scene of the outside world through the ARHUD display screen.

[0115] It should be noted that, since the present invention takes into account the changes in the state parameters of the target object and the vehicle under the cumulative delay of the target signal, the ARHUD calculation module also includes a delay compensation module, a slope compensation module, a function labeling calculation module, a driver observation point compensation module, and a distortion calibration and correction calculation module. The slope compensation module includes map positioning information, such as slope data of a discrete distance in front of the vehicle, slope at 0m, slope at 5m and other data. The function labeling calculation module contains labeling information, such as dangerous targets, line-crossing prompts, navigation arrows and other information. The purpose is to provide the driver with guidance or warning information that fits the real scene. On the other hand, the ARHUD system can also include an ARHUD mechanical adjustment device to assist in adjusting the reflection angle or image projection angle of the ARHUD display screen to increase the adjustment range of its visual display.

[0116] Figure 2 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ,like Figure 2 As shown, the data display method based on ARHUD may include the following steps:

[0117] S21. When a target signal is transmitted in a target transmission link, calculate the cumulative delay of the target signal being transmitted to each module.

[0118] Among them, the target signal is obtained by collecting signals from the target object through the acquisition module (i.e., environmental sensing module, such as camera, millimeter wave radar, ultrasonic radar, lidar, etc.). The target transmission link is the link from the acquisition module to the ARHUD. The target object refers to the object or feature that needs to be displayed in the ARHUD system, such as pedestrians, road signs and signals, passing vehicles and obstacles.

[0119] It is understandable that when the target object information is collected by the environmental sensing module and then converted into a target signal and transmitted in the target transmission link, it will pass through multiple modules (such as the sensor information fusion module, etc.), and because different acquisition modules may have different processing speeds and delays, by calculating the cumulative delay of each module, these delay differences can be identified and processed to ensure that the data of all modules can be aligned in time, thereby ultimately ensuring that consistent display information is provided on the ARHUD system, thereby improving the accuracy of the target object information display.

[0120] S22. If the cumulative delay of the target signal in any module exceeds the preset delay, a first state parameter of the vehicle is calculated based on the cumulative delay, the driving data of the vehicle, and the wheelbase.

[0121] The state parameters include position change parameters and heading angle change parameters.

[0122] It should be understood that since the vehicle is traveling at a high speed, if the cumulative delay of the target signal in any module exceeds the preset delay, it means that the accuracy of the target object displayed in ARHUD is low at this time, and the position of the target object displayed in ARHUD needs to be updated. In order to ensure that the position of the target object displayed in ARHUD is consistent with the actual scene, it is necessary to calculate the state change of the vehicle and the target object within the cumulative delay respectively, and adjust the display position of the target object in ARHUD according to the state change of the vehicle and the target object. First, it is necessary to obtain the state change of the vehicle within the cumulative delay, that is, the first state parameter.

[0123] For example, the vehicle's driving data recorded in the vehicle status information module is first used, including the steering wheel angle δ, vehicle speed v, and steering gear ratio R. The cumulative delay Δt of the target signal in the module and the vehicle wheelbase L are then obtained. Then, for ease of calculation, the center of the vehicle's front is taken as the origin, the positive direction of the x-axis is along the vehicle body, the positive direction of the y-axis is perpendicular to the x-axis and toward the left side of the vehicle body, and the positive direction of the z-axis is perpendicular to the xOy plane and upward. The steering wheel angle and heading angle are both counterclockwise as positive directions. Finally, the front wheel angle, instantaneous steering radius, instantaneous steering angular velocity, and the heading angle of the vehicle within the cumulative delay period Δt are calculated respectively. The specific calculation process is as follows:

[0124] 1) Calculate the front wheel angle: Calculate the front wheel angle θ using the steering wheel angle and steering gear ratio. The calculation formula is:

[0125]

[0126] Where θ is the front wheel angle; δ is the steering wheel angle; and R is the vehicle steering ratio.

[0127] 2) Calculate the instantaneous turning radius: Calculate the instantaneous turning radius R by the front wheel angle and vehicle wheelbase t The calculation formula is:

[0128]

[0129] Among them, R t is the instantaneous turning radius; L is the vehicle wheelbase; θ is the front wheel turning angle.

[0130] 3) Calculate the instantaneous steering angular velocity: Calculate the instantaneous steering angular velocity ω using the vehicle speed and instantaneous steering radius. The calculation formula is:

[0131]

[0132] Where, ω is the instantaneous steering angular velocity; v is the vehicle speed; R t is the instantaneous turning radius.

[0133] 4) Calculate the change in the vehicle's heading angle within the cumulative delay period Δt. The calculation formula is:

[0134] Δφ=ω×Δt

[0135] Where Δφ is the change in the vehicle’s heading angle within Δt; ω is the instantaneous steering angular velocity; and Δt is the accumulated delay.

[0136] It should be noted that when the steering wheel angle δ is 0 or its absolute value is less than a very small value, such as 10 -6 , and the turning radius R t Infinite, the steering angular velocity ω is zero, and the vehicle's motion can be simplified to linear motion. The position change of the vehicle at this time is:

[0137] Position change in the X direction: Δx = v × Δt

[0138] Among them, v is the vehicle speed; Δt is the cumulative delay, and Δx is the distance the vehicle travels forward.

[0139] Position change in the Y direction: Δy = 0

[0140] Wherein, Δy is the distance perpendicular to the x-axis toward the left side of the vehicle body.

[0141] The steering wheel angle δ is not 0 or its absolute value is greater than a very small value, such as 10 -6 , the vehicle moves along an arc, and the position change of the vehicle at this time is:

[0142] Position change in the X direction: Δx = R t ×sin(Δφ)

[0143] Where Δx is the distance the vehicle travels forward; R t is the instantaneous turning radius; Δφ is the change in the vehicle's heading angle within Δt.

[0144] Position change in the Y direction: Δy = R t ×(1-cos(Δφ))

[0145] It is understandable that if the cumulative delay of the target signal in any module exceeds the preset delay, it means that the cumulative delay may subsequently affect the display accuracy of the target object in ARHUD. Therefore, it is necessary to calculate the first state parameter of the vehicle based on the cumulative delay, the vehicle's driving data and wheelbase, so that the target object can be subsequently converted to the vehicle's coordinate system and the display position of the target object can be updated, thereby improving the display accuracy.

[0146] S23. Obtain a second state parameter of the target object.

[0147] Targets can be divided into static and dynamic objects based on their relative position to the ground. Static objects can include lane lines and curbs, while dynamic objects can include vehicles, people, and animals.

[0148] In a possible implementation, for targets in different states, their second state parameters are also different, specifically:

[0149] If the target object is a static target object, the second state parameter of the target object is determined to be 0; if the target object is a dynamic target object, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object are collected, and the second state parameter of the target object is calculated respectively by uniformly accelerated motion and uniformly accelerated steering based on the accumulated delay, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object.

[0150] It should be noted that, since the static target is stationary relative to the ground, there is no orientation change, so the second state parameter of the target is determined to be 0; for the dynamic target, since it is dynamic relative to the ground, it is necessary to compensate for its own orientation change, so the second state parameter of the target needs to be further calculated.

[0151] Optionally, for dynamic targets, the absolute velocity, absolute acceleration, steering angular velocity, and steering angular acceleration of the dynamic target from the sensor information fusion module can be used, and the position change parameters and heading angle change parameters of the dynamic target can be calculated based on the accumulated delay. In addition, if there are input quantities that cannot be obtained, the calculation can be simplified by treating them as zero.

[0152] In this way, the foundation is laid for the subsequent coordinate transformation of the target object, which can significantly improve the accuracy and real-time display of the target object in the ARHUD system, thereby enhancing driving safety and user experience.

[0153] S24. Calculate a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter.

[0154] In a possible implementation, the coordinate system of the second state parameter may be converted into a coordinate system corresponding to the vehicle based on the first state parameter and the second state parameter to generate a target state parameter of the target object relative to the vehicle.

[0155] Since the first state parameters (Δx, Δy, Δφ) of the vehicle are obtained in S24, the coordinate system transformation method is that the coordinate system of the target object on the vehicle is translated along the vector (Δx, Δy, 0) and rotated counterclockwise around the z-axis by Δφ.

[0156] It can be understood that by obtaining the first state parameter of the vehicle and the second state parameter corresponding to the target object through the above steps, the target object's coordinate system can be converted to the vehicle's coordinate system. By accurately calculating the target object's target state parameters relative to the vehicle, this method enables the ARHUD to accurately display the target object's real-time position and dynamic changes, allowing the driver to intuitively understand the target object's position and movement trends.

[0157] S25. Display the target object in the ARHUD according to the target state parameters.

[0158] In actual applications, after determining the target state parameters, the target object can be moved based on the current position of the target object according to the target state parameters, the target position of the target object in the ARHUD can be determined, and the target object can be updated to the target position in the ARHUD.

[0159] It can be understood that the target object is displayed in ARHUD according to the target state parameters, ensuring that no matter whether the vehicle is in a straight line or circular motion, or whether the target object is static or dynamic, ARHUD can provide accurate target object content, enhancing driving safety and user experience.

[0160] The data display method based on ARHUD provided by an embodiment of the present invention calculates the cumulative delay of the target signal transmitted to each module when the target signal is transmitted in the target transmission link; then, if the cumulative delay of the target signal in any module exceeds the preset delay, the first state parameter of the vehicle is calculated based on the cumulative delay, the vehicle's driving data and wheelbase; then, the second state parameter of the target object is obtained; then, based on the first state parameter and the second state parameter, the target state parameter of the target object relative to the vehicle is calculated; finally, according to the target state parameter, the target object is displayed in the ARHUD. This method can effectively compensate for the display error caused by the delay to achieve dynamic adjustment of the target object information displayed on the ARHUD. While improving the accuracy of the target object information displayed by the ARHUD, it also enhances the system's adaptability to complex driving environments, providing users with a safer and more reliable driving experience.

[0161] Furthermore, in S21, the specific process of calculating the cumulative delay of the target signal transmitted to each module may be:

[0162] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0163] In one possible implementation, since some modules in the system may not be able to calculate their processing delays, it is necessary to use preset delay values ​​for compensation and adjustment. Specifically, for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, then the first preset delay corresponding to the module is a positive number greater than 0; if the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, then the second preset delay corresponding to the module is a positive number greater than 0.

[0164] For example, assume there is a target transmission link consisting of the following five modules: module A, module B, module C, module D, and module E. In this link, module B cannot calculate the processing delay of the target signal within it, module D is the second-to-last module, and module E cannot calculate the processing delay of the target signal within it.

[0165] Then, since module B cannot calculate the processing delay of the target signal, a first preset delay must be set for module B, for example, 10ms; and since module D is the second-to-last module and module E cannot calculate the processing delay of the target signal within it, a second preset delay also needs to be set for module D, for example, 15ms, to avoid the accumulation of system errors caused by unpredictable delays.

[0166] Optionally, the processing delay of the target signal in the module can be calculated as follows:

[0167] The processing delay of the target signal in the module is calculated based on the signal input timestamp and signal output timestamp of the target signal in the module.

[0168] It should be understood that the clock inside the module will record the signal input timestamp when the signal arrives, and correspondingly, it will record the signal output timestamp when the signal is output. By calculating the difference between the signal input timestamp and the signal output timestamp, the processing delay of the target signal in the module can be obtained.

[0169] Optionally, in order to ensure that subsequent new target signal transmission is not affected by historical delays, after calculating the cumulative delay of the target signal in the module, the cumulative delay of the module also needs to be cleared.

[0170] Specifically, the cumulative delay calculation formula of the target transmission link is:

[0171] T d =T d1 +(T o -T i )+T DB +T DA

[0172] Among them, Td is the cumulative delay of the target signal in the current module; T d1 is the cumulative delay of the target signal in the previous module; T o Output timestamp for the signal; T i Input timestamp for the signal; T DB is the first preset delay; T DA This is the second preset delay.

[0173] It should be noted that if the transmission link has transmission performance test data, the average transmission delay is taken as the cumulative delay of the transmission link. If there is no performance test data for the transmission link, but there are performance index requirements, half of the maximum delay performance requirement can be taken as the cumulative delay. If there is performance test data for a longer transmission link that includes the transmission link:

[0174] During each transmission process, the test delay of the longer transmission link and the test delay of the computable portion of the longer transmission link are obtained. The test delays of the computable portion of the link during multiple transmission processes are then accumulated to obtain the accumulated test delay. The delay difference between the maximum and minimum test delays during each transmission process for the longer transmission link is calculated.

[0175] 1) If the ratio of the accumulated test delay to the delay difference is greater than the set ratio, the difference between the test delay of the longer transmission link and the calculable test delay is calculated for each transmission process, and the average of the differences is used as the accumulated delay of the transmission link;

[0176] 2) If the ratio of the accumulated test delay to the delay difference is less than or equal to the set ratio, the average of the test delays of the longer transmission link is used as the accumulated delay of the transmission link.

[0177] The first preset delay is generally performed in the ARHUD calculation module to facilitate the integration of all information for comprehensive calculation. Of course, due to considerations such as computing power, memory resources, and computational convenience, some or all of the calculations can be performed in other modules.

[0178] It can be understood that for each module, by accurately calculating the cumulative delay of the target signal in the module based on the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay, the delay of the entire target signal in the transmission link can be effectively managed and controlled to avoid display lag or errors caused by delay accumulation.

[0179] Figure 3 The cumulative delay transmission system diagram of the target transmission link provided by the embodiment of the present invention. Figure 3As shown in the figure, the cumulative delay of signal transmission to each module is achieved by using a system with a unified clock. This system can create a signal package for the signal, which includes four attributes: the cumulative delay of the target signal in the previous module, the signal input timestamp, the signal output timestamp, and the cumulative delay. However, if the signal itself is a set of data values ​​that are input and output at the same time, then this set of data also has only one signal input timestamp, signal output timestamp, and cumulative delay.

[0180] When a signal is input to the module, the signal input timestamp is first updated, and when the signal outputs the module, the signal output timestamp is updated, and the cumulative delay is updated. When the ARHUD calculation module receives information from the sensor information fusion module, the vehicle status information module, and the eye tracking sensor module, due to the transmission and calculation processing delays of each link, the dynamic real scene has changed when the ARHUD is rendered. If the delay of the ARHUD calculation module and the rendering module is large, and the real scene changes quickly, the dynamic real scene will change significantly during its processing. At this time, the impact of the delay cannot be ignored. Therefore, the second preset delay must be used for appropriate compensation to eliminate or reduce the impact of the delay.

[0181] Figure 4 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 ;like Figure 4 As shown, the specific steps of the compensation process for the processing delay in the cumulative delay in the present invention are as follows:

[0182] Step 41: Determine whether the current module performs delay compensation.

[0183] If yes, proceed to step 42; if no, end.

[0184] It should be understood that delay compensation refers to updating the target's position in the ARHUD when the cumulative delay exceeds the preset delay. Delay compensation is generally performed in the ARHUD calculation module to integrate all information for comprehensive calculation. Of course, due to considerations such as computing power, memory resources, and computational convenience, some or all calculations can be performed in other modules.

[0185] Step 42: Determine the accumulated delay of the target signal.

[0186] Step 43: When the accumulated delay exceeds the preset delay, the target state parameters of the target object relative to the vehicle are determined.

[0187] Step 44: Display the target object in the ARHUD according to the target state parameters.

[0188] Step 45: Clear the accumulated delay of the current module.

[0189] Step 46: Input the accumulated delay of the current module to the next module.

[0190] It should be understood that the specific implementation methods in each step are already described in Figure 3 The corresponding embodiments are described in detail, and the embodiments of the present application are not described in detail here.

[0191] Figure 5 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 ;like Figure 5 As shown, the specific steps of the process for calculating the position change of a dynamic target in the present invention are as follows:

[0192] Step 51: Determine whether the accumulated delay of the target signal in the current module exceeds the preset delay.

[0193] If yes, go to step 52; if no, go to step 54.

[0194] Step 52: Determine target state parameters of the target object relative to the vehicle.

[0195] Step 53: Determine the target position of the target object in the ARHUD based on the target state parameters of the target object relative to the vehicle.

[0196] Step 54: Output the target position of the target object.

[0197] It should be understood that if the cumulative delay of the target signal in the current module does not exceed the preset delay, the target position of the target object can be directly determined by the target signal value.

[0198] Figure 6 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 ;like Figure 6 As shown, the specific steps of the process of calculating the target state parameters of the target object relative to the vehicle in the present invention are as follows:

[0199] Step 61: Determine whether the accumulated delay of the target signal in the current module exceeds the preset delay.

[0200] If yes, go to step 62; if no, go to step 54.

[0201] Step 62: Calculate the first state parameter of the vehicle based on the accumulated delay, the vehicle's driving data, and the wheelbase.

[0202] Step 63: If the target object is a static target object, collect the second state parameter of the target object; if the target object is a dynamic target object, collect the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object, and calculate the second state parameter of the target object based on the accumulated delay.

[0203] Step 64: According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and the target state parameter of the target object relative to the vehicle is output.

[0204] Furthermore, since the sensor information fusion module can only output the status information of the target object, but cannot output or accurately output the height information of the target object, this will also cause the display content of the target object on the ARHUD to be inconsistent with the real scene, so the relative height of the target object needs to be calculated.

[0205] Figure 7 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 ;like Figure 7 As shown, the data display method based on ARHUD may further include:

[0206] S71. Obtain the vehicle's position and discrete slope information of the area in front of the vehicle based on the map positioning information.

[0207] Among them, map positioning information can be obtained using high-precision map data or in combination with the vehicle's built-in global positioning system; discrete slope information refers to a series of discontinuous slope values.

[0208] It can be understood that obtaining discrete slope information of the road ahead of the vehicle through high-precision maps and the vehicle body's global positioning system can enable the system to perceive road changes in advance so that necessary visual corrections can be made later.

[0209] S72: Determine a first height of the target object relative to the ground based on the position of the vehicle and discrete slope information of the area in front of the vehicle.

[0210] It's understandable that if a target object is located within a sloped area in front of the vehicle, its height relative to the ground will be affected by the slope. By utilizing discrete slope information and the vehicle's position, the target object's initial height relative to the ground can be determined. This helps the system understand the target object's relative position within changing terrain, laying the foundation for subsequent display corrections.

[0211] S73. Determine a second height of the target object relative to the vehicle according to the pitch angle of the vehicle, the first height of the target object, and the target state parameter.

[0212] The vehicle's pitch angle can be obtained through the vehicle status information module, that is, the vehicle's pitch angle (i.e., the angle at which the vehicle tilts forward and backward) is measured in real time through on-board sensors (such as gyroscopes, accelerometers, etc.).

[0213] It can be understood that by considering the pitch angle of the vehicle, the display deviation caused by the change of the vehicle posture can be compensated. On the other hand, by combining the pitch angle of the vehicle, the first height of the target object, and the target state parameters, the second height of the target object relative to the vehicle is calculated, which can ensure that the position of the target object displayed in the ARHUD is consistent with its actual position relative to the vehicle, thereby improving the driver's perception accuracy.

[0214] It should be noted that, in order to facilitate the understanding of the calculation process of the first height of the target object relative to the ground and the second height of the target object relative to the vehicle, the present invention is Figure 9 、 Figure 10 A specific scenario diagram is used for illustration, and the embodiments of the present invention are not described in detail here.

[0215] S74. Display the target object in the ARHUD according to the target state parameters and the second altitude.

[0216] As you can see, the ARHUD adjusts the display position of the target object based on the target state parameters and the second height. This allows the system to effectively respond to changes in road slope and vehicle posture, ensuring that the ARHUD display information always matches the actual environment. This not only enhances the driver's environmental awareness but also significantly improves driving safety in complex road conditions.

[0217] In one possible implementation, in order to enhance driving safety and experience, it is also necessary to add annotation information to the ARHUD so that the driver can more comprehensively perceive the surrounding environment. The specific implementation process is as follows:

[0218] When the vehicle triggers the target warning function, the marking information corresponding to the target warning function is displayed in the ARHUD.

[0219] Among them, the target warning function refers to an important function in ARHUD, which is used to detect and warn the driver in real time about potential dangerous targets or obstacles in front, on the side or in the surrounding area of ​​the vehicle; the annotation information refers to various auxiliary information or prompts presented to the driver in a visual way in ARHUD. The annotation information can be superimposed on the display content of ARHUD in the form of graphics, colors, text or symbols.

[0220] For example, text messages such as "Speed ​​limit 30 km / h" and "Pedestrians ahead" may be displayed in the ARHUD as a prompt.

[0221] It should be noted that the calculation of the orientation of the annotation information can be processed based on the accumulated delay and the calculation of the second height of the vehicle. For example, if the lane departure warning function is enabled when the vehicle crosses the lane without turning on the turn signal, a red warning color can be superimposed on the original lane line position for early warning. If there is a function to mark dangerous targets, the marking will be performed after calculating the second height of the vehicle. More complex function marking can be calculated based on the second state parameters of dynamic targets and static targets, and then processed uniformly with all target information.

[0222] Since the driver's observation point may also change due to seat adjustment or sitting posture changes, the target information displayed on the ARHUD will be inconsistent with the real scene, making it difficult for users to match the target information marked on the ARHUD with the real scene, and unable to achieve timely navigation, warning and other functions, which ultimately leads to users' distrust of the target information marked on the ARHUD and the frequency of ARHUD technology use. Therefore, in order to ensure that the target information displayed on the ARHUD is consistent with the real world, it is crucial to dynamically adjust the display information of the target on the ARHUD.

[0223] In one possible implementation, since the driver may adjust his or her sitting posture or seat position during driving, which may cause the driver's observation point to change, the target state parameters need to be further updated based on the driver's observation point before being displayed. The specific update process is as follows:

[0224] First, the coordinates of the driver's observation point are collected through the eye tracking sensor module; then, based on the coordinates of the observation point and the coordinates of the pre-calibrated observation origin, the position difference between the driver's observation point and the observation origin is calculated; finally, the target state parameters are updated according to the position difference to generate updated target state parameters.

[0225] The coordinates of the pre-calibrated observation origin are the coordinates of the driver's observation point in the standard sitting posture.

[0226] It is understandable that due to adjustments in the driver's sitting posture or seat position, the driver's current observation point shifts relative to the standard observation point. Therefore, the eye tracking sensor module needs to collect the coordinates of the driver's observation point in real time, calculate the position difference between the driver's observation point and the observation origin, and finally adjust the target state parameters in the ARHUD. By dynamically adjusting the target state parameters, the ARHUD can accurately display the target object regardless of changes in the position of the driver's head or eyes.

[0227] Figure 8 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 1 ,like Figure 8As shown in the figure, the road slope may cause the position of the target object displayed on the ARHUD to deviate from its actual position. When the vehicle is driving on a sloped road, the tilt of the vehicle body, that is, the pitch angle of the vehicle body, will visually cause the target object displayed on the ARHUD to appear vertically offset. If the vehicle is traveling uphill, the actual position of the vehicle may be higher than the position of the virtual image marked on the ARHUD; correspondingly, when traveling downhill, the actual position of the vehicle may be lower than the position of the virtual image marked on the ARHUD.

[0228] Figure 9 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 2 .like Figure 9 As shown in the figure, the discrete slope information of the vehicle road is shown. Since the height and pitch angle changes of the target object on the slope are generally difficult to detect accurately by sensors, they can be better calculated using the road surface information. Therefore, the vehicle position and the discrete slope information of the road ahead can be obtained using the map positioning information, such as (0, S0), (x1, S1), ..., (x n , S n ), where (0, S0) indicates that the slope corresponding to the vehicle coordinate system position is S0. Generally, the slope refers to the ratio of the rise (or fall) of the road to its horizontal distance, usually expressed as a percentage or angle. The slope can be positive (uphill) or negative (downhill). For ease of explanation, S is assumed here. i (i is 0 to n) is the angle, and the unit is radians; at x i The road surface height at h i .

[0229] Since we only need to obtain the relative height of the target object to the current vehicle, we can set h0 = 0; then use the linear method to calculate the height of each position point. For each i from 1 to n, the specific calculation formula is:

[0230] h i =h i-1 +(x i -x i-1 )*tan(S i -1)

[0231] Among them, h i For each position point i, the relative height of the target to the current vehicle; x i are different positions in the discrete slope; S is the slope.

[0232] After that, find the interval [x i-1 ,x i ], then linear interpolation is used to calculate the slope. The specific calculation formula is:

[0233] Sx =S i-1 +(xx i-1 ) / (x i -x i-1 )*(S i -S i-1 )

[0234] Among them, S x is the slope at position x; i is the different positions in the discrete slope; S i is the position slope at each position point i.

[0235] Finally, the first height of the target relative to the ground at a given position x can be calculated. The specific calculation formula is:

[0236]

[0237] Among them, h x is the first height of the target relative to the ground at a given position x; S i The position slope at each position point i; x i are different positions in the discrete slope.

[0238] It should be noted that the above calculation process is a calculation method adopted by the present invention, and linear interpolation or other similar mathematical algorithms can be used. In the actual calculation process, different calculation methods can be selected according to system performance, and the embodiment of the present invention does not make specific restrictions here. In addition, for more complex roads, when the vehicle position and the discrete slope information (x i , S i ) is too sparse, you can also use methods such as cubic spline interpolation to process it in advance to reduce (x i , S i ) between the two.

[0239] Furthermore, since the vehicle is on a slope, it is also necessary to calculate the second height of the target object relative to the vehicle based on the pitch angle of the vehicle, the first height of the target object, and the target state parameter.

[0240] Figure 10 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 3 .like Figure 10 As shown in the figure, the target's distance from the current vehicle in the slope direction is shown from a top-down perspective. As can be seen from the figure, the target's distance in the slope direction should be exactly the distance between points A and C. Because the in-vehicle ARHUD only displays objects in front of the vehicle, in practice, the target's longitudinal distance segment X relative to the vehicle can be used instead, i.e., the target's initial height relative to the ground.

[0241] Figure 11 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 4 (a) is a schematic diagram showing the height and pitch angle of the target relative to the vehicle on the ramp, and (b) is a schematic diagram showing the height and pitch angle of the target relative to the vehicle on the ramp. Line segment AD is the second height of the target relative to the vehicle.

[0242] The specific calculation process of the second height of the target object relative to the vehicle is as follows: first, the pitch angle β of the target object relative to the vehicle is calculated; then, the second height of the target object relative to the vehicle can be calculated. The specific calculation formula is:

[0243] β=(S x -α)=(S x -S0)

[0244] Where β is the pitch angle of the target object relative to the vehicle; S x is the slope at the x position; α is the pitch angle of the vehicle, and α=S0.

[0245] AD=(h x -x*tan(α))*cos(α)=h x *cos(α)-x*sin(α)=h x *cos(S0)-x*sin(S0)

[0246] Where AD is the second height of the target relative to the vehicle; h x is the first height of the target object relative to the ground at a given position x; α and S0 are both the pitch angles of the vehicle.

[0247] Figure 12 Scenario diagram of the data display method based on ARHUD provided in an embodiment of the present invention Figure 5 .exist Figure 12In the figure, it is assumed that the coordinates of the driver's pre-calibrated observation point in the eye tracking sensor module coordinate system are (x0, y0, z0). At this time, the ARHUD display content seen by the driver can be matched with the actual static target. When the driver's observation point moves relatively by (dx, dy, dz), in order to keep the display content consistent with the real scene, the position point of the target virtual image should meet the following conditions: the vector from the driver's observation point to the position point of the target virtual image should be the same as the vector from the observation origin to the initial position point of the target virtual image. Therefore, it is necessary to calculate the position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of the pre-calibrated observation origin, and then accumulate the position difference with the original target state parameters to display as the new target position to be rendered. It should be noted that some eye tracking sensor modules output the positions of the left and right eyes respectively, and the average position can be calculated as the observation point position.

[0248] Figure 13 Schematic diagram of the process of the data display method based on ARHUD provided in an embodiment of the present invention Figure 6 .like Figure 13 As shown in the figure, the complete process of the data display method based on ARHUD is shown, and its specific steps include:

[0249] Step 131: During the transmission of the target signal, the position of the target object in the ARHUD is corrected according to the accumulated delay of the target signal.

[0250] Step 132: Correct the position of the target in the ARHUD according to the pitch angle of the vehicle, the first height of the target, and the target state parameter.

[0251] Step 133: Correct the position of the target object in the ARHUD according to the coordinates of the observation point and the coordinates of the pre-calibrated observation origin.

[0252] Step 134: When it is determined that the vehicle triggers the target warning function based on the current position of the target object in the ARHUD, the annotation information corresponding to the target warning function is displayed in the ARHUD.

[0253] It should be understood that the specific implementation methods of each step have been specifically described in the corresponding embodiments above, and the embodiments of this application will not be elaborated here.

[0254] It should be understood that the embodiments of the present application do not limit the execution order of the various steps.

[0255] The following are embodiments of the apparatus of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the apparatus embodiments of the present invention, please refer to the method embodiments of the present invention.

[0256] Figure 14 This is a schematic diagram of the structure of the data display device based on ARHUD provided by an embodiment of the present invention. Figure 14 As shown, the ARHUD-based data display device 140 is applied to a vehicle and includes:

[0257] A first calculation module 141 is configured to calculate the cumulative delay of the target signal from being transmitted to each module when the target signal is transmitted in the target transmission link. The target signal is obtained by collecting signals from the target object by the collection module. The target transmission link is the link from the collection module to the ARHUD.

[0258] The second calculation module 142 is configured to calculate a first state parameter of the vehicle based on the accumulated delay, the vehicle's driving data, and the wheelbase if the accumulated delay of the target signal in any module exceeds a preset delay. The state parameters include a position change parameter and a heading angle change parameter.

[0259] An acquisition module 143 is used to acquire a second state parameter of the target object;

[0260] The third calculation module 144 is used to calculate the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter;

[0261] The display module 145 is used to display the target object in the ARHUD according to the target state parameters.

[0262] Furthermore, the first calculation module 141 is specifically configured to:

[0263] For each module in the target transmission link, the cumulative delay of the target signal in the module is calculated according to the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay and the second preset delay.

[0264] Furthermore, the first calculation module 141 is further configured to:

[0265] The processing delay of the target signal in the module is calculated based on the signal input timestamp and signal output timestamp of the target signal in the module.

[0266] Furthermore, the first calculation module 141 is further configured to:

[0267] For each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0;

[0268] If the subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

[0269] Furthermore, the acquisition module 143 is specifically configured to:

[0270] If the target object is a static target object, collecting a second state parameter of the target object;

[0271] If the target is a dynamic target, the absolute velocity, absolute acceleration, steering angular velocity, and steering angular acceleration of the target are collected;

[0272] A second state parameter of the target object is calculated based on the accumulated delay, the absolute speed, the absolute acceleration, the steering angular speed, and the steering angular acceleration of the target object.

[0273] Furthermore, the third calculation module 144 is specifically configured to:

[0274] According to the first state parameter and the second state parameter, the coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle to generate a target state parameter of the target object relative to the vehicle.

[0275] Furthermore, the ARHUD-based data display device 140 further includes a processing device, which, after calculating the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter, is further configured to:

[0276] Obtain the vehicle's location and the discrete slope information of the area in front of the vehicle based on the map positioning information;

[0277] determining a first height of the target object relative to the ground based on the position of the vehicle and discrete slope information of an area in front of the vehicle;

[0278] determining a second height of the target object relative to the vehicle based on the pitch angle of the vehicle, the first height of the target object, and the target state parameter;

[0279] Accordingly, the display module 145 is specifically configured to:

[0280] The target object is displayed in the ARHUD according to the target state parameters and the second altitude.

[0281] Furthermore, the processing module is also used to clear the accumulated delay of the module.

[0282] Furthermore, the processing module is further configured to:

[0283] The coordinates of the driver's observation point are collected through the eye tracking sensor module;

[0284] Calculate the position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of the pre-calibrated observation origin;

[0285] The target state parameters are updated according to the position difference to generate updated target state parameters.

[0286] Furthermore, the display module 145 is further configured to:

[0287] When the vehicle triggers the target warning function, the marking information corresponding to the target warning function is displayed in the ARHUD.

[0288] The ARHUD-based data display device provided in an embodiment of the present invention can be used to execute the ARHUD-based data display method in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0289] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element. They can also all be implemented in the form of hardware. Some modules can also be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. In addition, these modules can all or partly be integrated together or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0290] Figure 15 This is a schematic diagram of the structure of a vehicle provided by an embodiment of the present invention. Figure 15 As shown, the vehicle 150 may include: a processor 151, a memory 152, and computer-executable instructions stored in the memory 152 and executable on the processor 151. When the processor 151 executes the computer-executable instructions, the ARHUD-based data display method provided in any of the aforementioned embodiments is implemented.

[0291] Optionally, the above-mentioned components of the vehicle 150 can be connected via a system bus.

[0292] The memory 152 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0293] Optionally, the vehicle 150 may also include a communication interface for interacting with other devices.

[0294] It should be understood that the processor 151 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention can be directly implemented as a hardware processor or can be implemented by a combination of hardware and software modules in the processor.

[0295] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. System buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the diagram uses a single thick line, but this does not imply a single bus or type of bus. Memory may include random access memory (RAM) and non-volatile memory (NVM), such as at least one disk drive.

[0296] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-mentioned method embodiments. The aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0297] The vehicle provided in the embodiment of the present invention can be used to execute the ARHUD-based data display method provided in any of the above-mentioned method embodiments. The implementation principles and technical effects are similar and will not be repeated here.

[0298] An embodiment of the present invention provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer executes the above-mentioned ARHUD-based data display method.

[0299] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0300] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0301] An embodiment of the present invention also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, the above-mentioned ARHUD-based data display method can be implemented.

[0302] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof, which is limited only by the appended claims.

Claims

1. A data display method based on ARHUD, characterized in that: Applied to a vehicle, the method comprises: When a target signal is transmitted in a target transmission link, calculating the cumulative delay of the target signal to each module, wherein the target signal is obtained by collecting signals of a target object by an acquisition module, and the target transmission link is a link from the acquisition module to an augmented reality head-up display (ARHUD); If the cumulative delay of the target signal in any module exceeds a preset delay, calculating the first state parameter of the vehicle according to the cumulative delay, the driving data of the vehicle, and the wheelbase, the state parameter including a position change parameter and a heading angle change parameter; Acquiring a second state parameter of the target object; Calculating a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter; Displaying the target object in the ARHUD according to the target state parameters; The step of calculating the cumulative delay of transmitting the target signal to each module includes: For each module in the target transmission link, calculate the cumulative delay of the target signal in the module based on the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay, and the second preset delay; Wherein, for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0; If subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

2. The method according to claim 1, characterized in that The method further comprises: The processing delay of the target signal in the module is calculated according to the signal input timestamp and the signal output timestamp of the target signal in the module.

3. The method according to claim 1 or 2, characterized in that The obtaining of the second state parameter of the target object includes: If the target object is a static target object, collecting the second state parameter of the target object; If the target object is a dynamic target object, the absolute velocity, absolute acceleration, steering angular velocity and steering angular acceleration of the target object are collected; The second state parameter of the target object is calculated according to the accumulated delay, the absolute velocity of the target object, the absolute acceleration, the steering angular velocity, and the steering angular acceleration.

4. The method according to claim 1 or 2, characterized in that Calculating a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter includes: According to the first state parameter and the second state parameter, a coordinate system of the second state parameter is converted into a coordinate system corresponding to the vehicle, and a target state parameter of the target object relative to the vehicle is generated.

5. The method according to claim 1 or 2, characterized in that After calculating the target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter, the method further includes: Obtaining the position of the vehicle and discrete slope information of the area in front of the vehicle based on the map positioning information; determining a first height of the target object relative to the ground based on the position of the vehicle and discrete slope information of an area in front of the vehicle; determining a second height of the target relative to the vehicle based on the pitch angle of the vehicle, the first height of the target, and the target state parameter; Accordingly, displaying the target object in the ARHUD according to the target state parameter includes: The target object is displayed in the ARHUD according to the target state parameter and the second height.

6. The method according to claim 1 or 2, characterized in that The method further comprises: Clear the accumulated delay of the module.

7. The method according to claim 1 or 2, characterized in that The method further comprises: The coordinates of the driver's observation point are collected through the eye tracking sensor module; Calculating a position difference between the driver's observation point and the observation origin based on the coordinates of the observation point and the coordinates of a pre-calibrated observation origin; The target state parameter is updated according to the position difference to generate an updated target state parameter.

8. The method according to claim 1 or 2, characterized in that The method further comprises: When the vehicle triggers the target warning function, the annotation information corresponding to the target warning function is displayed in the ARHUD.

9. A data display device based on ARHUD, characterized in that: Applied to vehicles, including: a first calculation module, configured to calculate a cumulative delay of a target signal transmitted to each module when the target signal is transmitted in a target transmission link, wherein the target signal is obtained by collecting signals from a target object by an acquisition module, and the target transmission link is a link from the acquisition module to the ARHUD; a second calculation module, configured to calculate a first state parameter of the vehicle based on the accumulated delay, the driving data of the vehicle, and the wheelbase, if the accumulated delay of the target signal in any module exceeds a preset delay, the state parameter including a position change parameter and a heading angle change parameter; an acquisition module, configured to acquire a second state parameter of the target object; a third calculation module, configured to calculate a target state parameter of the target object relative to the vehicle based on the first state parameter and the second state parameter; A display module, configured to display the target object in the ARHUD according to the target state parameters; The first calculation module is specifically configured to: For each module in the target transmission link, calculate the cumulative delay of the target signal in the module based on the cumulative delay of the target signal in the previous module, the processing delay in the module, the first preset delay, and the second preset delay; Wherein, for each module in the target transmission link, if the previous module cannot calculate the processing delay of the target signal in the previous module, the first preset delay corresponding to the module is a positive number greater than 0; If subsequent modules of the module cannot calculate the processing delay, and the next module cannot calculate the processing delay of the target signal in the next module, the second preset delay corresponding to the module is a positive number greater than 0.

10. A vehicle comprising: A processor, a memory, and a computer-executable instruction stored in the memory and executable on the processor, wherein the processor executes the computer-executable instruction to implement the ARHUD-based data display method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the ARHUD-based data display method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it is used to implement the data display method based on ARHUD according to any one of claims 1 to 8.

Images