Method for generating augmented reality symbols for vehicle-mounted ar-hud

Abstract

The method for generating an augmented reality symbol of a vehicle-mounted AR-HUD of the application solves the technical problem of low image display function of the current vehicle-mounted AR-HUD. The method comprises generating a first symbol for lane departure warning display, specifically: when the ADAS auxiliary driving system detects that the current wheel is pressed against the line and produces a sound alarm, the lane line equation is obtained through the ADAS auxiliary driving system, and the first area for displaying the first symbol is determined through the lane line equation; no display is performed when the sound alarm is over; the auxiliary driving system is provided with a target type for classification; the type of the display target is input into the ADAS auxiliary driving system, the three-dimensional space coordinates of the type of the input target are determined through the ADAS auxiliary driving system according to the current position information of the vehicle, the three-dimensional space coordinates are converted into two-dimensional pixel coordinates, the type of the input target is colored in the two-dimensional pixel coordinates, and the type of the input target is displayed in a second color, and the distance of the real-time display output is displayed.

Description

Technical Field

[0001] This invention belongs to the technical field of in-vehicle AR-HUD image display methods, and particularly relates to a method for generating augmented reality symbols for in-vehicle AR-HUD. Background Technology

[0002] In-vehicle AR-HUDs have been widely used in recent years. The displays on the windshield are all conventional. Lane departure warnings are provided by the ADAS driver assistance system, but there are no specific symbols displayed. For example, lane changes and blind spot walls are not displayed, and the driver needs to check the rearview mirror for confirmation. Summary of the Invention

[0003] In view of this, the present invention provides a method for generating augmented reality symbols for in-vehicle AR-HUD, which at least partially solves the technical problem of the current low image display function of in-vehicle AR-HUD.

[0004] A method for generating augmented reality symbols for an in-vehicle AR-HUD is provided. The in-vehicle system is equipped with an ADAS (Advanced Driver Assistance System), which includes LDW (Label Wildcard), ACC (Advanced Accelerated Display), ALC (Advanced Control System), BSD (Advanced Boots Controller), and NOS (Noise Reduction System) functions. The method includes:

[0005] S101: Includes generating a first symbol for lane departure warning display. Specifically: when the ADAS driver assistance system detects that the current wheel is crossing the lane line and generates an audible warning, the ADAS driver assistance system obtains the lane line equation and determines a first area for displaying the first symbol. The lane line is covered with a first color in the first area for real-time display, and is not displayed when the audible warning ends.

[0006] S102: ADAS (Advanced Driver Assistance Systems) classifies targets by type;

[0007] Input the type of target to be displayed into the ADAS driver assistance system.

[0008] Based on the vehicle's current location information, the ADAS (Advanced Driver Assistance System) determines the three-dimensional spatial coordinates of the type of the input target, converts the three-dimensional spatial coordinates into two-dimensional pixel coordinates, colors the type of the input target within the two-dimensional pixel coordinates, displays it with a second color, and displays the output distance in real time.

[0009] The beneficial effects of the present invention are as follows:

[0010] By displaying different colors in different areas at different locations, the overall functionality of the AR-HUD is improved, and enhanced symbols are displayed, allowing the driver to view the graphic display on the windshield without looking down at the instrument panel. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is the lane departure warning display symbol of the present invention;

[0013] Figure 2 The display symbol showing the distance between the preset target location and the current vehicle;

[0014] Figure 3 The indicator for lane changes;

[0015] Figure 4 The symbol for intelligent navigation is displayed;

[0016] Figure 5 To generate a symbol display for the blind spot wall. Detailed Implementation

[0017] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0018] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0019] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0020] The present invention provides a method for generating augmented reality symbols for an in-vehicle AR-HUD. The vehicle is equipped with an ADAS (Advanced Driver Assistance System) that includes LDW (Lane Departure Warning), ACC (Adaptive Cruise Control), ALC (Automatic Lane Change), BCD (Blind Spot Detection), and NOC (Navigate on Cord). This invention enhances the functionality of the ADAS by generating and displaying AR-HUD symbols, thereby improving the usability of the in-vehicle AR-HUD device or product. The method includes:

[0021] S101: Includes generating a first symbol for lane departure warning display. Specifically: When the ADAS (Advanced Driver Assistance System) detects the current wheel crossing the lane line and generates an audible warning, the ADAS obtains the lane line equation and determines the first area for displaying the first symbol. The lane line is covered with a first color in the first area for real-time display, and is not displayed when the audible warning ends.

[0022] ADAS (Advanced Driver Assistance Systems) includes a forward-facing camera and millimeter-wave radar, among which:

[0023] The fitting equations for the left and right lane lines of the current road surface determined by the driver assistance system are: Y=C0+C1*X+1 / 2*C2*X^2+1 / 6*C3*X^3, where C0, C1, C2 and C3 are determined based on real-time image data of the left and right lanes, and C1 to C3 are the equation system.

[0024] Definition: The positive direction of the coordinate system is defined as follows: the direction of the vehicle's forward movement is the positive X-axis, upward is the positive Z-axis, and the left side of the direction of the vehicle's forward movement is the positive Y-axis. Furthermore, the origin of the vehicle's coordinate system is the center of the rear axle. In this case, the full scale of the ground is a constant value, for example: Zw = -0.339119, which is related to the tire diameter.

[0025] For the equations of the left and right lane lines, within the area Xw from ViewStart (the starting point of the lane line in the first area) to ViewEnd (the ending point of the lane line in the first area), calculate Yw by taking points at 1-meter intervals for Xw, and obtain the point set of Xw and Yw;

[0026] Call the AR fusion transformation function, such as the API, to determine the set of coordinates of all pixels within the first region of the AR-HUD product, such as... Figure 1 As shown, the set is colored with a first color, such as red, based on a given pixel width (e.g., 50), and the first color is not displayed when the alarm sound ends.

[0027] S102: ADAS (Advanced Driver Assistance System) classifies targets by type, such as cars, bicycles, pedestrians, or others. The type of target displayed is input into the ADAS system, for example, pedestrians. Specifically:

[0028] Based on the vehicle's current location information, the ADAS (Advanced Driver Assistance System) determines the three-dimensional spatial coordinates of the input target type and converts them into two-dimensional pixel coordinates. The type of the input target is colored within the two-dimensional pixel coordinates and displayed using a secondary color. The output distance is also displayed in real time. For example, the ADAS obtains the coordinates of the bottom of the left and rear wheels, the x and y coordinates of the rear axle center, and the width of the vehicle (L). Figure 2 As shown, the distance between the current person's position and the car is displayed in the second area using T-shaped or triangle symbols, such as...

[0029] 1. The coordinates of the bottom of the left rear wheel are (X, Y+L / 2, fixed ground coordinates -0.339119).

[0030] 2. The coordinates of the bottom of the right rear wheel are (X, Yo-L / 2, fixed ground coordinates -0.339119);

[0031] 3. Call the AR fusion conversion function to convert the bottom coordinates of the left and right rear wheels into second pixel coordinates;

[0032] 4. The type of input target is colored and displayed in the second pixel coordinates, and displayed with a preset geometric shape. For example, the following symbol is drawn: the pixel coordinates of the bottom of the left rear wheel and the bottom of the right rear wheel are used as the upper left and upper right vertices of the trapezoid symbol. The height of the trapezoid is 34 pixels. The lower left fixed point of the trapezoid is 20 pixels to the left of the upper left vertex, and the lower right fixed point is 20 pixels to the right of the upper right vertex. The upper vertex of the triangle is 5 pixels lower than the center of the bottom of the car.

[0033] As a specific implementation method provided in this case, it also includes a method for displaying information when changing lanes, which includes:

[0034] Lane change notifications, such as changing to the left lane, will be displayed in bold. Specifically...

[0035] When the ADAS driver assistance system receives an automatic lane change command from remote control, it determines the direction of the lane change, and obtains the lane line equation (same as the equation in S101) by detecting the current wheel crossing the line in the ADAS driver assistance system. It then determines the third pixel coordinate area of ​​the lane line for the lane change direction, colors the third pixel coordinate area with the third color, and displays it.

[0036] There are preset lane change trajectory lines. When changing lanes, the lane change trajectory line is displayed inside the third area after being colored with the third color, such as... Figure 3 As shown.

[0037] Furthermore, this also includes methods for blind spot warnings, such as... Figure 5 As shown, it includes:

[0038] When the ADAS (Advanced Driver Assistance System) detects other vehicles passing to the left or right of the current vehicle, it creates a blind spot wall above the lane lines of the other vehicles and displays it in a fourth color. Specifically: Figure 5 As shown, the blind spot wall is rectangular, including points 1, 2, 3, and 4, corresponding to... Figure 5 The figures are labeled 1, 2, 3, and 4, where:

[0039] A1: The coordinates of point 1 are Xw1 and Yw1. The dynamic view start point value ViewRangeStart output by the ADAS assisted driving system is used as the starting point of the lane line, which is the initial value of Xw1 of point 1. Then, the Yw1 of point 1 satisfies:

[0040] Yw1=C0+C1*Xw1+1 / 2*C2*Xw1^2+1 / 6*C3*Xw1^3, where C0, C1, C2 and C3 are the equation system, which is output in real time by the ADAS assisted driving system;

[0041] A2: Substitute the equation of point 1 into the fusion transformation function to determine the first pixel coordinate point of the lane line point set;

[0042] The steps to determine the coordinates of the second pixel corresponding to point 2 include:

[0043] B1: The coordinates of point 2 are Xw2 and Yw2. The dynamic view range endpoint value ViewRangeEnd output by the ADAS driver assistance system is used as the lane line endpoint Xw2, which is the lane line endpoint.

[0044] Yw2=C0+C1*X w2+1 / 2*C2*X w2^2+1 / 6*C3*X w2^3;

[0045] B2: Substitute the equation of point 2 into the fusion transformation function to determine the second pixel coordinate point of the lane line point set;

[0046] The steps to determine the coordinates of the third pixel corresponding to point 3 include:

[0047] C1: The coordinates of point 3 are Xw3 and Yw3. The pixel coordinates of point 3 are equal to the pixel coordinates of point 1 moved upward by 50mm, that is, Yw3 = Yw1 + 50mm.

[0048] C2: Substitute the equation of point 3 into the fusion transformation function to determine the coordinates of the third pixel in the lane line point set;

[0049] The steps to determine the coordinates of the fourth pixel corresponding to point 4 include:

[0050] D1: The coordinates of point 4 are Xw4 and Yw4. The pixel coordinates of point 4 are equal to the pixel coordinates of point 2 moved upward by 50mm, that is, Yw4 = Yw2 + 50mm.

[0051] D2: Substitute the equation of point 4 into the fusion transformation function to determine the coordinates of the fourth pixel in the lane line point set;

[0052] The outline of the four pixel coordinates is colored with the fourth color and displayed in the fourth area.

[0053] As a specific implementation method provided in this case, a method for obtaining the intelligent navigation guidance line equation is also included, which includes:

[0054] The equation for the real-world lane guide line is: Y = A0 + A1*X + A2*X 2 +A3*X 3 +A4*X 4 +A5*X 5 Where A0, A1, A2, A3, A4 and A5 are the equation coefficients output by the ADAS driver assistance system;

[0055] The starting and ending points of the X value are the starting and ending points of the navigation dynamic field of view in the ADAS driver assistance system. The full scale of the ground is a constant value, for example, Zw = -0.339119, and is related to the tire diameter.

[0056] The equation of the real-world lane guide line is substituted into the fusion transformation function to determine the fifth region;

[0057] In the existing intelligent navigation guidance line equation, within the area Xw from ViewStart (the starting point of the fifth area) to ViewEnd (the ending point of the fifth area), Yw is calculated by taking points at 1-meter intervals from Xw, thus obtaining the point set of Xw and Yw;

[0058] Call the AR fusion transformation function, such as API, which is a set of coordinates of all pixels in the fifth region of the AR-HUD. The set is colored with a fifth color according to a given pixel width (e.g., 50), such as blue. The fifth color is not displayed when the smart navigation function is exited.

[0059] The five scenarios mentioned above are enhanced by displaying five different colors in five different areas at different locations, thus improving the overall usability of the AR-HUD. Enhanced symbols are displayed, allowing the driver to view the graphic display on the windshield without having to look down at the instrument panel.

[0060] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for generating augmented reality symbols for an in-vehicle AR-HUD, wherein the in-vehicle system is equipped with an ADAS (Advanced Driver Assistance System), the ADAS system having LDW (Low Distance Warning), ACC (Adaptive Cruise Control), ALC (Advanced Driver Assistance Control), BCD (Browser Control Center), NOC (Noise of Conformity), and target type classification functions, characterized in that, The method includes: S101: Includes generating a first symbol for lane departure warning display. Specifically: when the ADAS driver assistance system detects that the current wheel is crossing the lane line and generates an audible warning, the ADAS driver assistance system obtains the lane line equation and determines a first area for displaying the first symbol. The lane line is covered and displayed in the first area with a first color, and is not displayed when the audible warning ends. S102: Input the type of target to be displayed into the ADAS driver assistance system; Based on the vehicle's current location information, the ADAS (Advanced Driver Assistance System) determines the three-dimensional spatial coordinates of the type of the input target, converts the three-dimensional spatial coordinates into two-dimensional pixel coordinates, colors the type of the input target within the two-dimensional pixel coordinates, displays it with a second color, and displays the output distance in real time.

2. The method according to claim 1, characterized in that, It also includes methods for displaying information when changing lanes, including: When the ADAS driver assistance system receives an automatic lane change command from remote control, it determines the direction of the lane change and the third pixel coordinate area of ​​the lane line in the lane change direction. The third pixel coordinate area is then colored with a third color and displayed. The preset lane change trajectory line is placed inside the area colored by the third color when changing lanes.

3. The method according to claim 1, characterized in that, It also includes methods for blind spot warning, which include: When the ADAS (Advanced Driver Assistance System) detects other vehicles passing to the left or right of the current vehicle, a blind spot wall is generated above the lane lines of the other vehicles and displayed in a fourth color, where: Calculate the four vertices of the generated blind zone wall, including point 1, point 2, point 3, and point 4, where: The steps to determine the coordinates of the first pixel corresponding to point 1 include: A1: The coordinates of point 1 are Xw1 and Yw1. The dynamic view start point value ViewRangeStart output by the ADAS assisted driving system is used as the starting point of the lane line, which is the initial value of Xw1 of point 1. Then, the Yw1 of point 1 satisfies: Yw1=C0+C1*Xw1+1 / 2*C2*Xw1^2+1 / 6*C3*Xw1^3, where C0, C1, C2 and C3 are the equation system, which is output in real time by the ADAS assisted driving system; A2: Substitute the equation of point 1 into the fusion transformation function to determine the first pixel coordinate point of the lane line point set; The steps to determine the coordinates of the second pixel corresponding to point 2 include: B1: The coordinates of point 2 are Xw2 and Yw2. The dynamic view range endpoint value ViewRangeEnd output by the ADAS driver assistance system is taken as the lane line endpoint Xw2. Then, the Yw2 of point 2 satisfies: Yw2=C0+C1*X w2+1 / 2*C2*X w2^2+1 / 6*C3*X w2^3; B2: Substitute the equation of point 2 into the fusion transformation function to determine the second pixel coordinate point of the lane line point set; The steps to determine the coordinates of the third pixel corresponding to point 3 include: C1: The coordinates of point 3 are Xw3 and Yw3. The pixel coordinates of point 3 are equal to the pixel coordinates of point 1 moved upward by 50mm, that is, Yw3 = Yw1 + 50mm. C2: Substitute the equation of point 3 into the fusion transformation function to determine the coordinates of the third pixel in the lane line point set; The steps to determine the coordinates of the fourth pixel corresponding to point 4 include: D1: The coordinates of point 4 are Xw4 and Yw4. The pixel coordinates of point 4 are equal to the pixel coordinates of point 2 moved upward by 50mm, that is, Yw4 = Yw2 + 50mm. D2: Substitute the equation of point 4 into the fusion transformation function to determine the coordinates of the fourth pixel in the lane line point set; The outline of the four pixel coordinates is colored or filled with a fourth color to serve as a blind zone wall and is then displayed.

4. The method according to claim 1, characterized in that, It also includes a method for obtaining the intelligent navigation guideline equation, which determines the fifth region, wherein: The equation for the real-world lane guide line is: Y = A0 + A1*X + A2*X 2 +A3*X 3 +A4*X 4 +A5*X 5 Where A0, A1, A2, A3, A4 and A5 are the equation coefficients output by the ADAS driver assistance system; The start and end points of the X value are the start and end points of the navigation dynamic field of view in the ADAS assisted driving system. The full scale of the ground is a constant value and is related to the tire diameter. The equation of the real-world lane guide line is substituted into the fusion transformation function to determine the fifth region; In the intelligent navigation guidance line equation, the fifth region is from the real-view starting point ViewStart to the real-view ending point ViewEnd. Yw is calculated by taking points at 1-meter intervals in Xw to obtain the point set of Xw and Yw. The intelligent navigation guide line equation is substituted into the AR fusion transformation function to determine the set of coordinates of all pixels in the fifth region of the AR-HUD. The set is colored with the fifth color according to the given pixel width, and the fifth color is not displayed when the intelligent navigation function is exited.

5. The method according to claim 1, characterized in that, ADAS (Advanced Driver Assistance Systems) include a forward-looking camera and millimeter-wave radar. The methods in S101 include: The fitting equations for the left and right lane lines of the current road surface are determined by the driver assistance system: Y=C0+C1*X+1 / 2*C2*X^2+1 / 6*C3*X^3; The positive direction of the coordinate system is defined as follows: the positive direction of the vehicle's forward movement is the X-axis, upward is the Z-axis, and the left side of the vehicle's forward movement is the Y-axis. Furthermore, the origin of the vehicle's coordinate system is the center of the rear axle, and the full scale of the ground is a constant value, which is related to the tire diameter. For the equations of the left and right lane lines, in the area from the start point of the lane line to the end point of the lane line in the first region, calculate Yw by taking points at 1-meter intervals for Xw, and obtain the point set of Xw and Yw; Call the AR fusion transformation function to obtain a set of coordinates of all pixels in the first region of the AR-HUD. The set is colored with the first color according to the given pixel width, and the first color is not displayed when the alarm sound ends.

6. The method according to claim 1, characterized in that, The method in S102 includes: The ADAS driver assistance system obtains the coordinates of the bottom of the left and rear wheels and the x and y coordinates of the center of the rear axle of the vehicle, and L is the width of the vehicle. The coordinates of the bottom of the left rear wheel satisfy: X, Y+L / 2, and the fixed coordinates on the ground are fixed values; The coordinates of the bottom of the right rear wheel satisfy: X, Yo-L / 2, and the fixed coordinates on the ground are fixed values; Call the AR fusion transformation function to convert the bottom coordinates of the left and right rear wheels into second pixel coordinates; The type of the input target is colored and displayed in the second pixel coordinates using a second eye method, and is displayed with a preset geometric shape.

Images