Display method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of navigation, and provides a display method, electronic equipment and a computer readable storage medium, and the method comprises the following steps: obtaining a first to-be-displayed coordinate, a left side lane line point set and a right side lane line point set; based on the first to-be-displayed coordinate, a to-be-converted image is output, the to-be-converted image comprises a first left side reference point, a second left side reference point, a first right side reference point, a second right side reference point and the first to-be-displayed coordinate, and the first longitudinal distance and the second longitudinal distance between the first left side reference point and the first right side reference point are larger than the to-be-displayed longitudinal distance. The third longitudinal distance and the fourth longitudinal distance between the second left reference point and the second right reference point are smaller than the longitudinal distance to be displayed. A first projection image is determined and displayed, the first projection image comprises projection coordinates, and the projection coordinates are used for guiding the intelligent device to move to an area corresponding to the first to-be-displayed coordinates. The projection range of the information points can be adjusted, and the driving safety and convenience are enhanced.

Description

Technical Field

[0001] This application relates to the field of navigation technology, and in particular, to a display method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of automotive technology, driving assistance systems have increasingly become a key factor in enhancing driving safety and experience. Among them, the dual-focus AR-HUD (Augmented Reality Head-Up Display) as an advanced technology emerging in recent years can, through the dual-focus design and AR technology, display driving information in layers and integrate it with the actual road environment, improving driving safety and intuitiveness.

[0003] However, when a POI (Point of Interest), such as a destination or a waypoint, is outside the field of view (FOV) of the AR-HUD, the user cannot see the position of the POI within the visual range of the AR-HUD, thus affecting the timeliness and accuracy of driving decisions. Summary of the Invention

[0004] Embodiments of this application provide a display method, an electronic device, and a computer-readable storage medium. It can adjust the projection range of the POI, avoid delays or errors in driving decisions caused by information loss, further improve the driving experience, and enhance driving safety and convenience.

[0005] In a first aspect, the present application provides a display method applied to an intelligent device. The method includes: obtaining a first coordinate to be displayed, and obtaining a left lane line point set and a right lane line point set. The left lane line point set includes a plurality of consecutive coordinate points corresponding to the left boundary line on the to-be-traveled route, and the right lane line point set includes a plurality of consecutive coordinate points corresponding to the right boundary line on the to-be-traveled route. Based on the first coordinate to be displayed, an image to be converted is output. The processing input of the image to be converted includes a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance between the first left reference point and the current position point and the second longitudinal distance between the first right reference point and the current position point are both greater than the to-be-displayed longitudinal distance between the first coordinate to be displayed and the current position point. The third longitudinal distance between the second left reference point and the current position point and the fourth longitudinal distance between the second right reference point and the current position point are both less than the to-be-displayed longitudinal distance. The first left reference point and the second left reference point are two adjacent coordinate points in the left lane line point set, and the first right reference point and the second right reference point are two adjacent coordinate points in the right lane line point set. A first projection image is determined according to the image to be converted; the first projection image is displayed within a preset projection range. The first projection image includes projection coordinates, and the projection coordinates are used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0006] In some embodiments, the image to be converted is an image in a first coordinate system. Determining the first projection image according to the image to be converted includes: determining a left information point according to the first left reference point, the second left reference point, and the first coordinate to be displayed. The abscissa corresponding to the left information point is the first horizontal coordinate, and the ordinate corresponding to the left information point is the first longitudinal coordinate. A right information point is determined according to the first right reference point, the second right reference point, and the first coordinate to be displayed. The abscissa corresponding to the right information point is the second horizontal coordinate, and the ordinate corresponding to the right information point is the second longitudinal coordinate. A to-be-converted coordinate is determined according to the left information point and the right information point. The abscissa corresponding to the to-be-converted coordinate is the mean value of the first horizontal coordinate and the second horizontal coordinate, and the ordinate corresponding to the to-be-converted coordinate is the mean value of the first longitudinal coordinate and the second longitudinal coordinate. The to-be-converted coordinate is converted to a second coordinate system to obtain projection coordinates. The second coordinate system is a coordinate system different from the first coordinate system. The first projection image is determined according to the projection coordinates.

[0007] In some embodiments, determining a left information point according to a first left reference point, a second left reference point, and a first coordinate to be displayed includes: determining a first vector according to the first left reference point and the second left reference point, where the starting point of the first vector is the second left reference point and the ending point of the first vector is the first left reference point; determining a second vector according to the first coordinate to be displayed and the second left reference point, where the starting point of the second vector is the second left reference point and the ending point of the second vector is the first coordinate to be displayed; determining the projection of the second vector in the direction of the first vector as a third vector; and determining the ending point of the third vector as the left information point.

[0008] In some embodiments, determining a right information point according to a first right reference point, a second right reference point, and a first coordinate to be displayed includes: determining a fourth vector according to the first right reference point and the second right reference point, where the starting point of the fourth vector is the second right reference point and the ending point of the fourth vector is the first right reference point; determining a fifth vector according to the first coordinate to be displayed and the second right reference point, where the starting point of the fifth vector is the second right reference point and the ending point of the fifth vector is the first coordinate to be displayed; determining the projection of the fifth vector in the direction of the fourth vector as a sixth vector; and determining the ending point of the sixth vector as the right information point.

[0009] In some embodiments, the first coordinate to be displayed is a coordinate point different from the second coordinate to be displayed, and the display serial number corresponding to the second coordinate to be displayed is the display serial number adjacent to and before the display serial number corresponding to the first coordinate to be displayed.

[0010] In some embodiments, the driving distance between the current position point and the first coordinate to be displayed is a first driving distance, and obtaining the first coordinate to be displayed includes: if the first driving distance is less than a first preset distance and the longitudinal distance to be displayed is greater than or equal to a second preset distance, then obtaining the first coordinate to be displayed, where the first preset distance is greater than the second preset distance.

[0011] In some embodiments, if the driving distance between the current position point and the first coordinate to be displayed is less than a third preset distance and the longitudinal distance to be displayed is greater than or equal to a second preset distance, then a second projection image is displayed within a preset projection range, where the second projection image includes enhanced projection coordinates for characterizing that the driving distance between the current position point and the position point corresponding to the first coordinate to be displayed is less than the third preset distance, and the coordinate information of the enhanced projection coordinates is the same as the coordinate information of the projection coordinates, and the third preset distance is greater than the second preset distance and less than the first preset distance.

[0012] In some embodiments, if the vertical distance to be displayed is less than the second preset distance, a third projection image is displayed within the preset projection range. The third projection image does not include projection coordinates and / or enhanced projection coordinates, and is used to indicate that the intelligent device has moved to the area corresponding to the first coordinate to be displayed.

[0013] In some embodiments, the visual field range corresponding to the current position point is the first visual field range. When the intelligent device is located at the current position point, the first field of view angle corresponding to the preset projection range is the same as the second field of view angle corresponding to the first visual field range. A fourth projection image is displayed within the preset projection range, and the fourth projection image includes the augmented reality coordinates corresponding to the first coordinate to be displayed.

[0014] In a second aspect, the present application provides a display device, including:

[0015] An acquisition module: configured to acquire the first coordinate to be displayed, and acquire a left lane line point set and a right lane line point set. The left lane line point set includes a plurality of consecutive coordinate points corresponding to the left boundary line on the to-be-traveled route, and the right lane line point set includes a plurality of consecutive coordinate points corresponding to the right boundary line on the to-be-traveled route.

[0016] A processing module, configured to output a to-be-converted image based on the first coordinate to be displayed. The to-be-converted image includes a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance and the second longitudinal distance between the first left reference point and the first right reference point and the current position point are greater than the vertical distance to be displayed between the first coordinate to be displayed and the current position point. The third longitudinal distance and the fourth longitudinal distance between the second left reference point and the second right reference point and the current position point are less than the vertical distance to be displayed. The first left reference point and the second left reference point are two adjacent coordinate points in the left lane line point set, and the first right reference point and the second right reference point are two adjacent coordinate points in the right lane line point set.

[0017] The processing module is further configured to determine a first projection image according to the to-be-converted image and display the first projection image within the preset projection range. The first projection image includes projection coordinates, and the projection coordinates are used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0018] In a third aspect, the present application provides a chip, which is used to execute the method in any one of the above first aspects.

[0019] In a fourth aspect, the present application provides an electronic device, including a processor and a memory. The processor is configured to execute a computer program stored in the memory to implement the method in any one of the above first aspects. Alternatively, the electronic device includes a chip as in the third aspect.

[0020] Fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the method in any one of the above first aspects.

[0021] Sixth aspect, an embodiment of the present application provides a computer program product including a computer program / instructions, which when executed by a processor cause an electronic device to execute the method in the above first aspect and any possible implementation manner thereof.

[0022] In the technical solution provided by the present application, after determining the first coordinate to be displayed, the intelligent device can determine the image to be converted through the left lane line point set and the right lane line point set corresponding to the to-be-traveled route. The image to be converted includes a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance and the second longitudinal distance between the first left reference point and the first right reference point and the current position point are greater than the to-be-displayed longitudinal distance between the first coordinate to be displayed and the current position point, and the third longitudinal distance and the fourth longitudinal distance between the second left reference point and the second right reference point and the current position point are less than the to-be-displayed longitudinal distance. Finally, a first projection image is determined according to the image to be converted, and the first projection image is displayed within a preset projection range. In the technical solution provided by the embodiment of the present application, the intelligent device can project the first coordinate to be displayed that was originally not within the field of view angle into the field of view angle. Thereby, the display strategy of information points is optimized, the problem that traditional AR-HUD cannot display key information points under the limitation of the field of view angle is solved, and the practicability of traditional AR-HUD is improved. Specifically, by adjusting the projection range, it is ensured that the driver can timely obtain key navigation information, avoiding delays or errors in driving decisions caused by information loss, further improving the driving experience, and enhancing driving safety and convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 It is a schematic software architecture diagram of a display method provided by an embodiment of the present application;

[0025] Figure 2 It is a schematic flowchart of obtaining a projection image of a display method provided by an embodiment of the present application;

[0026] Figure 3It is a schematic diagram of a coordinate data structure to be displayed in a display method provided by an embodiment of the present application;

[0027] Figure 4 It is a schematic diagram of a driving route to be traveled in a display method provided by an embodiment of the present application;

[0028] Figure 5 It is a schematic diagram of a vehicle coordinate system in a display method provided by an embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of the relative distribution of longitude and latitude and the vehicle coordinate system in a display method provided by an embodiment of the present application;

[0030] Figure 7 It is a schematic diagram of an image to be converted in a display method provided by an embodiment of the present application;

[0031] Figure 8 It is a schematic diagram of a boundary line in a display method provided by an embodiment of the present application;

[0032] Figure 9 It is another schematic diagram of an image to be converted in a display method provided by an embodiment of the present application;

[0033] Figure 10 It is still another schematic diagram of an image to be converted in a display method provided by an embodiment of the present application;

[0034] Figure 11 It is a schematic diagram of a first projected image in a display method provided by an embodiment of the present application;

[0035] Figure 12 It is a schematic diagram of an enhanced projection coordinate in a display method provided by an embodiment of the present application;

[0036] Figure 13 It is a schematic diagram of a display process in a display method provided by an embodiment of the present application;

[0037] Figure 14 It is a schematic diagram of an augmented reality coordinate in a display method provided by an embodiment of the present application;

[0038] Figure 15 It is a schematic diagram of a display icon outside the field of view angle in a display method provided by an embodiment of the present application;

[0039] Figure 16 It is a schematic diagram of the structure of a display device provided by an embodiment of the present application;

[0040] Figure 17 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0041] In the following description, specific details such as specific system architectures, technologies, etc. are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of the present application.

[0042] It should be understood that when used in the specification and claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0043] It should also be understood that the term "and / or" used in the specification and claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0044] As used in the specification and claims of the present application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0045] In addition, in the description of the specification and claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0046] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0047] A HUD (Head-Up Display) is a virtual image formed by projecting an image onto the windshield of a vehicle through an optical projection technology, using a projection source such as a liquid crystal or DLP (Digital Light Processing) display. This allows the driver to see the information on the display screen while looking straight ahead at the road, without having to shift their line of sight. It can enhance driving safety and reduce driver distraction.

[0048] AR (Augmented Reality) is a technology that combines virtual information with the real world, enhancing the user's perception of the real environment.

[0049] Dual-Focal Plane means that the display can project information on two different focal planes, which can include a near focal plane and a far focal plane. For example, the near focal plane can be 2 - 3 meters away from the driver and can display non-AR elements such as vehicle speed, gear position, navigation thumbnail, fuel consumption, warning prompts, environmental perception simulation, and intelligent image cards in the W (Windshield, directly projected onto the windshield)-HUD process static information. The far focal plane can be 7 - 15 meters away from the driver and can display dynamic information such as navigation arrows (including turns, straight, U-turns, roundabouts, waypoints, and destinations) and lane departure warnings in the AR process.

[0050] Dual-Focal Plane AR-HUD is a technology that combines HUD and AR, enabling hierarchical display of driving information and integrating this information with the actual road environment to enhance the visualization of information during driving, reduce the driver's distraction caused by looking at the instrument panel or navigation device, and thus improve driving safety and intuitiveness.

[0051] However, due to the field of view limitation of the HUD, when an information point (such as a waypoint or destination) is outside the field of view of the HUD, the AR-HUD cannot display the information point, and the driver cannot see the information point through the AR-HUD, thus unable to make timely driving decisions and affecting the driver's driving experience.

[0052] In view of this, the embodiments of the present application provide a display method that can break through the limitation of the field of view, display information points outside the field of view within the driver's field of vision, optimize the display strategy of information points, and improve the practicality of the Dual-Focal Plane AR-HUD.

[0053] The technical solutions provided in the embodiments of this application can be applied to intelligent devices. For example, transportation vehicles (including land transportation vehicles, air transportation vehicles, and water transportation vehicles. Land transportation vehicles such as cars, intelligent locomotives, motorcycles, bicycles, electric scooters, etc. Air transportation vehicles such as drones, helicopters, airplanes, and water transportation vehicles such as boats, submarines, water motorcycles, etc.), robots (including service robots, industrial robots, and medical robots, etc.), automatic guided vehicles, automatic delivery devices (including automatic delivery robots, delivery drones), agricultural equipment (pesticide spraying drones, harvesters, etc.), medical equipment (mobile hospital beds, drug delivery robots, disinfection robots, etc.), cleaning equipment (floor cleaning robots, automatic floor scrubbers, window cleaning robots, etc.), entertainment equipment (electric scooters, remote control cars, remote control airplanes, VR devices, etc.), scientific research equipment (exploration vehicles, underwater robots, drones, etc.) and other intelligent devices. The embodiments of this application do not make limitations here.

[0054] Figure 1 It is a schematic diagram of the software architecture of a display method provided in the embodiments of this application. As Figure 1 shown:

[0055] It includes ADF (Autonomous Driving Domain) and CDF (Cockpit Domain).

[0056] 1. ADF (Autonomous Driving Domain) ADF is the core module for implementing autonomous driving functions and mainly includes the following components:

[0057] Relative localization module: It is used to output the degree of attitude change between the current moment and the previous moment of the intelligent device, and provide the position change information of the vehicle in the local coordinate system.

[0058] Ethernet information processing module (SOA_Adapter, SOA message processing module) (Relative_localization): It is used to send and receive SOA messages.

[0059] WorldModel module: It is used to build a dynamic model of the vehicle's surrounding environment and support path planning and decision control.

[0060] Global Localization module: It is used to output the longitude and latitude information of the vehicle in the global coordinate system to achieve high-precision positioning.

[0061] 2. CDF (Cockpit Domain) CDF is the core module for implementing intelligent cockpit functions and mainly includes the following two systems:

[0062] The first system (QNX system, embedded system):

[0063] Head-up display module (AR-HUD APP): It includes a first process output module and a second process output module.

[0064] The first process output module (W-HUD): It is used to output information of non-AR elements such as vehicle speed, gear position, navigation thumbnail, environmental perception simulation, intelligent image card, etc.

[0065] The second process output module (AR-HUD): It is used to output information of AR elements such as turning, going straight, U-turn, roundabout, waypoint, destination, etc., enhancing the driver's navigation experience.

[0066] Vehicle control information processing module (vehicle_info): It is used to receive and process vehicle control information to ensure the coordinated work of the cockpit system and other vehicle modules.

[0067] Navigation communication module (NaviInfo): It is used to realize the transmission and communication of navigation information.

[0068] Ethernet message processing module (SOA_Manager).

[0069] The second system (Android system, Android system):

[0070] Navigation module (Navi): It is used to retrieve and transmit navigation information, supporting path planning and real-time navigation functions.

[0071] Figure 1 The shown world model module may include lane line data (lane_line) and road edge data (road_edge).

[0072] The lane line data includes lane line ID (ll_id): uint32, lane line role (||_role): HMi LineRole, and lane line point chain data (||_point): Hmi_Point.

[0073] Lane line role (||_role): HMi LineRole includes: LR_Unkown: 0 (unknown), LR_Host Left: 1 (left lane line), LR_Host_Right: 2 (right lane line), LR_Adjacent Left Left: 3 (left lane left lane line), LR_Adjacent Left Right: 4 (left lane right lane line), LR_Adiacent Right Left: 5 (right lane left lane line), LR_Adjacent_Right_Right: 6 (right lane right lane line), LR_LeftLeft Left: 7 (left left lane left lane line), LR_LeftLeft_Right: 8 (left left lane right lane line), LR_RightRight_Left: 9 (right right lane left lane line), and LR_RightRight_Right: 10 (right right lane right lane line).

[0074] Lane line point chain data (||_point): Hmi_Point includes point chain point attribute (point_type): Hmi_LinePointType, point chain point color (point_color): Hmi_Color, and point chain point coordinate (point position): Vector3f. The point chain point attribute includes: 0: LPT Unknown (unknown), 1: LPT_Solid (solid line), 2: LPT Dash (dashed line), 3: LPT Solid Dash (solid and dashed line), 4: LPT Dash Solid (dashed and solid line), 5: LPT_Solid_Solid (double solid line), 6: LPT Deceleration Dash (dashed deceleration line), 7: LPT Deceleration Solid (solid deceleration line), 8: LPT Guide (diversion line), 9: LPT Dash Dash (double dashed line), 10: LPT Direct (variable guide line), 11: LPTSolid Four (quadruple solid line).

[0075] The point chain point color includes: 0: Coler_Unknown (unknown color), 1: Color white (white), 2: ColorBlack (black), 3: Color_Grey (grey), 4: Color_Red (red), 5: Color_Yellow (yellow), 6: Color_Blue (blue), 7: Color orange (orange), 8: Color_Green (green).

[0076] The road edge data includes the road edge (road_edge), the road edge attribute (re_side): HMi_Road_Rdgeside, and the road edge point coordinates (Re_points): Vector3f. The road edge attribute (re_side) includes 0: RES_Unknown (unknown), 1: RES_Left (left road edge), and 2: RES_Right (right road edge).

[0077] The following describes the technical solution of the embodiment of the present application in combination with Figures 2 to 15 the examples in

[0078] As Figure 2 shown, it is a schematic diagram of the projection image acquisition process of a display method provided by an embodiment of the present application. The method includes the following steps:

[0079] Step S201: Obtain the first coordinate to be displayed.

[0080] For example, when it is detected that the first driving distance between the current position point and the first coordinate to be displayed is less than the first preset distance, the first coordinate to be displayed is obtained. In the embodiment of the present application, the current position point can be the coordinate point where the intelligent device is currently located in the vehicle coordinate system. The first coordinate to be displayed can be the coordinate point where the waypoint or the end point is located in the vehicle coordinate system on the planned driving route of the intelligent device. Exemplarily, the first coordinate to be displayed can be the coordinate of devices such as a charging station or a battery swapping station, and the present application does not limit this here.

[0081] The intelligent device can obtain the first coordinate to be displayed through SD navigation information (Standard Definition Navigation Information).

[0082] Exemplarily, the intelligent device can be installed with a navigation APP (Application). The navigation APP on the intelligent device can obtain the first coordinate to be displayed. The data format of the first coordinate to be displayed can be ViaPnt (Via Point). Figure 3 It is a schematic diagram of the data structure of a coordinate to be displayed of a display method provided by an embodiment of the present application. As Figure 3As shown, view_location is used to represent longitude and latitude. vector2f is a two-dimensional vector containing two floating-point numbers, representing latitude and longitude respectively; poi_category is used to represent the type of POI (i.e., the first coordinate to be displayed). When poi_category is 0, it means the type of POI is ordinary. When poi_category is 1, it means the type of POI is a charging station. When poi_category is 2, it means the type of POI is a battery swapping station; is_dest is used to represent a waypoint or a destination. is_dest:bool is a boolean variable used to indicate whether a certain location or object is a destination; remainDistance To Next Navi Point:int represents the distance from the vehicle (intelligent device) to the POI point and is an integer variable.

[0083] The first driving distance can be the driving distance between the current position point and the first coordinate to be displayed on the planned driving route of the intelligent device.

[0084] For example, Figure 4 is a schematic diagram of the driving route to be traveled for a display method provided by an embodiment of the present application. As Figure 4 shown, the current position point can be point B. Point B is a waypoint between the starting point (point A) and the ending point (point C). The first coordinate to be displayed can be point C, and the first driving distance can be the driving distance between point B and point C.

[0085] In the embodiment of the present application, the first preset distance can be 500 meters, 450 meters, 550 meters, etc. The specific value of the first preset distance in the embodiment of the present application is not limited. The driver can also set the specific value of the first preset distance according to actual driving needs.

[0086] For example, the application program for the intelligent device to detect that the first driving distance is less than the first preset distance (500m) can be {viaPnt}.{remainDistanceToNextNaviPnt} < 500.

[0087] When the first driving distance is less than the first preset distance, it means that the intelligent device is relatively close to the first coordinate to be displayed. The intelligent device can then obtain the first coordinate to be displayed, determine the projection coordinate corresponding to the first coordinate to be displayed, and display the first projection image containing the projection coordinate within the preset projection range of the intelligent device, so as to guide the driver to move the intelligent device to the area corresponding to the first coordinate to be displayed and improve the timeliness of driving decisions.

[0088] The preset projection range may be the field of view of the AR-HUD corresponding to the intelligent device. For example, when the intelligent device is a vehicle, the preset projection range may be a partial range or the entire range of the vehicle's windshield.

[0089] In some embodiments, after the intelligent device detects that the first driving distance is less than the first preset distance, it may further determine whether the first coordinate to be displayed is the same as the previous coordinate to be displayed. The previous coordinate to be displayed may be the second coordinate to be displayed, and the display serial number corresponding to the second coordinate to be displayed is the display serial number adjacent to the display serial number corresponding to the first coordinate to be displayed and before the display serial number corresponding to the first coordinate to be displayed.

[0090] If it is the same coordinate to be displayed, the intelligent device uses the display method corresponding to the previous coordinate to be displayed to guide the driver to drive the intelligent device to move to the area corresponding to the previous coordinate to be displayed. If it is not the same coordinate to be displayed, the intelligent device uses the technical solution provided in the embodiments of the present application to guide the driver to drive the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0091] In some embodiments, before the intelligent device determines whether the first coordinate to be displayed is the same as the previous coordinate to be displayed, it may also determine the deviation distance between the current position point corresponding to the intelligent device and the driving route corresponding to the previous coordinate to be displayed. If the deviation distance is less than or equal to the preset deviation distance value, it determines whether the first coordinate to be displayed is the same as the previous coordinate to be displayed. If the deviation distance is greater than the preset deviation distance value, it directly uses the technical solution provided in the embodiments of the present application to guide the driver to drive the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0092] When the deviation distance is less than or equal to the preset deviation distance value and the first coordinate to be displayed is the same as the previous coordinate to be displayed, the intelligent device no longer displays the coordinate to be displayed. This can avoid the situation of projection screen freezing, error, and incoherence caused by multiple calibrations, and further improve the user's driving experience.

[0093] Step S202: Determine a plurality of reference points according to the first coordinate to be displayed.

[0094] In the embodiments of the present application, in the vehicle coordinate system of the intelligent device, the intelligent device may determine the vertical driving direction of the vehicle body as the longitudinal direction, that is, the forward direction of the intelligent device, which is represented as the x-axis direction. The direction perpendicular to the vertical driving direction is determined as the transverse direction, which is represented as the y-axis direction.

[0095] For example, Figure 5 is a schematic diagram of the vehicle coordinate system of a display method provided in the embodiments of the present application. As Figure 5As shown, the forward direction of the intelligent device is the x-axis direction. The left side direction of the intelligent device is the y-axis direction, and the left-turn arrow indicates that the intelligent device is about to turn left currently.

[0096] In some embodiments, the vehicle coordinate system of the intelligent device may further include the vertical direction of the intelligent device, that is, above the intelligent device, which is represented as the z-axis direction.

[0097] In the embodiments of the present application, the intelligent device can describe its vehicle attitude (such as the pitch, roll, and yaw of the vehicle) through Euler Angles. Specifically, the Euler Angles can be used to describe the three-dimensional attitude change of the intelligent device relative to the ground coordinate system. The Euler Angles can describe the attitude of an object in three-dimensional space through three rotation angles, usually including: pitch angle (Pitch), roll angle (Roll), and yaw angle (Yaw). Among them, the pitch angle is used to characterize the angle of rotation around the transverse axis of the vehicle, indicating the degree of front-back tilt of the vehicle. The value range of the pitch angle is 0° to ±180°. The pitch angle is positive when the front of the vehicle pitches downward and negative when the front of the vehicle pitches upward; the roll angle is used to characterize the angle of rotation around the longitudinal axis of the vehicle, indicating the degree of left-right tilt of the vehicle. The value range of the roll angle is 0° to ±180°. The roll angle is positive when the left side of the vehicle is higher than the right side and negative when the left side of the vehicle is lower than the right side; the yaw angle is the angle of rotation around the vertical axis of the vehicle, indicating the degree of left-right steering of the vehicle. The value range of the yaw angle is 0° to ±180°. The yaw angle is 0 when the front of the vehicle faces due east. The yaw angle is positive when the front of the vehicle rotates counterclockwise and negative when the front of the vehicle rotates clockwise.

[0098] For example, Figure 6 is a relative distribution diagram of longitude and latitude and the vehicle coordinate system for a display method provided by the embodiments of the present application. As Figure 6 shown, the waypoint is located in the front right of the intelligent device, and the forward direction of the intelligent device in the vehicle coordinate system is the northeast direction in the longitude and latitude coordinate system.

[0099] In the embodiments of the present application, the multiple reference points can be multiple consecutive coordinate points corresponding to the left boundary line on the to-be-traveled route corresponding to the intelligent device in the vehicle coordinate system of the intelligent device, and among the multiple consecutive coordinate points corresponding to the left boundary line, the multiple coordinate points with the longitudinal distance closest to the first to-be-displayed target.

[0100] Exemplarily, the intelligent device may first obtain a left lane line point set and a right lane line point set in the vehicle coordinate system. The left lane line point set includes a plurality of consecutive coordinate points corresponding to the left boundary line on the to-be-traveled route, and the right lane line point set includes a plurality of consecutive coordinate points corresponding to the right boundary line on the to-be-traveled route. The plurality of reference points may include a first left reference point, a second left reference point, a first right reference point, and a second right reference point. The first longitudinal distance and the second longitudinal distance between the first left reference point and the first right reference point and the current position point are greater than the to-be-displayed longitudinal distance between the first to-be-displayed coordinate and the current position point. The third longitudinal distance and the fourth longitudinal distance between the second left reference point and the second right reference point and the current position point are less than the to-be-displayed longitudinal distance. The first left reference point and the second left reference point are two adjacent coordinate points in the left lane line point set, and the first right reference point and the second right reference point are two adjacent coordinate points in the right lane line point set.

[0101] For example, as Figure 7 shown, the first left reference point is point A, the first right reference point is point B, the second left reference point is point C, the second right reference point is point D, the current position point is point O, and the first to-be-displayed coordinate is point P. As Figure 7 shown, the longitudinal distances between point A and point B and point O are x1, the longitudinal distance between point P and point O is x2, and the longitudinal distances between point C and point D and point O are x3, where x1 > x2 > x3.

[0102] In the embodiments of the present application, the left boundary line may be a lane line or a road edge line, and the right boundary line may be a lane line or a road edge line. The embodiments of the present application do not make any limitations in this regard.

[0103] For example, Figure 8 is a schematic diagram of the boundary line of a display method provided by the embodiments of the present application. When both the left boundary line and the right boundary line are lane lines, as shown in (a) of Figure 8 , the left boundary line may be boundary line 1, and the right boundary line may be boundary line 2. When both the left boundary line and the right boundary line are road edge lines, as shown in (b) of Figure 8 , the left boundary line may be boundary line 3, and the right boundary line may be boundary line 4.

[0104] Step S203: Output a to-be-converted image according to the plurality of reference points and the first to-be-displayed coordinate.

[0105] In the embodiments of the present application, the to-be-converted image may be an image including the first left reference point, the second left reference point, the first right reference point, the second right reference point, and the first to-be-displayed coordinate.

[0106] For example, continue to refer to Figure 7, is a schematic diagram of an image to be converted for a display method provided by an embodiment of the present application.

[0107] Step S204: Determine a left information point according to the first left reference point, the second left reference point, and the first coordinate to be displayed. The abscissa corresponding to the left information point is the first horizontal coordinate, and the ordinate corresponding to the left information point is the first vertical coordinate.

[0108] In an embodiment of the present application, the method for the intelligent device to determine the left information point through the first left reference point, the second left reference point, and the first coordinate to be displayed may include:

[0109] Step A1: Determine a first vector according to the first left reference point and the second left reference point. The starting point corresponding to the first vector is the second left reference point, and the ending point corresponding to the first vector is the first left reference point.

[0110] For example, for Figure 7 the corresponding first left reference point (point A) and the second left reference point (point C), the corresponding first vector may be as Figure 9 shown. The starting point of the first vector is the second left reference point (point C), and the ending point of the first vector is the first left reference point (point A), denoted as

[0111] Step A2: Determine a second vector according to the first coordinate to be displayed and the second left reference point. The starting point corresponding to the second vector is the second left reference point, and the ending point corresponding to the second vector is the first coordinate to be displayed.

[0112] For example, for Figure 7 the corresponding second left reference point (point C) and the first coordinate to be displayed (point P), the corresponding second vector may be as Figure 9 shown. The starting point of the second vector is the second left reference point (point C), and the ending point of the second vector is the first coordinate to be displayed (point P), denoted as

[0113] Step A3: Determine the projection of the second vector in the direction of the first vector as the third vector.

[0114] Exemplarily, if the first vector is and the second vector is then the third vector The calculation formula is:

[0115]

[0116] It can be understood that since θ represents and the included angle between them, the modulus of the third vector is: Due to the modulus of the third vector Then the third vector The calculation formula is:

[0117] For example, continue to refer to Figure 9 The first vector is The second vector is Then the third vector is The projection in the direction

[0118] Step A4: Determine the end point corresponding to the third vector, which is the left information point.

[0119] For example, continue to refer to Figure 9 If the third vector is Then it can be determined that the end point E of the third vector is the left information point.

[0120] Step S205: Determine the right information point according to the first right reference point, the second right reference point and the first coordinate to be displayed. The abscissa corresponding to the right information point is the second horizontal coordinate, and the ordinate corresponding to the right information point is the second vertical coordinate.

[0121] In the embodiments of the present application, the method for the intelligent device to determine the right information point through the first right reference point, the second right reference point and the first coordinate to be displayed may include:

[0122] Step B1: Determine the fourth vector according to the first right reference point and the second right reference point. The starting point of the fourth vector is the second right reference point, and the ending point of the fourth vector is the first right reference point.

[0123] For example, for Figure 7 the corresponding first right reference point (point B) and the second right reference point (point D), the corresponding fourth vector can be as Figure 10 shown. The starting point of the fourth vector is the second right reference point (point D), and the ending point of the fourth vector is the first right reference point (point B), denoted as

[0124] Step B2: Determine the fifth vector according to the first coordinate to be displayed and the second right reference point. The starting point of the fifth vector is the second right reference point, and the ending point of the fifth vector is the first coordinate to be displayed.

[0125] For example, for Figure 7 the corresponding second right reference point (point D) and the first coordinate to be displayed (point P), the corresponding fifth vector can be as Figure 10As shown. The starting point of the fifth vector is the second right reference point (point D), and the ending point of the fifth vector is the first coordinate to be displayed (point P), denoted as

[0126] Step B3: Determine the projection of the fifth vector in the direction of the fourth vector, which is the sixth vector.

[0127] Exemplarily, if the fourth vector is and the fifth vector is then the sixth vector The calculation formula is:

[0128]

[0129] It can be understood that since represents the angle between then the magnitude of the sixth vector is: Since the magnitude of the sixth vector then the sixth vector The calculation formula is:

[0130] For example, continue to refer to Figure 10 , the fourth vector is and the fifth vector is then the sixth vector is The projection in the direction

[0131] Step B4: Determine the ending point corresponding to the sixth vector, which is the right information point.

[0132] For example, continue to refer to Figure 10 , if the sixth vector is then it can be determined that the ending point F of the sixth vector is the right information point.

[0133] Step S206: Determine the coordinate to be converted according to the left information point and the right information point. The abscissa corresponding to the coordinate to be converted is the average value of the first horizontal coordinate and the second horizontal coordinate, and the ordinate corresponding to the coordinate to be converted is the average value of the first vertical coordinate and the second vertical coordinate.

[0134] For example, if the left information point is (65, 97) and the right information point is (-9, 101), then the average value of the first abscissa 65 and the second abscissa -9 is (65 + (-9)) / 2 = 56 / 2 = 28, and the average value of the first ordinate 97 and the second ordinate 101 is (97 + 101) / 2 = 99. That is, the coordinate to be converted is (28, 99).

[0135] Continue to refer toFigure 10 If the left information point is E and the right information point is F, the coordinate to be converted can be determined as G.

[0136] Step S207: Convert the coordinate to be converted to the second coordinate system to obtain a projection coordinate, where the second coordinate system is a coordinate system different from the first coordinate system.

[0137] In the embodiment of the present application, the first coordinate system is the vehicle coordinate system, and the second coordinate system can be the screen coordinate system. The coordinate to be converted is a point in the vehicle coordinate system, and the intelligent device can convert the coordinate to be converted in the vehicle coordinate system to the screen coordinate system through a three-dimensional registration matrix to obtain a projection coordinate.

[0138] Step S208: Determine a first projection image according to the projection coordinate.

[0139] After the intelligent device determines the projection coordinate in the screen coordinate system, it can, based on the projection coordinate, set an icon corresponding to the first coordinate to be displayed at the position point corresponding to the projection coordinate and determine the first projection image. It can be understood that the first projection image may include other information to be displayed in addition to the icon corresponding to the first coordinate to be displayed. For example, one or more of a left-turn icon, a right-turn icon, a straight-ahead icon, a roundabout icon, etc.

[0140] It can be understood that the intelligent device can determine the corresponding icon according to the category of the first coordinate to be displayed. Exemplarily, if the first coordinate to be displayed is the end point, the icon corresponding to the first coordinate to be displayed can be in the shape of a flag; if the first coordinate to be displayed is a gas station, the icon corresponding to the first coordinate to be displayed can be in the shape of a gas station; if the first coordinate to be displayed is a service station, the icon corresponding to the first coordinate to be displayed can be in the shape of a service station. Details are not described herein in the present application.

[0141] For example, Figure 11 is a schematic diagram of the first projection image of a display method provided by an embodiment of the present application. As Figure 11 shown, the first projection image includes an icon corresponding to the projection coordinate, that is, icon 1, the current vehicle speed, a left-turn guiding mark, and a drivable lane.

[0142] Step S209: Display the first projection image within a preset projection range, where the first projection image includes a projection coordinate, and the projection coordinate is used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0143] After determining the first projection image, the intelligent device can display the first projection image within a preset projection range to facilitate guiding the driver to drive the intelligent device to move to the area corresponding to the first coordinate to be displayed, so as to improve the timeliness of driving decisions.

[0144] In some embodiments, if the intelligent device detects that the distance between the current position point and the first coordinate to be displayed is small, it can enhance the display of the icon to prompt the driver that they are approaching the first coordinate to be displayed, so as to improve the immediacy of driving decisions, avoid time waste caused by the driver driving past the position, and further enhance the driving experience.

[0145] Specifically, if the driving distance between the current position point and the first coordinate to be displayed is less than the third preset distance, and the vertical distance to be displayed is greater than or equal to the second preset distance, the intelligent device displays a second projection image within the preset projection range. The second projection image includes enhanced projection coordinates, which are used to represent that the driving distance between the current position point and the position point corresponding to the first coordinate to be displayed is less than the third preset distance. The coordinate information of the enhanced projection coordinates is the same as that of the projection coordinates. The third preset distance is greater than the second preset distance and less than the first preset distance.

[0146] Exemplarily, the enhanced projection coordinates can be in the form of flashing, color enhancement, icon enlargement, etc., which are not limited in this application.

[0147] For example, when the first preset distance is 500m and the third preset distance is 50m. As Figure 12 shown, it is a schematic diagram of enhanced projection coordinates of a display method provided by an embodiment of this application.

[0148] In some embodiments, if the intelligent device detects that it has reached the first coordinate to be displayed or the area corresponding to the first coordinate to be displayed, it can not display the projection coordinates or the enhanced projection coordinates to prompt the driver that they have currently reached the area corresponding to the first coordinate to be displayed. The technical solution provided by the embodiment of this application can further avoid time waste caused by the driver driving past the position and enhance the driving experience.

[0149] Specifically, if the vertical distance to be displayed is less than the second preset distance, the intelligent device displays a third projection image within the preset projection range. The third projection image does not include the projection coordinates and / or the enhanced projection coordinates, and the third projection image is used to represent that the intelligent device has moved to the area corresponding to the first coordinate to be displayed.

[0150] It can be understood that the technical solution provided by the embodiment of this application can be applied to the situation where the first driving distance is less than the first preset distance and the vertical distance to be displayed is greater than or equal to the second preset distance. If the vertical distance to be displayed is less than the second preset distance, it indicates that the intelligent device has reached the area corresponding to the first coordinate to be displayed, and the intelligent device does not need to execute the display process provided by the embodiment of this application to determine the projection coordinates.

[0151] For example, when the first preset distance is 500 m and the third preset distance is 50 m, the second preset distance can be a distance close to 0 m, such as -5 m, 1 m, 5 m, etc. The embodiments of the present application do not limit this here.

[0152] In some embodiments, the present application further provides a schematic diagram of a display process of a display method. As Figure 13 shown, the display method provided by the embodiments of the present application may include the following steps:

[0153] Step S1301: Obtain a first coordinate to be displayed, and obtain a left lane line point set and a right lane line point set. The left lane line point set includes a plurality of consecutive coordinate points corresponding to the left boundary line on the to-be-traveled route, and the right lane line point set includes a plurality of consecutive coordinate points corresponding to the right boundary line on the to-be-traveled route.

[0154] In the embodiments of the present application, the first coordinate to be displayed may be a coordinate point not within the field of view angle range. The intelligent device may at least include a display device, and the display device may be a projection device. The display device is used to control the display information within a preset projection range. When the intelligent device is an automobile, the preset projection range may be the windshield of the automobile.

[0155] When the intelligent device detects that the first driving distance is less than the first preset distance and the longitudinal distance to be displayed is greater than or equal to the second preset distance, the first coordinate to be displayed in the preset coordinate system is obtained.

[0156] The method for the intelligent device to obtain the first coordinate to be displayed, the left lane line point set and the right lane line point set may refer to Figure 2 the obtaining method in the corresponding embodiments, and the embodiments of the present application do not elaborate here.

[0157] Step S1302: Based on the first coordinate to be displayed, output an image to be converted. The image to be converted includes a first left reference point, a second left reference point, a first right reference point, a second right reference point and the first coordinate to be displayed. The first longitudinal distance between the first left reference point and the current position point and the second longitudinal distance between the first right reference point and the current position point are both greater than the longitudinal distance to be displayed between the first coordinate to be displayed and the current position point. The third longitudinal distance between the second left reference point and the current position point and the fourth longitudinal distance between the second right reference point and the current position point are both less than the longitudinal distance to be displayed. The first left reference point and the second left reference point are two adjacent coordinate points in the left lane line point set, and the first right reference point and the second right reference point are two adjacent coordinate points in the right lane line point set.

[0158] In the embodiments of the present application, the method for the intelligent device to determine the image to be converted based on the first coordinate to be displayed may refer to Figure 2The determination method in the corresponding embodiment will not be elaborated in the embodiments of the present application.

[0159] Step S1303: Determine a first projection image according to the image to be converted.

[0160] In the embodiments of the present application, for the method by which the intelligent device determines the first projection image based on the image to be converted, reference can be made to Figure 2 The determination method in the corresponding embodiment will not be elaborated in the embodiments of the present application.

[0161] Step S1304: Display the first projection image within a preset projection range. The first projection image includes projection coordinates, which are used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0162] In the technical solution provided by the embodiments of the present application, after determining the first coordinate to be displayed, the intelligent device can determine the image to be converted through the set of left lane line points and the set of right lane line points corresponding to the route to be traveled. The image to be converted includes a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance and the second longitudinal distance between the first left reference point and the first right reference point and the current position point are greater than the longitudinal distance to be displayed between the first coordinate to be displayed and the current position point. The third longitudinal distance and the fourth longitudinal distance between the second left reference point and the second right reference point and the current position point are less than the longitudinal distance to be displayed. Finally, a first projection image is determined according to the image to be converted, and the first projection image is displayed within a preset projection range. In the technical solution provided by the embodiments of the present application, the intelligent device can project the first coordinate to be displayed, which was originally not within the field of view angle, into the field of view angle. Thereby optimizing the display strategy of information points, solving the problem that traditional AR-HUDs cannot display key information points under the limitation of the field of view angle, and improving the practicability of the dual-focus AR-HUD. Specifically, by adjusting the projection range, it is ensured that the driver can timely obtain key navigation information, avoiding delays or errors in driving decisions caused by information loss, further improving the driving experience, and enhancing driving safety and convenience.

[0163] In some embodiments, when the first coordinate to be displayed is within the field of view angle of the smart device and the projection coordinate corresponding to the first coordinate to be displayed is within the preset projection range in the screen coordinate system of the smart device, the smart device can convert the projection coordinate into an AR projection coordinate and display the AR projection coordinate in the real scene corresponding to the preset projection range, so as to improve the intuitiveness of POI point display and further optimize the display strategy of information points by displaying the specific position of the first coordinate to be displayed. Exemplarily, if the first coordinate to be displayed is within the first field of view, the smart device displays a fourth projection image within the preset projection range, and the fourth projection image includes the augmented reality coordinate corresponding to the first coordinate to be displayed.

[0164] For example, as Figure 14 shown, it is a schematic diagram of the augmented reality coordinate of a display method provided by an embodiment of the present application.

[0165] In some embodiments, the smart device can also divide a smooth transition area within the preset projection range. When the projection coordinate corresponding to the first coordinate to be displayed is within the smooth transition area within the preset projection range, an icon (which can be a 2D icon) of the augmented reality coordinate corresponding to the first coordinate to be displayed is displayed within the smooth transition area, and the icon of the augmented reality coordinate is displayed near the boundary line corresponding to the smooth transition area or the field of view angle. The smooth transition area can be a range within the preset projection range that is 30 pixels away from the boundary of the preset projection range, or it can be other ranges, which are not limited in the present application.

[0166] For example, continue to refer to Figure 14 , the smooth transition area can be as Figure 14 shown.

[0167] It should be understood that on the premise of no logical conflict, the above-mentioned various embodiments of the application can be combined and implemented with each other to meet the actual application requirements. The specific embodiments or implementation schemes obtained after these combinations still fall within the protection scope of the present application.

[0168] In some embodiments, when the first coordinate to be displayed is outside the field of view angle of the smart device, the smart device can also render the icon corresponding to the first coordinate to be displayed into an icon form, display the icon corresponding to the first coordinate to be displayed near the boundary line corresponding to the smooth transition area or the field of view angle, and slide closely along the left edge, upper edge, and right edge of the smooth transition area or the field of view angle.

[0169] Exemplarily, if the first coordinate to be displayed is the end point, the icon corresponding to the first coordinate to be displayed can be in the shape of a flag; if the first coordinate to be displayed is a gas station, the icon corresponding to the first coordinate to be displayed can be in the shape of a gas station; if the first coordinate to be displayed is a service station, the icon corresponding to the first coordinate to be displayed can be in the shape of a service station.

[0170] For example, the figure is a schematic diagram of an icon displayed outside the field of view angle of a display method provided by an embodiment of the present application. As Figure 15 shown, the arrow corresponding to the icon can point to the line connecting the vehicle's own position and the POI position.

[0171] In some embodiments, Figure 2 the projection coordinate acquisition program of the projection image acquisition method provided in the corresponding embodiment can be as follows:

[0172] When both the left boundary line and the right boundary line on the to-be-driven route are lane lines, the projection coordinate acquisition program may include:

[0173] Step A1: Obtain the left lane line point set and the right lane line point set corresponding to the left lane line and the right lane line.

[0174] a. {lane_line}[p].{ll_role} == "1Host_left" (obtain the left lane line point set).

[0175] b. {lane_line}[q].{ll_role} == "2Host_right" (obtain the right lane line point set).

[0176] Step A2: Obtain the left reference points, including the first left reference point (i) and the second left reference point (i + 1).

[0177] a. {lane_line}[p].{ll_points}[i + 1].x > poi.x (obtain the first left reference point i).

[0178] b. {lane_line}[p].{ll_points}[i].x < poi.x (obtain the second left reference point i + 1).

[0179] Step A3: Obtain the right reference points, including the first right reference point (j) and the second right reference point (j + 1).

[0180] a. {lane_line}[q].{ll_points}[j + 1].x > poi.x (obtain the first right reference point j).

[0181] b. {lane_line}[q].{ll_points}[j].x < poi.x (obtain the second right reference point j + 1).

[0182] Step A4: Determine the left information points.

[0183] a. Determine the vector A pointing from {lane_line}[p].{||_points}[i] to the poi.

[0184] b. Determine the vector B pointing from {lane_line}[p].{||_points}[i] to {lane_line}[p].{||_points}[i + 1].

[0185] c Determine the projection point of vector A onto vector B:

[0186] {noileft} = (A * B / (|B| * |B|)) * B + {lane_line}[p].{||points is the left information point.

[0187] Step A5: Determine the right information point.

[0188] a. Determine the vector A' pointing from {lane_line}[q].{||_points}[j] to the poi.

[0189] b. Determine the vector B' pointing from {lane_line}[q].{||_points}[j] to {lane_line}[q].{||_points}[j + 1].

[0190] c. Find the projection point of vector A' onto vector B:

[0191] {poi_right} = (A' * B' / (|B'| * |B'|)) * B' + {[lane_line}[q].{||_points}[j] is the right information point.

[0192] Step A6: Determine the projection coordinates.

[0193] a. {poi_final}.x = ({poi_left}.x + {poi_right}.x) / 2; Determine the x - coordinate of the projection coordinates.

[0194] b. {poi_final}.y = ({poi_left}.y + {poi_right}.y) / 2; Determine the y - coordinate of the projection coordinates.

[0195] When both the left and right boundary lines on the to - be - traveled route are curbs, the projection coordinate acquisition program may include:

[0196] Step B1: Obtain the left - hand lane line point set and the right - hand lane line point set corresponding to the left curb and the right curb.

[0197] a. {road_edge}[p].{re_side} == "1RES_Left" (obtain the left lane line point set).

[0198] b. {road_edge}[q].{re_side} == "2RES_Right" (obtain the right lane line point set).

[0199] Step B2: Obtain the left reference points, including the first left reference point (i) and the second left reference point (i + 1).

[0200] a. {road_edge}[p].{re_points}[i + 1].x > poi.x (obtain the first left reference point i).

[0201] b. {road_edge}[p].{re_points}[i].x < poi.x (obtain the second left reference point i + 1).

[0202] Step B3: Obtain the right reference points, including the first right reference point (j) and the second right reference point (j + 1).

[0203] a. {road_edge}[q].{re_points}[j + 1].x > poi.x (obtain the first right reference point j).

[0204] b. {road_edge}[q].{re_points}[j].x < poi.x (obtain the second right reference point j + 1).

[0205] Step B4: Determine the left information point.

[0206] a. Determine the vector A from {road_edge}[p].{re_points}[i] to poi.

[0207] b. Determine the vector B from {road_edge}[p].{re_points}[i] to {road_edge}[p].{re_points}[i + 1].

[0208] c Determine the projection point of vector A projected onto vector B:

[0209] {noileft} = (A * B / (|B| * |B|)) * B + {road_edge}[p].{re points is the left information point.

[0210] Step B5: Determine the right information point.

[0211] a. Determine the vector A' pointing from {road_edge}[q].{re_points}[j] to the poi.

[0212] b. Determine the vector B' pointing from {road_edge}[q].{re_points}[j] to {road_edge}[q].{re_points}[j + 1].

[0213] c. Find the projection point of the vector A' onto the vector B:

[0214] {poi_right} = (A' * B' / (|B'| * |B'|)) * B' + {road_edge}[q].{re_points}[j] is the right-side information point.

[0215] Step B6: Determine the projection coordinates.

[0216] a. {poi_final}.x = ({poi_left}.x + {poi_right}.x) / 2; Determine the x coordinate of the projection coordinates.

[0217] b. {poi_final}.y = ({poi_left}.y + {poi_right}.y) / 2; Determine the y coordinate of the projection coordinates.

[0218] In some embodiments,

[0219] Corresponding to the display method in the above embodiments, an embodiment of the present application provides a display device, which can be implemented by software, hardware, or a combination of both as part or all of a computer device, and is used to execute the steps in the display method in the above embodiments.

[0220] Figure 16 The structural schematic diagram of a display device 160 provided by an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiments of the present application are shown.

[0221] Refer to Figure 16 , the device 160 can be an intelligent device, including an acquisition module 1610 and a processing module 1620.

[0222] The acquisition module 1610 is used to acquire the first coordinate to be displayed, and to acquire the left lane line point set and the right lane line point set. The left lane line point set includes multiple consecutive coordinate points corresponding to the left boundary line on the to-be-driven route, and the right lane line point set includes multiple consecutive coordinate points corresponding to the right boundary line on the to-be-driven route.

[0223] A processing module 1620, configured to output an image to be converted based on a first coordinate to be displayed. The image to be converted includes a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance between the first left reference point and the current position point, and the second longitudinal distance between the first right reference point and the current position point are both greater than the longitudinal distance to be displayed between the first coordinate to be displayed and the current position point. The third longitudinal distance between the second left reference point and the current position point, and the fourth longitudinal distance between the second right reference point and the current position point are both less than the longitudinal distance to be displayed. The first left reference point and the second left reference point are two adjacent coordinate points in the left lane line point set, and the first right reference point and the second right reference point are two adjacent coordinate points in the right lane line point set.

[0224] The processing module 1620 is further configured to determine a first projection image according to the image to be converted, and display the first projection image within a preset projection range. The first projection image includes projection coordinates, and the projection coordinates are used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

[0225] Specifically, the processing module 1620 is configured to: determine a left information point according to the first left reference point, the second left reference point, and the first coordinate to be displayed. The abscissa corresponding to the left information point is the first horizontal coordinate, and the ordinate corresponding to the left information point is the first longitudinal coordinate.

[0226] Determine a right information point according to the first right reference point, the second right reference point, and the first coordinate to be displayed. The abscissa corresponding to the right information point is the second horizontal coordinate, and the ordinate corresponding to the right information point is the second longitudinal coordinate.

[0227] Determine the coordinate to be converted according to the left information point and the right information point. The abscissa corresponding to the coordinate to be converted is the average value of the first horizontal coordinate and the second horizontal coordinate, and the ordinate corresponding to the coordinate to be converted is the average value of the first longitudinal coordinate and the second longitudinal coordinate.

[0228] Convert the coordinate to be converted to a second coordinate system to obtain projection coordinates. The second coordinate system is a coordinate system different from the first coordinate system.

[0229] Determine the first projection image according to the projection coordinates.

[0230] The processing module 1620 is specifically configured to: determine a first vector according to a first left reference point and a second left reference point, where the starting point corresponding to the first vector is the second left reference point, and the ending point corresponding to the first vector is the first left reference point. Determine a second vector according to a first coordinate to be displayed and the second left reference point, where the starting point corresponding to the second vector is the second left reference point, and the ending point corresponding to the second vector is the first coordinate to be displayed. Determine the projection of the second vector in the direction of the first vector as a third vector. Determine the ending point corresponding to the third vector as the left information point.

[0231] The processing module 1620 is specifically configured to: determine a fourth vector according to a first right reference point and a second right reference point, where the starting point corresponding to the fourth vector is the second right reference point, and the ending point corresponding to the fourth vector is the first right reference point. Determine a fifth vector according to a first coordinate to be displayed and the second right reference point, where the starting point corresponding to the fifth vector is the second right reference point, and the ending point corresponding to the fifth vector is the first coordinate to be displayed. Determine the projection of the fifth vector in the direction of the fourth vector as a sixth vector. Determine the ending point corresponding to the sixth vector as the right information point.

[0232] In some embodiments, the first coordinate to be displayed is a coordinate point different from the second coordinate to be displayed, and the display serial number corresponding to the second coordinate to be displayed is the display serial number adjacent to the display serial number corresponding to the first coordinate to be displayed before the display serial number corresponding to the first coordinate to be displayed.

[0233] In some embodiments, the driving distance between the current position point and the first coordinate to be displayed is a first driving distance. The acquisition module 1610 is specifically configured to: if the first driving distance is less than a first preset distance and the longitudinal distance to be displayed is greater than or equal to a second preset distance, acquire the first coordinate to be displayed in a preset coordinate system, where the first preset distance is greater than the second preset distance.

[0234] The processing module 1620 is further configured to: if the driving distance between the current position point and the first coordinate to be displayed is less than a third preset distance and the longitudinal distance to be displayed is greater than or equal to the second preset distance, display a second projection image within a preset projection range, where the second projection image includes enhanced projection coordinates, and the enhanced projection coordinates are used to represent that the driving distance between the current position point and the position point corresponding to the first coordinate to be displayed is less than the third preset distance, and the coordinate information of the enhanced projection coordinates is the same as the coordinate information of the projection coordinates. The third preset distance is greater than the second preset distance and less than the first preset distance.

[0235] The processing module 1620 is further configured to: if the longitudinal distance to be displayed is less than the second preset distance, display a third projection image within a preset projection range, where the third projection image does not include projection coordinates and / or enhanced projection coordinates, and the third projection image is used to represent that the intelligent device has moved to the area corresponding to the first coordinate to be displayed.

[0236] In some embodiments, the field of view corresponding to the current position point is the first field of view. When the display device is at the current position point, the first field of view angle corresponding to the preset projection range is the same as the second field of view angle corresponding to the first field of view range. A fourth projection image is displayed within the preset projection range, and the fourth projection image includes the augmented reality coordinates corresponding to the first coordinates to be displayed.

[0237] It should be noted that, for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same inventive concept as the method embodiments of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details are not elaborated herein.

[0238] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0239] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0240] Figure 17 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 17 shown, the electronic device 17 in this embodiment includes: at least one processor 1710 ( Figure 17 only one is shown), a memory 1720, and a communication module 1740. A computer program 1730 that may run on the processor 1710 is stored in the memory 1720. When the processor 1710 executes the computer program 1730, the steps in the above display method embodiment are implemented, such as Figure 2 steps 201 to 209 shown in Figure 13 or steps 1301 to 1304 shown in Figure 16 . When the processor 1710 executes the computer program 1730, the functions of each module / unit in the above device embodiments are implemented, such as Figure 16 the functions of module 1610 to module 1620 shown in

[0241] The communication module 1740 may be a separate communication unit for communicating with an external server or a terminal device. Figure 17 The electronic device 17 may include, but is not limited to: a processor 1710 and a memory 1720. Those skilled in the art can understand that Figure 17 merely an example of the electronic device 17, which does not constitute a limitation to the electronic device 17, and may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device 17 may further include an input and sending device, a network access device, a bus, etc.

[0242] The processor 1710 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0243] In some embodiments, the memory 1720 may be an internal storage unit of the electronic device 17, such as the hard disk or memory of the electronic device 17. The memory 1720 may also be an external storage device of the electronic device 17, such as a plug-in hard disk equipped on the electronic device 17, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. The memory 1720 may also include both the internal storage unit of the electronic device 17 and the external storage device. The memory 1720 is used to store an operating system, application programs, a BootLoader, data, and other programs, such as the program code of the computer program 1730. The memory 1720 may also be used to temporarily store data that has been sent or will be sent.

[0244] In addition, those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example for illustration. In practical applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. In each embodiment of the present application, each functional unit may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0245] The embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program runs on an electronic device, the electronic device is caused to execute the steps in the above method embodiments.

[0246] An embodiment of the present application provides a chip, which includes a processor and a memory. A computer program is stored in the memory, and when the computer program is executed by the processor, the steps in the above-mentioned method embodiments are implemented.

[0247] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device is enabled to execute the steps in the above-mentioned method embodiments.

[0248] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0249] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0250] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein again.

[0251] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0252] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled professionals may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0253] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0254] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0255] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0256] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the large-screen device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0257] Finally, it should be noted that the above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A display method, characterized in that, Applied to an intelligent device, the method includes: Obtaining a first coordinate to be displayed, and obtaining a set of left lane line points and a set of right lane line points. The set of left lane line points includes a plurality of consecutive coordinate points corresponding to the left boundary line on the route to be traveled, and the set of right lane line points includes a plurality of consecutive coordinate points corresponding to the right boundary line on the route to be traveled; Based on the first coordinate to be displayed, outputting an image to be converted, the image to be converted including a first left reference point, a second left reference point, a first right reference point, a second right reference point, and the first coordinate to be displayed. The first longitudinal distance between the first left reference point and the current position point and the second longitudinal distance between the first right reference point and the current position point are both greater than the longitudinal distance to be displayed between the first coordinate to be displayed and the current position point. The third longitudinal distance between the second left reference point and the current position point and the fourth longitudinal distance between the second right reference point and the current position point are both less than the longitudinal distance to be displayed. The first left reference point and the second left reference point are two adjacent coordinate points in the set of left lane line points, and the first right reference point and the second right reference point are two adjacent coordinate points in the set of right lane line points; Determining a first projection image according to the image to be converted; Displaying the first projection image within a preset projection range. The first projection image includes projection coordinates, and the projection coordinates are used to guide the intelligent device to move to the area corresponding to the first coordinate to be displayed.

2. The display method according to claim 1, wherein The image to be converted is an image in a first coordinate system. The determining the first projection image according to the image to be converted includes: Determining a left information point according to the first left reference point, the second left reference point, and the first coordinate to be displayed. The abscissa corresponding to the left information point is a first horizontal coordinate, and the ordinate corresponding to the left information point is a first longitudinal coordinate; Determining a right information point according to the first right reference point, the second right reference point, and the first coordinate to be displayed. The abscissa corresponding to the right information point is a second horizontal coordinate, and the ordinate corresponding to the right information point is a second longitudinal coordinate; Determining coordinates to be converted according to the left information point and the right information point. The abscissa corresponding to the coordinates to be converted is the average of the first horizontal coordinate and the second horizontal coordinate, and the ordinate corresponding to the coordinates to be converted is the average of the first longitudinal coordinate and the second longitudinal coordinate; Converting the coordinates to be converted to a second coordinate system to obtain the projection coordinates. The second coordinate system is a coordinate system different from the first coordinate system; Determining the first projection image according to the projection coordinates.

3. The display method according to claim 2, wherein The determining the left information point according to the first left reference point, the second left reference point, and the first coordinate to be displayed includes: Determining a first vector according to the first left reference point and the second left reference point. The starting point corresponding to the first vector is the second left reference point, and the ending point corresponding to the first vector is the first left reference point; Determine a second vector based on the first coordinate to be displayed and the second left reference point, where the starting point of the second vector is the second left reference point and the ending point of the second vector is the first coordinate to be displayed; Determine the projection of the second vector in the direction of the first vector as a third vector; Determine the ending point corresponding to the third vector as the left information point.

4. The display method according to claim 2, wherein The determining the right information point according to the first right reference point, the second right reference point and the first coordinate to be displayed includes: Determine a fourth vector based on the first right reference point and the second right reference point, where the starting point of the fourth vector is the second right reference point and the ending point of the fourth vector is the first right reference point; Determine a fifth vector based on the first coordinate to be displayed and the second right reference point, where the starting point of the fifth vector is the second right reference point and the ending point of the fifth vector is the first coordinate to be displayed; Determine the projection of the fifth vector in the direction of the fourth vector as a sixth vector; Determine the ending point corresponding to the sixth vector as the right information point.

5. The display method according to claim 1, wherein The first coordinate to be displayed is a coordinate point different from the second coordinate to be displayed, and the display serial number corresponding to the second coordinate to be displayed is the display serial number adjacent to and before the display serial number corresponding to the first coordinate to be displayed.

6. The display method according to claim 1, wherein The driving distance between the current position point and the first coordinate to be displayed is a first driving distance. The obtaining the first coordinate to be displayed includes: If the first driving distance is less than a first preset distance and the longitudinal distance to be displayed is greater than or equal to a second preset distance, then obtain the first coordinate to be displayed, where the first preset distance is greater than the second preset distance.

7. The display method according to claim 6, wherein After the method, it further includes: if the driving distance between the current position point and the first coordinate to be displayed is less than a third preset distance and the longitudinal distance to be displayed is greater than or equal to the second preset distance, then display a second projection image within the preset projection range, where the second projection image includes enhanced projection coordinates for characterizing that the driving distance between the current position point and the position point corresponding to the first coordinate to be displayed is less than the third preset distance, and the coordinate information of the enhanced projection coordinates is the same as the coordinate information of the projection coordinates, and the third preset distance is greater than the second preset distance and less than the first preset distance.

8. The display method according to claim 7, characterized in that, After the method, it further includes: if the longitudinal distance to be displayed is less than the second preset distance, then display a third projection image within the preset projection range, where the third projection image does not include the projection coordinates and / or the enhanced projection coordinates, and the third projection image is used to characterize that the intelligent device has moved to the area corresponding to the first coordinate to be displayed.

9. The display method according to any one of claims 1 to 8, characterized in that, The visual field range corresponding to the current position point is a first visual field range. When the intelligent device is located at the current position point, the first field of view angle corresponding to the preset projection range is the same as the second field of view angle corresponding to the first visual field range; If the first coordinate to be displayed is within the first visual field range, the method further includes: displaying a fourth projection image within the preset projection range, where the fourth projection image includes the augmented reality coordinate corresponding to the first coordinate to be displayed.

10. An electronic device, characterized in that, comprising a processor and a memory, the processor is configured to execute a computer program stored in the memory to implement the display method described in any one of claims 1-9 above.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the display method described in any one of claims 1-9 above.

Images