Navigation method and apparatus

Abstract

Embodiments of the present application provide a navigation method and apparatus, for use in providing lane-level navigation guidance, thereby providing more accurate navigation guidance, and improving the accuracy of vehicle driving. The method comprises: first, acquiring navigation data and lane perception data, wherein the navigation data comprises a navigation path of a vehicle when driving on a map and can comprise a path generated on the basis of a driving starting point and a destination of the vehicle, and the lane perception data comprises information of at least one lane that is obtained on the basis of information collected in an environment where the vehicle is located, and specifically comprises information of at least one lane determined from environmental perception data collected by a sensor in the vehicle; generating navigation guidance on the basis of the navigation data and the lane perception data; and displaying the navigation guidance, wherein the navigation guidance can be used for identifying a drivable lane corresponding to the navigation path in the environment, and the drivable lane comprises a vehicle drivable path obtained on the basis of the lane perception data.

Description

Navigation method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 19, 2023, with application number 202311762546.6 and application name “A Navigation Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of vehicles, and in particular to a navigation method and device. Background Art

[0003] With the rapid development of cities, the number of lanes is increasing, and road patterns are becoming increasingly complex. The interface of full-scenario navigation software has remained relatively unchanged. Especially in complex road scenarios, it is becoming increasingly difficult for users to compare map navigation data with the real world, making it easy to take the wrong intersection or miss the opportunity to change lanes.

[0004] From a data perspective, high-definition (HD) maps are rich in information and closely resemble the real world. However, they are expensive to build, difficult to guarantee freshness, and subject to significant regulatory constraints. This makes it difficult to provide HD guidance data in urban areas, failing to meet users' advanced navigation needs.

[0005] Therefore, how to provide users with more accurate navigation guidance has become an urgent problem to be solved. Summary of the Invention

[0006] The present application provides a navigation method and apparatus for providing lane-level navigation guidance, thereby providing more accurate navigation guidance and improving the accuracy of vehicle navigation.

[0007] In view of this, on the first aspect, the present application provides a navigation method, comprising: first, obtaining navigation data and lane perception data, the navigation data including a navigation path of a vehicle when traveling on a map, and may include a path generated based on the vehicle's driving starting point and destination, the lane perception data including information of at least one lane determined based on information collected in the environment where the vehicle is located, and specifically may include determination based on environmental perception data collected by sensors in the vehicle; subsequently, generating navigation guidance based on the navigation data and lane perception data, and displaying the navigation guidance, the navigation guidance being used to provide guidance for the vehicle to travel along the navigation path, the navigation guidance being used to identify a drivable lane corresponding to the navigation path in the environment, the corresponding drivable lane in the environment including the vehicle's driving path obtained based on the lane perception data, and the path may be a lane with divided lane lines or a lane without divided lane lines that the vehicle can actually travel.

[0008] In the implementation of this application, lane-level navigation guidance can be generated through environmental perception, thereby providing more accurate navigation guidance. Furthermore, the method provided in this application senses lanes in the environment and generates guidance, so even more accurate lane-level guidance can be achieved without relying on high-precision maps. For example, in scenarios where lane lines on a map are unclear or absent, environmental perception can be used to identify drivable paths for the vehicle, providing more accurate navigation guidance and improving vehicle driving accuracy and safety.

[0009] In one possible implementation, the aforementioned method may further include: lane perception data may be determined based on data collected by the vehicle's sensors, such as lane information may be determined based on lane lines included in the lane perception data, or lane perception data may be obtained by outputting lane perception data as input to a pre-trained lane perception model; the lane perception data may specifically include information about at least one lane in the vehicle's environment, and may specifically include information such as lane lines, lane positioning, or the number of lanes of at least one lane. The at least one lane may specifically include the lane the vehicle is currently traveling in, the lane adjacent to the vehicle, or the lane in front of the vehicle. Therefore, in the method provided in the embodiment of the present application, lane information in the vehicle's environment may be perceived based on the perception of the vehicle's environment.

[0010] In one possible implementation, the lane perception data may specifically include data collected by sensors in the vehicle. For example, such sensors may include, but are not limited to, image sensors, radars, infrared sensors, or depth sensors. In other words, the lane perception data may also include, but is not limited to, environmental data perceived by sensors such as image sensors, radars, infrared sensors, or depth sensors. Therefore, in the implementation of this application, sensor-perceived data can be used to understand the specific conditions of the vehicle's environment, thereby determining lane-level navigation guidance based on the perception results, without relying on HD maps.

[0011] In one possible implementation, the aforementioned navigation data may specifically include map data and a navigation route. The map data refers to data on a map of the vehicle's driving area, and the navigation route refers to the route planned by the vehicle within the corresponding map area. This navigation data may be generated based on user triggering or based on the vehicle's autonomous driving function, depending on the actual application scenario.

[0012] In one possible implementation, the aforementioned generation of navigation guidance based on navigation data and lane perception data may include: using map data and the vehicle's historical motion information as input to a pre-trained prediction model, and outputting the vehicle's predicted path. The prediction model is used to output the vehicle's predicted path based on the input data. The vehicle's historical motion information may include information generated by the vehicle's movement in a historical period, such as the vehicle's speed, acceleration, angle, angular velocity, and other information; and then generating navigation guidance based on the predicted path, navigation path, and lane perception data.

[0013] In the implementation manner of the present application, a pre-trained prediction model can be used to predict the vehicle's driving path, so that navigation instructions that better match the vehicle's driving path can be generated based on the vehicle's predicted path, thereby improving the navigation accuracy of the driving vehicle.

[0014] In a possible implementation, the aforementioned generation of navigation guidance based on the predicted path, navigation path, and lane perception data may include: if the navigation path in the navigation data includes a path for the vehicle to change lanes, such as turning at an intersection or changing lanes in a straight line, obtaining a first path and a second path of the navigation path, the first path being the path for the vehicle to enter the target lane from the current lane. For example, when the vehicle needs to turn, the first path may be the path for the vehicle to enter the curve in the navigation path, or, when the vehicle needs to change lanes, the first path may be the path for the vehicle to transfer from the current lane to the target lane, that is, one of the at least one lanes mentioned above, and the second path includes the path for the vehicle after it travels to the target lane planned for the vehicle. For example, when the vehicle needs to turn, the second path is the path for the vehicle to exit the curve in the navigation path. The driving path, or, when the vehicle needs to change lanes, the second path can be the driving path of the vehicle after transferring from the current lane to the target lane, and the target lane is the lane where the vehicle needs to turn or change lanes; then, multiple travel points are determined based on the predicted path, the first path, and the second path. The multiple travel points may include points that the vehicle may pass through when driving. In addition, the predicted path can be used to bind the first path and the second path in the navigation path, such as integrating the predicted path into the first path and the second path, so that when the travel points are determined later, navigation points that are more in line with the real scene can be calculated as travel points; the multiple travel points may include entry and exit travel points, the entry travel points are based on the points determined by the first path, and the exit travel points include the points determined based on the second path, so as to facilitate navigation guidance for the vehicle entering and exiting the curve; navigation guidance is generated based on multiple travel points.

[0015] In the implementation manner of the present application, navigation guidance may be generated based on the granularity of the points that the vehicle may pass through, thereby generating more accurate navigation guidance based on the points that the vehicle may pass through.

[0016] In a possible embodiment, the aforementioned generation of navigation guidance based on multiple row points may include: fitting the multiple row points to generate a steering guidance curve, the steering guidance curve is used to indicate the lane in which the vehicle is traveling in the environment, and the curve model is a model of fitting the curve from multiple points; and then generating navigation guidance based on the steering guidance curve.

[0017] Therefore, in the embodiments of the present application, a smoother guide line can be fitted to generate, thereby improving the user's visual experience of the navigation guidance and improving the user's accuracy in identifying the navigation guidance.

[0018] Optionally, when fitting the steering guide curve, a curve model may be used to fit a smoother guide line to improve the user's viewing experience.

[0019] Optionally, when determining multiple travel points, sampling can be performed from the initially determined multiple travel points to select multiple travel points that are more compatible with the vehicle's actual travel path. These multiple sampled points can then be fitted to generate a steering guidance curve. Therefore, in the embodiments of the present application, the generation of unreachable curves can be avoided by using travel point sampling.

[0020] In a possible embodiment, when the vehicle turns, the aforementioned fitting of multiple row points to generate a steering guidance curve may include: obtaining a set of broken line points, which may include the current position of the vehicle, points on the untraveled path in the predicted path, and row points that the vehicle has not reached among the multiple row points, and a broken line is formed between these multiple row points; then fitting the broken line point set according to the curve model to generate a steering guidance curve, and fitting may be performed on different types of row points to generate a curve with more accurate pointing.

[0021] In one possible implementation, the aforementioned method of fitting multiple line points based on a curve model to generate a steering guidance curve further includes: when the vehicle reaches a curve entry point, fitting line points on a first path to generate a steering guidance curve pointing toward a curve exit point; and when the vehicle reaches a curve exit point, fitting line points on a second path to generate a steering guidance curve from the vehicle toward the end of the predicted path. Therefore, the method provided by this application provides more granular guidance based on line points for both curve entry and exit, thereby providing more accurate navigation guidance.

[0022] In a possible implementation, the aforementioned prediction model may specifically include:

[0023] A map encoder is used to extract features from map data to obtain map features;

[0024] A motion encoder is used to extract features from the historical motion information of the vehicle to obtain motion features;

[0025] Fusion encoder, used to extract features from map features and motion features to obtain global features;

[0026] A reference path generator is used to generate a reference path for the vehicle based on the predicted path of the previous frame;

[0027] Path decoder, used to generate a predicted path based on global features and reference path.

[0028] Therefore, in the embodiments of the present application, a prediction model can be used to extract features from multiple dimensions to obtain an accurate prediction path.

[0029] In a possible implementation, the aforementioned prediction model may further include:

[0030] A visual encoder is used to extract features from input lane perception data (such as images or point clouds, etc., which are information perceived from the environment) to obtain visual features;

[0031] The path decoder is also used to generate a predicted path based on visual features, global features and path features.

[0032] Therefore, in the embodiment of the present application, the path prediction accuracy is improved by combining the visual coding encoder with the environmental perception information.

[0033] In one possible embodiment, the aforementioned reference path generator is further used to combine the predicted path of the previous frame and the motion constraint path to obtain a reference path, i.e., a achievable driving path for the vehicle. The motion constraint path is a path generated based on a physical motion model of the vehicle's travel, and the motion constraint path is used as a constraint for outputting the reference path. The predicted path of the previous frame is the path output by the prediction model when the navigation guidance was last generated or updated, and the reference path is the path that the vehicle may travel during the current path prediction process. In other words, under the constraints of the motion constraint path, a reference path that conforms to the vehicle's motion principles can be obtained, thereby avoiding the generation of an inaccessible path for the vehicle.

[0034] In the embodiment of the present application, the input of the reference trajectory generator may further include a predicted path generated based on the motion model, which is equivalent to introducing the motion model as a constraint to make the predicted path a reachable path, thereby improving the prediction accuracy.

[0035] In a possible implementation, the aforementioned generation of navigation guidance based on navigation data, map data, and lane perception data may also include: determining a lane change decision of the vehicle based on the navigation data, map data, and lane perception data, the lane change decision being used to indicate whether the vehicle changes lanes; and generating navigation guidance based on the lane change decision.

[0036] In the implementation manner of the present application, it is possible to more accurately determine whether a vehicle changes lanes in an actual application scenario based on navigation data, map data, and lane perception data, thereby generating accurate lane-level navigation guidance based on the decision of whether to change lanes.

[0037] In one possible implementation, the aforementioned determination of the vehicle's lane change decision based on navigation data, map data, and lane perception data may include: determining the lane in which the vehicle is currently traveling based on the map data and lane perception data; obtaining the probability of the vehicle transferring from the current lane to each of N lanes based on the lane perception data and navigation data, where N is a positive integer and the N lanes are determined from the lane perception data; determining the vehicle's transfer path based on the probability of the vehicle transferring from the current lane to each of the N lanes; and obtaining the lane change decision based on the transfer path.

[0038] Therefore, the implementation of this application is equivalent to introducing a hidden Markov chain to determine possible lane change paths and a transition probability to determine the vehicle's possible path, thereby improving the accuracy of the resulting transition path. Furthermore, even in complex road conditions, such as those with multiple lanes, worn lane markings, or other vehicles blocking lanes, where lane perception is inaccurate, the vehicle's lane can be accurately identified, providing highly accurate navigation guidance.

[0039] In one possible embodiment, the aforementioned method of obtaining the probability of the vehicle shifting from the current lane to each of the N lanes based on lane perception data and navigation data may include: using the map data and lane perception data as inputs to a classification network, dividing the map into multiple regions through the classification network, wherein the multiple regions include high-confidence regions, and the high-confidence regions include regions with confidence levels higher than a preset value; expanding the map based on the high-confidence regions to obtain an expanded map, such as expanding the low-confidence regions in the map based on the high-confidence regions, thereby increasing the information richness of the low-confidence regions; and obtaining the probability of the vehicle and navigation path shifting from the current lane to each of the N lanes based on the expanded map. Therefore, in the embodiment of the present application, even for some maps with low information richness, the map information richness can be improved by expanding the map based on the high-confidence regions, thereby reducing the reliance on high-precision maps and still achieving accurate navigation guidance under standard definition (SD) maps.

[0040] In one possible implementation, generating navigation guidance based on the lane change decision may include: if the lane change decision is to change lanes, generating navigation guidance based on a transition path; if the lane change decision is to maintain lanes, maintaining the generated guideline indicating lane keeping as the navigation guidance. For lane change decisions, navigation guidance may be generated based on a transition path determined based on the transition probability; for lane maintain decisions, the generated guideline indicating lane keeping may be maintained as the navigation guidance.

[0041] In one possible implementation, the aforementioned vehicle is further provided with a heads-up display (HUD) or display screen to display navigation instructions, including: displaying the navigation instructions via the HUD, with the display position of the navigation instructions matching the environment; or displaying a scene image from the lane perception data on the display screen, and overlaying the navigation instructions within the lanes of the scene image. Therefore, in the method provided in this application, the navigation instructions can be overlaid on the scene image displayed on the HUD or display screen, thereby displaying the navigation instructions on the actual scene and improving the usability of the navigation instructions.

[0042] In a second aspect, the present application provides a navigation device, comprising:

[0043] An acquisition module is configured to acquire navigation data and lane perception data, wherein the map data includes data of a map of the vehicle's driving area, the navigation data includes a navigation path of the vehicle while driving on the map, and the lane perception data includes information of at least one lane determined based on information collected from the vehicle's environment;

[0044] A processing module, configured to generate navigation guidance based on the navigation data and lane perception data;

[0045] The display module is used to display navigation instructions, which are used to provide guidance for the vehicle to travel along the navigation path. The navigation instructions are used to identify the corresponding drivable lanes in the environment corresponding to the navigation path. The corresponding drivable lanes in the environment include the vehicle's travel path obtained based on lane perception data.

[0046] In one possible implementation, the aforementioned processing module is further configured to: perceive and determine lane perception data based on data collected by the vehicle's sensors, such as determining lane information based on lane lines included in the lane perception data, or outputting lane perception data by using the lane perception data as input to a pre-trained lane perception model; the lane perception data may specifically include information about at least one lane in the vehicle's environment, including lane lines, lane positioning, or the number of lanes of at least one lane. The at least one lane may specifically include the vehicle's current lane, an adjacent lane, or a lane in front of the vehicle. Therefore, in the method provided in the embodiment of the present application, lane information in the vehicle's environment can be perceived based on the perception of the vehicle's environment.

[0047] In one possible implementation, the lane perception data may specifically include data collected by sensors in the vehicle. For example, such sensors may include, but are not limited to, image sensors, radars, infrared sensors, or depth sensors. In other words, the lane perception data may also include, but is not limited to, environmental data perceived by sensors such as image sensors, radars, infrared sensors, or depth sensors. Therefore, in the implementation of this application, sensor-perceived data can be used to understand the specific conditions of the vehicle's environment, thereby determining lane-level navigation guidance based on the perception results, without relying on HD maps.

[0048] In one possible implementation, the aforementioned navigation data may specifically include map data and a navigation route. The map data refers to data on a map of the vehicle's driving area, and the navigation route refers to the route planned by the vehicle within the corresponding map area. This navigation data may be generated based on user triggering or based on the vehicle's autonomous driving function, depending on the actual application scenario.

[0049] In one possible implementation, the aforementioned is specifically used to: use map data and historical movement information of the vehicle as input to a prediction model to output a predicted path of the vehicle, the prediction model being used to output the vehicle's driving path based on the input data; and generate navigation guidance based on the predicted path, navigation data, and lane perception data.

[0050] In one possible embodiment, the aforementioned processing module is specifically configured to: if the navigation path in the navigation data includes a path for the vehicle to change lanes, obtain a first path and a second path of the navigation path, wherein the first path is the path for the vehicle to enter a target lane from a current lane. For example, when the vehicle needs to turn, the first path is the path of the vehicle in the navigation path when entering a curve, or, when the vehicle needs to change lanes, the first path may be the path of the vehicle when transferring from the current lane to the target lane, where the target lane is one of the at least one lanes mentioned above; and the second path includes a path planned for the vehicle after the vehicle enters the target lane. For example, when the vehicle needs to turn, the second path is the path of the vehicle in the navigation path when exiting a curve, or, when the vehicle needs to change lanes, the second path may be the path of the vehicle after transferring from the current lane to the target lane; determine a plurality of travel points based on the predicted path, the first path, and the second path, the plurality of travel points including a curve entry point and a curve exit point, the curve entry point including a point obtained based on the first path, and the curve exit point including a point obtained based on the second path; and generate navigation guidance based on the plurality of travel points.

[0051] In a possible implementation, the aforementioned processing module is specifically configured to perform fitting on a plurality of row points to generate a steering guidance curve, where the steering guidance curve is used to indicate a lane in which the vehicle is traveling in an environment.

[0052] Optionally, when the processing module fits the steering guide curve, a curve model can be used to fit a smoother guide line to improve the user's viewing experience.

[0053] In a possible embodiment, when the vehicle turns, the aforementioned processing module is specifically used to: obtain a set of broken line points, the broken line point set including the current position of the vehicle, points on the untraveled path in the predicted path, and broken lines formed between multiple row points that the vehicle has not reached; fit the broken line point set to generate a steering guide curve.

[0054] In one possible embodiment, the aforementioned processing module is specifically used to: when the vehicle travels to the turning point, fit the line points in the first path according to the curve model to generate a steering guidance curve pointing to the turning point; when the vehicle travels to the turning point, fit the line points in the second path according to the curve model to generate a steering guidance curve from the vehicle to the end of the predicted path.

[0055] In a possible implementation, the aforementioned prediction model specifically includes:

[0056] A map encoder is used to extract features from map data to obtain map features;

[0057] A motion encoder is used to extract features from the historical motion information of the vehicle to obtain motion features;

[0058] Fusion encoder, used to extract features from map features and motion features to obtain global features;

[0059] A reference path generator is used to generate a reference path for the vehicle based on the predicted path of the previous frame;

[0060] Path decoder, used to generate a predicted path based on global features and reference path.

[0061] In a possible implementation, the aforementioned prediction model further includes:

[0062] A visual encoder is used to extract features from input images or point cloud data to obtain visual features. The image may be data collected by an image sensor, and the point cloud data may include data collected by a radar.

[0063] The path decoder is also used to generate a predicted path based on visual features, global features and path features.

[0064] In one possible embodiment, the aforementioned reference path generator is further configured to combine the predicted path from the previous frame and the motion-constrained path to generate a reference path. The motion-constrained path is a path generated based on a physical motion model of the vehicle, and the motion-constrained path serves as a constraint for outputting the reference path. The predicted path from the previous frame is the path output by the prediction model when the navigation guidance was last generated or updated, and the reference path is the path the vehicle may travel during the current path prediction process. Specifically, under the constraints of the motion-constrained path, a reference path that conforms to the vehicle's motion principles can be obtained, thereby avoiding the generation of an inaccessible path for the vehicle.

[0065] In one possible implementation, the aforementioned processing module is specifically used to: determine the vehicle's lane change decision based on navigation data, map data, and lane perception data, where the lane change decision is used to indicate whether the vehicle changes lanes; and generate navigation guidance based on the lane change decision.

[0066] In one possible implementation, the aforementioned processing module is specifically used to: determine the lane in which the vehicle is currently traveling based on map data and lane perception data; obtain the probability of the vehicle transferring from the current lane to each of N lanes based on the lane perception data and navigation data, where N is a positive integer and the N lanes are determined from the lane perception data; determine the vehicle's transfer path based on the probability of the vehicle transferring from the current lane to each of the N lanes; and obtain a lane change decision based on the transfer path.

[0067] In one possible implementation, the aforementioned processing module is specifically configured to: use the map data and lane perception data as inputs to a classification network, and divide the map into multiple regions through the classification network, wherein the multiple regions include high-confidence regions, and the high-confidence regions include regions with confidence levels higher than a preset value; expand the map based on the high-confidence regions to obtain an expanded map; and obtain the probability of the vehicle shifting from the current lane to each of N lanes based on the expanded map and the navigation path.

[0068] In one possible implementation, the aforementioned processing module is specifically configured to: generate navigation guidance based on the transfer path if the lane change decision is to change lanes; and maintain the generated guide line indicating lane keeping as a navigation guidance if the lane change decision is to maintain lanes.

[0069] In a possible embodiment, the aforementioned vehicle is further provided with a head-up display HUD or display screen, a display module, which is specifically used to: display navigation instructions through the HUD, and the display position of the navigation instructions matches the environment; or, display a scene image of the vehicle's environment on the display screen, and superimpose the navigation instructions in the lane of the scene image.

[0070] In a third aspect, the present application provides an electronic device, comprising a display device, a memory, and one or more processors, wherein the memory stores code for a graphical user interface of an application, and the one or more processors are configured to execute the code for the graphical user interface (GUI) stored in the memory to display the graphical user interface on the display device, wherein the graphical user interface includes:

[0071] Navigation guidance is displayed, wherein the navigation guidance is generated based on navigation data, map data and scene data, the map data includes data of a map of the vehicle's driving area, the navigation data includes a navigation path for the vehicle when driving on the map, and may include a path generated based on the vehicle's driving starting point and destination, and the scene data is information collected in the environment where the vehicle is located, and may specifically include environmental perception data collected by sensors in the vehicle; the navigation guidance can be used to identify a drivable lane corresponding to the navigation path in the environment, the corresponding drivable lane in the environment includes the vehicle's driving path obtained by perception based on the scene data, or a lane with divided lane lines or a lane without divided lane lines that the vehicle can actually travel, and the navigation guidance is displayed superimposed on the perceived lane.

[0072] Therefore, in the embodiment of the present application, navigation guidance can be superimposed and displayed in the lane in the GUI interface, thereby achieving more accurate lane-level guidance.

[0073] Optionally, the aforementioned display device may specifically include a HUD or a display screen.

[0074] In one possible embodiment, if the aforementioned display device includes a HUD, the projection position of the vehicle's drivable lane in the windshield can be determined based on the user's line of sight and scene data, and navigation guidance can be displayed at the projection position, thereby improving the usability of the navigation guidance.

[0075] In a possible embodiment, if the aforementioned display device includes a display screen, a scene image, that is, an image of the vehicle's drivable area, can be displayed on the display screen, and navigation guidance can be superimposed on the lane in the scene image, thereby improving the usability of the navigation guidance.

[0076] In addition, the method for generating the navigation guidance can refer to the aforementioned first aspect or any optional implementation of the first aspect, and will not be repeated here.

[0077] In a fourth aspect, embodiments of the present application provide a navigation device comprising: a processor and a memory, wherein the processor and the memory are interconnected via a circuit, and the processor invokes program code in the memory to execute the processing-related functions of any of the methods described in the first aspect. Optionally, the navigation device may be a chip.

[0078] In the fifth aspect, an embodiment of the present application provides a navigation device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform functions related to processing as described in the first aspect or any optional embodiment of the first aspect.

[0079] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional embodiment of the first aspect.

[0080] In a seventh aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute a method in any optional implementation of the first or second aspect above.

[0081] In an eighth aspect, an embodiment of the present application provides a vehicle comprising the device as described in the third aspect and at least one display device.

[0082] Optionally, the display device may specifically include a HUD or a display screen, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] FIG1 is a schematic structural diagram of a vehicle provided by the present application;

[0084] FIG2 is a schematic structural diagram of another vehicle provided by the present application;

[0085] FIG3 is a schematic structural diagram of a HUD provided by this application;

[0086] FIG4 is a schematic structural diagram of another HUD provided by this application;

[0087] FIG5 is a flowchart of a navigation method provided by the present application;

[0088] FIG6 is a schematic diagram of the structure of a prediction model provided by this application;

[0089] FIG7 is a schematic diagram of a navigation guidance generation method provided by the present application;

[0090] FIG8 is a flow chart of another navigation method provided by the present application;

[0091] FIG9 is a flow chart of another navigation method provided by the present application;

[0092] FIG10 is a flow chart of another navigation method provided by the present application;

[0093] FIG11 is a schematic diagram of another method for generating navigation guidance provided by the present application;

[0094] FIG12 is a schematic diagram of the structure of another prediction model provided by this application;

[0095] FIG13 is a schematic diagram of another method for generating navigation guidance provided by the present application;

[0096] FIG14 is a schematic diagram of another method for generating navigation guidance provided by the present application;

[0097] FIG15 is a schematic diagram of another method for generating navigation guidance provided by the present application;

[0098] FIG16 is a schematic diagram of a GUI for navigation guidance provided by the present application;

[0099] FIG17 is a schematic diagram of a GUI for navigation guidance provided by the present application;

[0100] FIG18 is a schematic diagram of a GUI for navigation guidance provided by the present application;

[0101] FIG19 is a schematic diagram of a navigation guide GUI provided by the present application;

[0102] FIG20 is a schematic structural diagram of a navigation device provided by the present application;

[0103] FIG21 is a schematic structural diagram of another navigation device provided in this application. DETAILED DESCRIPTION

[0104] The following will describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0105] The method provided in this application can be applied to vehicle driving scenarios to provide lane-level navigation guidance, providing navigation guidance that is more in line with actual application scenarios.

[0106] First, the method provided in this application can be applied to devices with display devices, such as vehicles or electronic devices. The display device can specifically include an electronic device display screen, a head-up display (HUD), a central control screen, an instrument screen, etc.

[0107] The method provided in this application can be applied to vehicle driving scenarios, which may specifically include user-driven vehicles, autonomous driving vehicles, or assisted driving vehicles.

[0108] The following describes the structure of the vehicle provided by this application or the vehicle used in it. The vehicle has one or more display devices, and lane-level navigation guidance can be displayed on the one or more display devices during vehicle driving, thereby providing vehicle drivers with navigation guidance that is more closely aligned with the real environment.

[0109] For example, FIG1 is a schematic diagram of the structure of a vehicle provided in an embodiment of the present application. FIG1 is a functional block diagram of a vehicle 100 provided in an embodiment of the present application. Vehicle 100 can be configured in either a fully or partially autonomous driving mode. For example, vehicle 100 can obtain environmental information about its surroundings through perception system 120 and, based on analysis of the environmental information, derive an autonomous driving strategy to achieve fully autonomous driving, or present the analysis results to a user to achieve partially autonomous driving.

[0110] Vehicle 100 may include various subsystems, such as an infotainment system 110, a perception system 120, a decision control system 130, a drive system 140, and a computing platform 150. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means. The models required for verification mentioned below in this application may include models for implementing various systems or subsystems in the vehicle.

[0111] In some embodiments, the infotainment system 110 may include a communication system 111 , an entertainment system 112 , and a navigation system 113 .

[0112] The data that needs to be displayed on the display screen mentioned in the following embodiments of the present application may include data generated by various systems in the vehicle during the operation of the vehicle, such as the operating status of each system or the collected data.

[0113] The communication system 111 may include a wireless communication system 111 that can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 111 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 111 may communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system may communicate directly with the device using an infrared link, Bluetooth, or ZigBee. The wireless communication system 111 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0114] The entertainment system 112 may include a central control screen, a microphone, and speakers. Users can use the entertainment system 112 to listen to the radio and play music in the vehicle. Alternatively, they can connect their mobile phone to the vehicle and project their phone's screen onto the central control screen, which may be touch-sensitive and user-operated. In some cases, the microphone can capture the user's voice signal and, based on analysis of the voice signal, enable the user to control certain aspects of the vehicle 100, such as adjusting the vehicle's temperature. In other cases, the speakers can play music to the user.

[0115] The navigation system 113 may include map services provided by a map provider to provide navigation for the vehicle 100. The navigation system 113 may be used in conjunction with the vehicle's global positioning system 121 and inertial measurement unit 122. The map services provided by the map provider may be standard definition (SD) maps, high definition (HD) maps, SD+ maps, SD Pro maps, or ADAS maps, although this application does not limit these.

[0116] In addition, in some driving scenarios, the vehicle's environmental information or navigation information can also be displayed on the central control screen. The central control screen can be one of the display screens mentioned in the following embodiments of the present application.

[0117] The perception system 120 may include several types of sensors that sense information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a global positioning system 121 (the global positioning system may be a GPS system, or a BeiDou system or other positioning systems), an inertial measurement unit (IMU) 122, a lidar 123, a millimeter-wave radar 124, an ultrasonic radar 125, and a camera 126. The perception system 120 may also include sensors of the internal systems of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). This detection and identification is a key function for the safe operation of the vehicle 100. The lane perception data mentioned in the embodiments of the present application may include but is not limited to data collected by sensors provided in the perception system.

[0118] The global positioning system 121 may be used to determine the geographic location of the vehicle 100 .

[0119] The inertial measurement unit 122 is used to sense changes in position and orientation of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 may be a combination of an accelerometer and a gyroscope.

[0120] LiDAR 123 may utilize laser light to sense objects in the environment in which vehicle 100 is located. In some embodiments, LiDAR 123 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0121] The millimeter wave radar 124 can use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the millimeter wave radar 124 can also be used to sense the speed and / or heading of the objects.

[0122] The ultrasonic radar 125 may sense objects around the vehicle 100 using ultrasonic signals.

[0123] The camera device 126 may be used to capture image information of the surrounding environment of the vehicle 100. The camera device 126 may include a monocular camera, a binocular camera, a structured light camera, a panoramic camera, etc. The image information acquired by the camera device 126 may include static image information or video stream information.

[0124] The decision control system 130 includes a computing system 131 that analyzes and makes decisions based on the information obtained by the perception system 120. The decision control system 130 also includes a vehicle controller 132 that controls the power system of the vehicle 100, as well as a steering system 133, throttle 134 and braking system 135 for controlling the vehicle 100.

[0125] The computing system 131 may process and analyze various information acquired by the perception system 120 to identify targets, objects, and / or features in the environment surrounding the vehicle 100 .

[0126] The vehicle controller 132 can be used to coordinate and control the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100.

[0127] The steering system 133 may be used to adjust the direction of the vehicle 100 .

[0128] The throttle 134 is used to control the operating speed of the engine 141 and thus the speed of the vehicle 100 .

[0129] The braking system 135 is used to control the deceleration of the vehicle 100. The braking system 135 can use friction to slow down the rotation speed of the wheels 144.

[0130] In addition, the decision control system 130 may also include an instrument screen (not shown in Figure 1). The instrument screen is usually set near the steering wheel of the vehicle, or set at a position in the vehicle that is convenient for the user to observe. The instrument screen can also be a type of display screen mentioned in this application.

[0131] In some possible scenarios, the data displayed on the central control screen and the instrument screen can be combined into one display screen, or the central control screen and the instrument screen can be the same display screen or partitioned display areas on the same display screen.

[0132] The drive system 140 includes components that provide powered movement for the vehicle 100. In one embodiment, the drive system 140 may include an engine 141, an energy source 142, a transmission system 143, and wheels 144.

[0133] Examples of energy source 142 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 142 may also provide energy to other systems of vehicle 100.

[0134] The transmission system 143 may transmit mechanical power from the engine 141 to wheels 144. The transmission system 143 may include a gearbox, a differential, and a drive shaft.

[0135] Some or all functions of the vehicle 100 are controlled by a computing platform 150. The computing platform 150 may include at least one processor 151 that can execute instructions 153 stored in a non-transitory computer-readable medium such as a memory 152. In some embodiments, the computing platform 150 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0136] The processor 151 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor 151 may include a graphics processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), a microcontroller unit (MCU), an application-specific integrated circuit (ASIC), or a combination thereof. The processor 151 may be located on a device remote from the vehicle and communicate wirelessly with the vehicle.

[0137] In some embodiments, memory 152 may contain instructions 153 (eg, program logic) that may be executed by processor 151 to perform various functions of vehicle 100 .

[0138] In addition to instructions 153, memory 152 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other similar vehicle data, as well as other information. This information may be used by vehicle 100 and computing platform 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0139] The computing platform 150 may control functions of the vehicle 100 based on input received from various subsystems (eg, the drive system 140 , the perception system 120 , and the decision control system 130 ).

[0140] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 100. For example, the memory 152 may be partially or completely separate from the vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0141] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 1 should not be understood as a limitation to the embodiments of the present application.

[0142] An autonomous vehicle, such as vehicle 100 above, traveling on a road can identify objects in its surroundings to determine adjustments to its current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and the speed adjustment to be made to the autonomous vehicle can be determined based on its respective characteristics, such as its current speed, acceleration, and distance from the vehicle.

[0143] Optionally, the vehicle 100 or a sensing and computing device associated with the vehicle 100 (e.g., computing system 131, computing platform 150) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0144] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0145] The vehicle 100 may be a car, truck, motorcycle, bus, ship, airplane, helicopter, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and the embodiments of the present application do not impose any particular limitation thereto.

[0146] Figure 2 shows a schematic diagram of the system architecture of a vehicle 100 based on some embodiments. The vehicle 100 includes multiple vehicle integration units (VIU) 11, a telematic box (T-BOX) 12, a cockpit domain controller (CDC) 13, a mobile data center (MDC) 14, a vehicle domain controller (VDC) 15 and other units, which can be used to support the vehicle to implement various control or information interaction functions.

[0147] The vehicle 100 also includes various types of sensors disposed on the vehicle body, including: a laser radar 21, a millimeter-wave radar 22, an ultrasonic radar 23, and a camera 24. The sensors mentioned in the embodiments of the present application may include these various types of sensors. Each type of sensor may include multiple sensors. It should be understood that although FIG2 shows the positional layout of different sensors on the vehicle 100, the number and positional layout of the sensors in FIG2 are merely illustrative, and those skilled in the art may reasonably select the type, number, and positional layout of the sensors as needed.

[0148] In conjunction with the vehicle structure shown in Figures 1 and 2 above, one or more display screens may be provided in the vehicle, such as a central control screen (i.e., a display screen typically provided in the central control position), an instrument screen (i.e., a display screen typically provided on the instrument panel or a display screen used to display the instrument panel), a head-up display (HUD), a rearview mirror display, or a rear-seat display, etc. Optionally, the HUD in the vehicle provided in this application may specifically be an augmented reality head-up display (AR-HUD) to display navigation instructions more vividly.

[0149] Taking the display of navigation instructions in the following HUD as an example, the navigation instructions displayed in the HUD can be centered on the driver's line of sight, and the content that the driver needs to pay attention to when driving the vehicle can be projected onto the windshield, so that the driver can know this information without having to lower his head or turn his head as much as possible, thereby reducing the occurrence of accidents and improving driving safety. Through the method provided in this application, navigation instructions that are more closely related to the actual road can be displayed on the windshield through the HUD, and navigation instructions that are more suitable for the actual road can be generated without relying on high-precision maps, which can adapt to more scenarios, such as scenarios with complex road conditions. In order to further improve the driving experience and display more comprehensive navigation instructions, AR-HUD can also be used to display navigation instructions.

[0150] Referring to FIG3 , for ease of understanding, the imaging structure of the HUD is described below. A HUD 30 may be provided in a vehicle and may include multiple components, such as a correction mirror 301 , a concave mirror 302 , and an image generation unit 303 , as shown in FIG3 .

[0151] The image generation unit 303 may be used to generate an optical signal of an image to be displayed;

[0152] The concave mirror 302 reflects the light signal to the correction mirror 301;

[0153] The correction mirror 301 reflects the incident light signal toward the windshield of the vehicle.

[0154] The concave mirror 302 and the correction mirror 301 form a light amplification circuit, thereby amplifying the light signal generated by the image generation unit 303 and displaying it on the windshield.

[0155] For AR-HUD, compared with traditional HUD, its virtual image distance (VID) is usually farther, as shown in the distance d in Figure 3, and the field of view (FOV) is wider. The FOV can be shown in Figure 4, that is, the angle formed by the driver's eyes (that is, the sight point 40 shown in Figure 4) and the virtual image range, which can be specifically divided into horizontal field of view angle θ and vertical field of view angle α. From the user's perspective, the AR-HUD display range seen by them is larger, and the range indicated for the actual scene is also larger. That is, in the method provided in this application, AR-HUD can be used to display navigation guidance, thereby displaying lane-level navigation guidance with a wider field of view and a closer fit with the actual scene for the user.

[0156] Below, the method flow provided by this application is first introduced.

[0157] Refer to FIG5 , which is a flowchart of a navigation method provided by the present application.

[0158] It should be noted that the method provided in this application can be applied to devices such as vehicles or vehicle-mounted terminals. For ease of understanding, the devices to which the method provided in this application is applied are collectively referred to as electronic devices. It should be understood that the electronic devices mentioned below may specifically include vehicles or other devices with display devices or devices connected to display devices, etc., and the specific application scenarios can be adjusted according to the actual application scenario, and this application does not limit this.

[0159] In addition, it should be noted that the navigation instructions can be updated in real time during the vehicle's movement. In the implementation mode of this application, for example, the generation method of one frame of navigation instructions is introduced. The generation frequency or update frequency of the navigation instructions can be determined according to the actual application scenario, and this application does not limit this.

[0160] 501. Obtain navigation data and lane perception data.

[0161] Among them, the navigation data is specifically divided into map data and navigation path. The map data may include data of a map of the vehicle's driving area. The map data may be pre-stored in the electronic device or loaded from a server when the navigation is turned on; the navigation path may include the path of the vehicle when it is driving on the map. The path may be a path generated based on the user's trigger operation, or a path recommended based on the user's historical data, etc., and can be used to guide the path that the vehicle needs to travel.

[0162] Lane perception data may include information about at least one lane obtained by performing lane perception based on data collected by sensors installed in the vehicle. The lane perception data may also be referred to as perception data or other names, etc.

[0163] In one possible implementation, lane perception data can be determined based on data collected by the vehicle's sensors. For example, lane information can be determined based on lane lines included in the lane perception data, or lane perception data can be obtained by outputting the lane perception data as input to a pre-trained lane perception model. The lane perception data can specifically include information about at least one lane in the vehicle's environment, including lane lines, lane location, or number of lanes. The at least one lane can specifically include the vehicle's current lane, an adjacent lane, or a lane in front of the vehicle. Therefore, in the method provided in the embodiments of the present application, lane information in the vehicle's environment can be perceived based on the perception of the vehicle's environment.

[0164] In one possible implementation, the lane perception data may specifically include data collected by sensors in the vehicle. For example, the sensors may include but are not limited to image sensors, radars, infrared sensors, or depth sensors, etc., that is, the lane perception data may also include but is not limited to data of the environment perceived by sensors such as image sensors, radars, infrared sensors, or depth sensors. Therefore, in the implementation of the present application, the data perceived by the sensors can be used to perceive the specific conditions of the environment in which the vehicle is located, so as to determine lane-level navigation guidance based on the perception results without relying on HD maps. In addition, optionally, the vehicle's historical motion information can also be collected, so that when subsequently predicting the vehicle path, lane-level positioning, etc., the vehicle's motion data can be combined to obtain more accurate calculation results, thereby improving the accuracy of navigation guidance. The vehicle's historical motion information may specifically include information such as the vehicle's speed, steering angle, and driving direction.

[0165] 502. Generate lane-level navigation guidance based on the navigation data and lane perception data.

[0166] The vehicle's environment can then be perceived using map data and lane perception data, and lane-level guidance can be generated based on the perceived scene, navigation data, and map data, thereby providing users with more accurate navigation guidance that is more relevant to the actual scene.

[0167] Among them, navigation guidance can be used to instruct the vehicle to travel according to the navigation path. It can perceive the roads in the vehicle's environment based on lane perception data and map data. When guiding the navigation path, it can follow the perceived lanes; that is, the navigation guidance can be used to identify the drivable lanes corresponding to the navigation path in the environment, thereby realizing lane-level navigation guidance.

[0168] That is, in the method provided in this application, the roads on which the vehicle can drive in real scenarios can be identified based on lane perception data and map data. Specifically, these can be lanes with divided lane lines, or lanes without divided lane lines on which the vehicle can actually drive; lane-level navigation guidance is generated based on the perceived lanes, thereby providing navigation guidance that is more adapted to the real scenario without relying on high-precision maps.

[0169] This application exemplarily introduces some possible ways of generating navigation instructions. There may be multiple ways of generating navigation instructions. For example, for different scenarios, you can adaptively select an appropriate way to generate navigation instructions. Some possible generation methods are introduced below.

[0170] It should be noted that the present application may include multiple methods for generating navigation guidance. For ease of understanding, the following describes the methods for generating navigation guidance based on path prediction and lane change decision-making. It should be understood that the methods based on prediction models and lane change decision-making mentioned below can be implemented in combination, or different generation methods can be selected for different scenarios to generate navigation guidance. The appropriate generation method can be selected based on the actual application scenario. In other words, the present application provides multiple different generation methods for generating navigation guidance, and there is no limitation on the combination of different generation methods.

[0171] Method 1: Based on path prediction

[0172] This application provides a possible implementation method that can use a pre-trained prediction model to predict the trajectory of a vehicle, such as using map data and historical vehicle motion information (i.e., vehicle motion data within a historical period) as input to the prediction model, outputting the vehicle's predicted path, and combining the predicted path map data and lane perception data to generate navigation guidance. In this implementation method, a neural network can be combined to predict the vehicle's predicted path, that is, the path that the vehicle can travel within a certain period of time in the future, so as to generate more accurate navigation guidance based on the predicted path by combining the perception data and map data of the actual scene.

[0173] First, the structure of possible prediction models is introduced.

[0174] Referring to FIG6 , the prediction model may include but is not limited to one or more of the following modules:

[0175] A map encoder 601 is used to extract features from input map data to obtain map features;

[0176] A motion encoder 602 is configured to extract features from the vehicle's historical motion information to obtain motion features. The historical motion information may include information generated by the vehicle's motion over a previous period of time, such as the vehicle's speed, steering angle, and historical path.

[0177] The fusion encoder 603, also known as the map-motion encoder, is used to extract features from the map features and the motion features to obtain global features;

[0178] A reference path generator 604 is configured to generate a reference path for the vehicle based on the predicted path of the previous frame. The navigation instructions during vehicle driving typically change in real time and can be updated on a per-frame basis. The predicted path of the previous frame is the predicted path output by the prediction model when generating or updating the navigation instructions of the previous frame. The reference path is a possible driving trajectory of the vehicle within a future period of time, generated based on the vehicle's historical motion information.

[0179] The path decoder 605 is configured to generate a predicted path based on the global features and the reference path.

[0180] In the implementation manner of the present application, a pre-trained prediction model can be used, and information of various dimensions can be used as input of the prediction model to output the current predicted path that the vehicle can travel, so that when navigation guidance is subsequently generated, more accurate navigation guidance can be generated based on the predicted path.

[0181] To improve the accuracy of the predicted path, the method provided in this application can further combine lane perception data to enhance environmental perception, thereby improving the accuracy of the predicted path by combining the enhanced environmental perception. That is, this application also provides an optional embodiment, in which the aforementioned prediction model can also include:

[0182] A visual encoder 606 is configured to extract features from input lane perception data (e.g., collected images or point cloud data of the environment) to obtain visual features;

[0183] The path decoder 605 is further configured to generate a predicted path based on visual features, global features, and path features.

[0184] Therefore, in the embodiments of the present application, the environment can be perceived by collecting lane perception data of the vehicle's environment, thereby generating a more accurate predicted path in combination with the environmental perception results.

[0185] In addition, in order to further improve the accessibility of the predicted path, the present application can also combine the vehicle's motion model to obtain a predicted path that is more consistent with the vehicle's physical motion model. That is, the present application also provides a possible implementation method, and a motion constraint path can also be generated based on the vehicle's physical motion model. The reference path generator is also used to combine the predicted path of the previous frame and the motion constraint path to obtain a reference path. The motion constraint path is a path generated based on the physical motion model of the vehicle. The motion constraint path is used as a constraint for outputting the reference path, thereby avoiding outputting an inaccessible path for the vehicle.

[0186] Further optionally, the method of generating navigation guidance may specifically include: for a lane change path, that is, if the predicted path is a scenario in which the vehicle needs to change lanes, such as the vehicle changes lanes when traveling straight or the vehicle changes lanes when passing an intersection, etc., the first path and the second path in the navigation path may be obtained, the first path being the path for the vehicle to enter the target lane (that is, the lane to which the vehicle needs to change lanes) in the navigation path, and the second path being the path for the vehicle to exit the curve when traveling toward the target lane in the navigation path. Subsequently, a plurality of line points are determined based on the predicted path, the first path, and the second path, and the plurality of line points may include at least a curve entry point and an exit point, the curve entry point being a point in the curve entry path (determined based on the first path) for the vehicle to enter the curve, and the exit point being a point in the curve exit path (determined based on the second path) for the vehicle to exit the curve. When the vehicle is traveling, navigation guidance may be generated in real time based on the plurality of line points. Typically, a navigation path only plans the vehicle's driving path according to a map, so there may be deviations between the navigation path and the vehicle's actual driving path. Therefore, in the solution provided by this application, a prediction is made for the path that the vehicle will travel in an actual application scenario. When determining the travel point, the first path can be adsorbed into the predicted path, or the first path and the predicted path can be fused, or a path in the predicted path that is parallel to the first path can be used as the first path, etc., so that the navigation path is adsorbed into a more accurate predicted path, thereby obtaining a first path that can represent the possible entry path of the vehicle in the actual application scenario. When generating navigation guidance, the first path that matches the actual scenario can be combined to generate navigation guidance, thereby achieving more accurate guidance.

[0187] For example, Figure 7 shows the curve entry and exit paths. When a vehicle needs to turn, the vehicle's current position is point 1, point 2 is the predicted turning point, and point 3 is the point after the vehicle exits the curve. The path formed by points 1 and 2 is the curve entry path, and the path formed by points 2 and 3 is the curve exit path. For ease of understanding, the curve entry path can be understood as the path formed from the vehicle's current position to the midpoint of the curve when entering the curve. The curve mentioned in this application is a road that requires the vehicle to have a certain steering angle when turning, turning, or changing lanes. The curve exit path is the path from the end point of the curve to the straight lane after exiting the curve.

[0188] Therefore, in the implementation manner of the present application, navigation guidance can be generated in point units, and lane-level guidance can be updated in real time based on the vehicle's travel points while the vehicle is traveling, thereby providing the vehicle with clearer navigation guidance when the vehicle changes lanes.

[0189] Optionally, when determining multiple travel points, sampling can be performed from the initially determined multiple travel points to select multiple travel points that are more compatible with the vehicle's actual travel path. These multiple sampled points can then be fitted to generate a steering guidance curve. Therefore, in the embodiments of the present application, the generation of unreachable curves can be avoided by using travel point sampling.

[0190] Optionally, to smooth the generated navigation guidance and provide more accurate guidance, a curve model can be used to generate the guidance, so that the navigation path more closely matches the vehicle's actual driving path. For example, when generating navigation guidance based on multiple points, the curve model can be used to fit these points to generate a turning guidance curve, and navigation guidance can be generated based on this turning guidance curve. For example, the turning guidance curve can be used as the navigation guidance, or as part of the navigation guidance.

[0191] Specifically, when the vehicle changes lanes, in order to provide more accurate navigation guidance, a more clearly pointed navigation guidance can be obtained by fitting a broken line. For example, a broken line point set (also referred to as a line point set, a line point broken line, etc.) can be obtained first. The broken line point set can include the current position of the vehicle, points on the untraveled path in the predicted path, and line points that the vehicle has not reached among multiple line points. A broken line is formed between the multiple line points, wherein the points on the untraveled path in the predicted path are points on the path that the vehicle has not yet reached in the predicted path, and the line points that the vehicle has not yet reached among multiple line points are points that the vehicle has not yet reached among the multiple line points determined above; a broken line is fitted in the broken line point set, and a steering guidance curve is generated based on the collected broken line. Therefore, in the embodiment of the present application, a smooth curve can be generated by fitting a broken line, so that a reachable curve can be generated, thereby improving the accuracy of the generated navigation guidance.

[0192] Optionally, in order to make the vehicle's turn-in and turn-out navigation more accurate, when generating a turning guidance curve during vehicle driving, a curve pointing to the turn-in point or the turn-out point is generated based on the fitting of the line points, so that the vehicle's driving destination is more accurate. For example, when the vehicle reaches the turn-in point, the line points in the first path are fitted according to the curve model to generate a turning guidance curve pointing to the turn-out point. When the vehicle reaches the turn-out point, the line points in the second path are fitted according to the curve model to generate a turning guidance curve pointing to the target lane. In the embodiment of the present application, when the vehicle changes lanes, it can be fitted based on the line points to generate a guidance curve for the curve, thereby improving the guidance accuracy for the curve. For complex road conditions, such as when there are multiple parallel lanes or no lane lines after turning, more accurate navigation guidance can be achieved based on the method provided in the present application.

[0193] Method 2: Lane-changing decision-making

[0194] This application also provides a possible implementation method for determining a vehicle's lane change decision based on navigation data, map data, and lane perception data. The lane change decision indicates whether the vehicle should change lanes; and generating navigation guidance based on the lane change decision. Therefore, in this implementation method, whether the vehicle should change lanes can be determined based on the vehicle's environment and the path indicated by the navigation path, and navigation guidance can be generated based on the lane change decision. This is applicable to scenarios where the vehicle needs to change lanes.

[0195] Optionally, when determining the lane change decision, the probability of the vehicle traveling from the current position to each lane can be calculated, and based on the probability, the probability corresponding to each possible transfer path of the vehicle can be determined, and based on the probability, whether the vehicle changes lanes can be determined. Specifically, the lane in which the vehicle is currently traveling can be determined based on map data and lane perception data; the probability of the vehicle transferring from the current lane to each of N lanes can be obtained based on the lane perception data and navigation data, where N is a positive integer, and the N lanes are the lanes in the vehicle's environment, such as the lanes determined from the lane perception data; the vehicle's transfer path can be determined based on the probability of the vehicle transferring from the current lane to each of the N lanes; and the lane change decision can be obtained based on the transfer path. This is equivalent to the method provided in the embodiment of the present application, in which the probabilities corresponding to each possible transfer path of the vehicle can be calculated based on a Markov chain, thereby selecting a suitable transfer path based on the probabilities of each transfer path, thereby more accurately determining the lane change decision.

[0196] Furthermore, the method provided by this application does not rely on high-precision maps, so there may be inaccurate data in some map areas, such as lane positioning in the map data not matching the actual lane position, or some drivable roads not existing in the map data. Therefore, to address these situations, this application divides the map into multiple regions, including high-confidence regions, i.e., regions where the degree of match between the positioning in the map data and the positioning in the collected lane perception data is greater than a certain value. Specifically, map data and lane perception data can be used as inputs to the classification network, and the classification results of each area, that is, the confidence of each area, can be output; the area can be divided into high-confidence area and low-confidence area, and the confidence of the high-confidence area is higher than that of the low-confidence area. For example, the area with confidence higher than the confidence threshold can be regarded as the high-confidence area, and the area with confidence not higher than the confidence threshold can be regarded as the low-confidence area; then the map is expanded based on the high-confidence area to obtain the expanded map. For example, the high-confidence area usually has a higher accuracy, so it can be expanded based on the high-confidence area combined with the lane perception data to obtain a more comprehensive expanded map; and based on the expanded map, the probability of the vehicle transferring from the current lane to each of the N lanes is calculated.

[0197] Specifically, the method provided in the embodiments of this application uses a classification network to match lane locations in a map with lane locations in the actual scene, outputting high-confidence regions with a high degree of match and low-confidence regions with a low degree of match. To improve overall map accuracy, the map is expanded based on the high-confidence regions, resulting in an expanded map with a higher degree of match between lane locations and lane locations in the actual scene. For example, for maps with low map accuracy or unclear lane lines, the method provided in this application can achieve accurate lane location without relying on high-precision maps, thereby generating subsequent, more accurate navigation guidance based on more accurate lane location.

[0198] Optionally, when generating navigation guidance, if the current lane change decision is to change lanes, navigation guidance can be generated based on the transfer path, and the guidance displayed on the navigation guidance display interface can be updated; if the current lane change decision is to maintain the current lane, the straight guide line can be maintained as the navigation guidance guidance, and the straight guide line is the navigation guidance instructing the vehicle to maintain the current lane and drive straight.

[0199] 503. Display navigation instructions.

[0200] After the navigation instructions are generated, they can be displayed on a display device. For example, the navigation instructions can be projected into the vehicle via a HUD, or the actual scene or a rendering of the actual scene can be displayed on the display screen and the navigation instructions can be superimposed.

[0201] In this application, the vehicle's lane information in real-world scenarios can be perceived based on sensor data collected by the vehicle's sensors, generating lane-level navigation guidance that is more tailored to the real-world scenario. Furthermore, using sensor data to perceive lanes in real-world scenarios allows for more accurate lane positioning and navigation guidance without relying on high-precision maps.

[0202] The above is an overview of the method provided by this application. The following is a more detailed introduction to the method provided by this application in combination with specific application scenarios.

[0203] The method provided in this application can be applied to vehicle driving scenarios, such as autonomous driving, assisted driving, or user driving. Assisted driving can include assisting users in driving the vehicle, or it can include auxiliary functions during autonomous driving, where the vehicle automatically drives.

[0204] For example, the architecture of the method provided in this application can be seen in Figure 8. The method provided in this application can be deployed in a vehicle controller or a vehicle-mounted terminal connected to the vehicle. As shown in Figure 8, the method provided in this application is deployed in an AR engine.

[0205] First of all, the input data may specifically include SD basic navigation data, positioning data, lane perception data, etc. The input data may be data read from the storage, or may include data from the vehicle's intelligent driving system or automated driving system (ADS), etc., or may also be directly collected through the interface provided by the vehicle. For example, as shown in the vehicle structure in Figure 1 or Figure 2 above, when the vehicle is running, the various VIUs in the vehicle can transmit data to each other, or transmit data to the vehicle's controller, etc., and the data required for this application can be collected through the interface in the vehicle, such as data collected by vehicle sensors, such as information about the vehicle's environment collected by lidar, millimeter-wave radar, infrared sensor, depth sensor or image sensor.

[0206] The AR engine is the module within the device that drives the AR display. As shown in Figure 8, it can be used to generate map guidance information for navigation modes such as straight ahead, intersections, or lane changes, as well as lane-level positioning and corresponding guide lines.

[0207] After obtaining the navigation guidance information, the navigation guidance can be displayed on the AR device. Specifically, the navigation guidance can be displayed on the display screen, such as displaying the real-time vehicle environment image and overlaying the AR navigation guidance on the real-time environment image; or the navigation guidance can be displayed in the AU-HUD.

[0208] In the following implementation manner of the present application, for example, the display of navigation instructions in AU-HUD is taken as an example for illustration, but it is not intended to be limiting. The AR-HUD mentioned below may also be replaced by a display screen, which may be adjusted according to the actual application scenario.

[0209] For example, the method flow provided in this application may be shown in FIG9 .

[0210] First, the input data may include vehicle motion parameters (such as vehicle speed, steering angle, etc.), navigation information (including map data), and lane perception data (such as data collected by vehicle sensors).

[0211] Navigation information and lane perception information can be used to model complex intersections and construct local maps. Navigation guidance is generated based on these results. For scenarios without lane markings, lane estimation and prediction, guidance timing decisions, and guide line generation are all possible. For scenarios with lane markings, lane-level positioning and lane change decisions can be achieved. Navigation guidance is displayed in real time on the AR-HUD.

[0212] The following is a more comprehensive introduction to the process of the method provided in this application in combination with specific application scenarios.

[0213] Referring to FIG10 , a flowchart of another navigation method provided by the present application is shown as follows.

[0214] It should be noted that, for ease of understanding, the aforementioned multiple navigation guidance methods are applied to different scenarios respectively. For example, the aforementioned method one can be applied to the intersection scenario, and the aforementioned method two can be applied to the straight-ahead lane change scenario, etc. Of course, in actual application scenarios, method one can also be applied to the straight-ahead lane change scenario, and method two can be applied to the intersection scenario, etc., or method one and method two can be integrated in different scenarios, etc. The specific application can be adjusted according to the actual application scenario, and this application does not limit this.

[0215] 1001. Obtain vehicle motion data, navigation data, and lane perception data.

[0216] First, after triggering navigation for the vehicle's itinerary, the data required for navigation can be collected in real time while the vehicle is driving. The required data can be specifically divided into vehicle motion data, navigation data, and lane perception data.

[0217] Vehicle motion data can specifically collect factors such as the vehicle's position and orientation in the world coordinate system, or data collected from sensors such as GNSS, IMU, and WSS wheel speedometers, such as the vehicle's speed, acceleration, angle, angular velocity, etc.

[0218] Navigation data may include the navigation path in the map generated based on the map data when navigation is started. Specifically, it may include, but is not limited to, the vehicle's planned path in the SD map, lane traffic attributes (such as whether the lane allows turns or only allows straight driving, etc.), etc.

[0219] Environmental perception data can specifically come from environmental perception data such as images or point clouds, and can specifically include lane lines, lane curbs, lane line types (such types include but are not limited to dotted lines, solid lines, semi-solid and semi-dotted lines, white lines, yellow lines or double lines, etc.), etc.

[0220] While the vehicle is driving, the sensors installed in the vehicle can collect information about the vehicle's location and current status in real time, so that the navigation guidance can be updated in real time based on the navigation path.

[0221] 1002. Determine whether the vehicle has reached an intersection. If so, execute step 1003; if not, execute step 1009.

[0222] During the vehicle's driving process, it can be determined whether the vehicle has reached an intersection. If the vehicle has reached an intersection, the above-mentioned method 1 can be used to generate navigation instructions, that is, execute step 1003. If the vehicle has not reached an intersection, the above-mentioned method 2 can be used to generate navigation instructions, that is, execute step 1009.

[0223] Specifically, the method for determining whether a vehicle is at an intersection can be determined by positioning the vehicle in a map, identifying whether it is an intersection based on collected environmental perception data, or combining the vehicle's positioning results in the map with environmental perception results to determine whether the vehicle is at an intersection. The specific method can be adjusted according to the actual application scenario. For example, the vehicle's positioning in the map can be used to determine whether the vehicle is at an intersection, or the environmental perception data can be used as input to a pre-trained recognition network to identify whether the vehicle is at an intersection. Alternatively, the recognition results of the vehicle's positioning can be fused with the recognition results based on the environmental perception data to more accurately identify whether the vehicle is at an intersection.

[0224] 1003. Kinematic path prediction.

[0225] For the vehicle motion data collected above, the high-dimensional combination space of physical quantities such as the vehicle's short-term speed, acceleration, angular velocity, and angular acceleration can be predicted based on the vehicle's kinematic models, such as the constant turn rate and velocity model (CTRV) and the constant turn rate and acceleration model (CTRA), so as to calculate the predicted path that the vehicle may produce based on these parameters.

[0226] Taking the Constant Turn Rate and Velocity (CTRV) model as an example, assuming that the vehicle’s position at time t is (x t ,y t ,θ t ), then the vehicle posture (x t+1 ,y t+1 ,θ t+1 ) can be expressed as:

[0227] Among them, v t represents the speed of the vehicle at time t, w t Represents the angular velocity of the vehicle at time t. The change in posture between time t and time t+1 can determine the path of the vehicle from time t to time t+1.

[0228] For example, the effect achieved based on the motion model can be shown in Figure 11. The vehicle motion model constructs mathematical equations based on the vehicle sensor input of the tth frame and before, and derives and calculates the possible future motion trajectory of the vehicle without training, realizing knowledge-driven path prediction and avoiding outputting unreachable paths.

[0229] 1004. Streaming path prediction.

[0230] Map data, environmental perception data, and kinematic predicted paths can be used as inputs to the prediction model, which outputs a predicted path.

[0231] The input to the streaming trajectory prediction model includes multiple components: vehicle sensor input from frame t and earlier (including but not limited to images or point clouds captured by onboard cameras or LiDAR), multiple output trajectories from the streaming trajectory prediction model for frame t-1, the output trajectory of the vehicle motion model for frame t, map information, and vehicle motion information over the past period of time. The output of the streaming trajectory prediction model is multiple possible predicted paths for the vehicle over a period of time. Streaming trajectory prediction models typically use deep learning-based methods and require pre-training.

[0232] For example, as shown in Figure 12, the predicted path of the vehicle motion model in the tth frame and the predicted path of the (t-1) frame can be used as the input of the prediction model, and the predicted path of the tth frame is output; when predicting the predicted path of the next frame (t+1), the predicted path obtained based on the vehicle motion model can be used as the input of the prediction model, and the predicted path of the (t+1) frame is output, and so on.

[0233] Specifically, the prediction model can refer to the aforementioned Figures 6 to 7 and the corresponding descriptions, and similarities are not repeated here.

[0234] Map encoder: Receives vectorized map information, divides the road network information in the map into segments according to the minimum length (such as 5 meters), and saves it as N map ×D map Tensor, N map is the number of map elements divided into segments, D map is the number of attributes of each map element (including starting point coordinates, lane categories, etc.), and the graph neural network is used as the map encoder to perform hierarchical encoding on the above tensor to obtain N map ×D hidden map features.

[0235] Motion encoder: receiving shape is T×D car Vehicle historical motion information tensor, T is the length of the historical trajectory, D car is the number of historical motion attributes (including coordinates, speed, orientation angle, etc.), and LSTM is used as a motion encoder to encode the motion information to obtain T×D hidden vehicle motion characteristics.

[0236] Map-motion encoder (i.e. fusion encoder 603): concatenates the obtained map features and vehicle motion features to obtain (N map +T)×D hidden The input is then processed using the self-attention mechanism to obtain the form (N map +T)×D hidden global characteristics.

[0237] Reference trajectory generator: receives K×T future ×2, K is the number of prediction paths, T future To predict the time length, 2 represents the two-dimensional coordinate information, and M×T future ×2 predicted path of the current frame motion model. The reference trajectory generator concatenates the two and transforms them to the current vehicle coordinate system, removes redundant predictions, and completes the missing predictions to obtain (K+M)×T future ×2 reference trajectory.

[0238] Image encoder: Receives an H×W×3 RGB vehicle foreground image and processes it using a convolutional neural network to obtain h×w×c image features.

[0239] The path decoder uses (K+M)×T future The reference trajectory of ×2 is generated by MLP (K+M)×D hiddenFirst, we use the query and the global features obtained by the map-motion encoder to do cross-attention and get (K+M)×D hidden The query feature 1 of the vehicle is then projected onto the vehicle foreground image using the vehicle camera parameters and assuming the trajectory height coordinates. Based on the position of the trajectory in the foreground image, feature pooling and deformable convolution are used to extract the image features of each reference trajectory and perform a projection transformation to obtain (K+M)×D hidden The trajectory feature is added to the query feature 1 and projected to obtain (K+M)×D hidden The query feature 2. The above feature extraction process is repeated R times, and the final (K+M)×D hidden Output (K+M)×T through an MLP future ×2 predicted paths and their confidence (K+M)×T future ×1. During the training process, the predicted path and the true value trajectory are trained using the minimum mean square error and winner-take-all strategy. During the inference process, (K+M)×T future The K trajectories with the highest confidence are selected from the ×2 predicted path as the input for the next moment.

[0240] 1005. Navigation line point matching.

[0241] In the method provided in this application, the vehicle motion predicted posture is combined to re-adaptively construct the navigation points and reduce the error between the navigation points and the real environment.

[0242] For example, as shown in Figure 13, based on the segment labels of the SD map, the points in the navigation path at the intersection can be divided into entry points (such as points 1-2 in Figure 13) and exit points (such as points 2-3 in Figure 13). The movement of the vehicle through the intersection can be divided into three stages, as shown in (a), (b), and (c) in Figure 13:

[0243] For (a): Before the vehicle enters a curve, the entry point is attached to the vehicle (Attach): Before entering the curve, the vehicle can follow the predicted path along the curve, meaning the direction of the curve vector has a high degree of confidence. At this point, the entry point is laterally translated to the end of the predicted path. The exit point remains unchanged due to the lack of additional reference information, and the two are recombined to generate a recombined point (such as points 4-5-6 in (a) of Figure 13).

[0244] For (b): Detachment of the vehicle's track point during a turn: During a turn, the vehicle travels in an open area at the intersection, not following the entry or exit track point. In this case, the reorganized track point remains unchanged (e.g., points 4-5-3 in Figure 13 (b)), guiding the vehicle to the correct exit track point.

[0245] For (c): When the vehicle completes a turn, the exit point is attached to the vehicle's predicted path. After the turn is completed, the vehicle follows the exit path, indicating a high confidence level for the exit vector direction. At this point, the exit point is laterally translated to the end of the predicted path, while the entry reassembly point remains unchanged. The two are then reassembled to create the reassembly point (see points 4-6-7 in (c) of Figure 13).

[0246] Furthermore, for navigation data, when the vehicle enters a new intersection, the reorganized line points are initialized as the original navigation line points, and are split into the curve line points according to the road segment labels. e The broken line P at the exit o .

[0247] For the predicted path outputted in step 1004, project its end to the reorganized row point, and denote the projection point as p r , the projection point p r The vector direction is θ r .

[0248] When the projection point belongs to the curve point and the vector direction of the projection point is close to the direction of the predicted path end, the vehicle real-time position p is calculated. v Lateral error σ from the turning point r =p v -p r , the entry point is translated horizontally to the end of the predicted path, then the translation of the entry point broken line P′ e =P e +σ p .

[0249] Regroup and Generate a reorganized line point polyline P. For P' e ,remember Indicates As endpoints, with vector is the direction of the ray, Indicated by point with dot For the line segment with endpoints. o ,remember Indicates As endpoints, with vector is the direction of the ray, Indicated by point with dot is the line segment with endpoints. Calculate and The intersection point P t , then reorganize the line point polyline Among them, nt and mt represent P t exist The intersection index in .

[0250] When the vector direction of the projected point is not close to the vehicle's orientation, the reconstructed row points remain unchanged.

[0251] When the projection point belongs to the exit point and the vector direction of the projection point is close to the direction of the predicted path end, the vehicle real-time position p is calculated. v Lateral error σ from the exit point r =p v -p r , the exit point is horizontally translated to the end of the predicted path, then the translation of the exit point broken line P′ o =P o +σ p .

[0252] Regroup and Generate a reorganized line point polyline P. For P e ,remember Indicates As endpoints, with vector is the direction of the ray, Indicated by point with dot For the line segment with endpoints. o ,remember Indicates As endpoints, with vector is the direction of the ray, Indicated by point with dot is the line segment with endpoints. Calculate and The intersection point P t , then reorganize the line point polyline Among them, nt and mt represent P t exist The intersection index in .

[0253] 1006. Guide line point search.

[0254] More specifically, for the reorganized points in step 1005 and the predicted path in step 1004, the first projection point of the vehicle's current position on the reorganized points and the second projection point of the predicted path end on the reorganized points are calculated. The predicted path, the navigation points between the first projection point and the second projection point are weighted fused to calculate the near-end guidance point P n Calculate the intersection of the vehicle's proximal guide point and the reorganized point. If the intersection exists, use the intersection as the starting point. If the intersection does not exist, use the second projection point as the starting point and take the point after the starting point in the reorganized point as the distal guide point P. f Combine the near-end guiding point and the far-end guiding point to obtain the vehicle's current guiding point P = {p n ,P f}.

[0255] In addition, as shown in FIG14 , for the local guide line in the navigation guidance, the polyline l formed by the control points of the local guide line is l={p t ,p′ t ,p i ,p i+1 ,…,p n}By the vehicle's current position p t , trajectory prediction position p t ′, the reorganization point where the vehicle has not arrived {p i ,p i+1 ,…,p n}(0≤i≤n).

[0256] will p t ′ is mapped to the reorganized line points to obtain the forward road line point sequence {p c ,p c+1 ,…,p n} (as shown in Figure (a) point {p1, p2}). Calculate the intersection of the search space (the possible future trajectory of the vehicle) and the forward road line point as a new control point to avoid reverse distortion of the guide line, such as a right-turn curve with a left-turn trend. If there is a legal intersection point p k When i=k, combination p t , p′ t 、p k The forward road line point after the intersection forms the guide line control point polyline l = {p t ,p′ t ,p k ,p k+1 ,…,p n}; If there is no legal intersection, i = c, combination p t , p′ t Together with the forward road points, a guide line control point polyline l = {p t ,p′t ,p c ,p c+1 ,…,p n}.

[0257] 1007. Generate guide lines.

[0258] After collecting the line points, the line points can be sampled using methods such as equal-interval sampling and curvature threshold sampling. Guideline control points are formed based on the sampled points to ensure the stability of the guide line between frames. Specific curve models that can be used include, but are not limited to, B-spline curves, Bezier curves, and other curve models to generate smooth turning guide lines. Taking the B-spline curve as an example, the guideline curve equation C(t) can be expressed as:

[0259] Among them, B i,deg (t) is the basis function, deg is the curve order, knot is the node table, P i (P i ∈P) represents the control point.

[0260] After the guide line of the navigation guidance is generated, the navigation guidance may be displayed, ie, step 1015 is executed.

[0261] 1008. High confidence region division.

[0262] In non-intersection scenarios, lane-level matching and positioning can be performed to accurately identify the vehicle's next driving path.

[0263] First, the map represented by the map data can be divided into multiple regions. Regions with high confidence can be selected, such as those with confidence levels above a confidence threshold, or where the degree of match between the vehicle's position in the map and the vehicle's position as detected by the vehicle's sensors exceeds a certain threshold. For example, the lane-level matching positioning model is initialized when the vehicle is in the left or right edge lanes of the road, or when the perception data accurately identifies all lane lines in a good field of view. At this point, the SD map route data and positioning data are aligned.

[0264] Among them, the high confidence area can be determined by using a lightweight image classification network such as MobileNetV3 for identification, which can obtain more accurate lane-level initialization information. The current positioning position is recorded as point A xy , the location bound to the SD route is recorded as point B xy From the perception information, we can know the current lane number X and the left offset distance W in the current lane l , Current lane width W all , the current road direction h can be known from the positioning information.

[0265] The left boundary point of the current road is recorded as Cxy , point A xy and point C xy Distance between: dist=W all *(X-1)+W l

[0266] Then point C xy : C x =A x +dist*(cos(h-90)-sin*(h-90)) C y =A y +dist*(cos(h-90)+sin*(h-90))

[0267] Then determine the lateral relative position of the SD route on the current road, that is, the offset α xy α xy =C xy -B xy

[0268] Therefore, based on the method provided in this application, lanes in actual application scenarios can be accurately identified, thereby compensating for the lane positioning accuracy in the map and obtaining accurately positioned lane information.

[0269] 1009. SD map expansion.

[0270] The collection principle of map making can be used to infer the offset α in a limited time and space ahead xy The Euclidean distance will not change. Combining the navigation information such as lane line perception, positioning, number of lanes, etc., the current lane-level positioning map is constructed based on the skeleton information of the SD map route. The current position is recorded as point A xy , the location point bound to the SD route is recorded as B xy , by the offset α xy Combined with the current road direction h, it can be inferred that the intersection of the normal direction of the current vehicle's driving direction and the left boundary of the road is point C. x =B x +||α xy ||*(cos(h-90)-sin*(h-90)) C y =B y +||α xy ||*(cos(h-90)+sin*(h-90))

[0271] The current number of navigation lanes is N, and the center points of each lane are P1, P2…P N , current lane width W all

[0272] The center points of each lane obtained based on the current method and the perceived lane lines are weightedly fused to determine the final center points of each lane.

[0273] 1010. Lane probability matching.

[0274] Perception, navigation, and positioning results all have a certain impact on the formation of the probability of the current lane position. The probability distribution of the vehicle's lane position changes with vehicle movement and perception and navigation information. The probability is composed of the initial probability, observation probability, and transition probability. The initial probability is evenly distributed among the number of lanes:

[0275] The observation probability consists of the positioning observation probability P1, the lane number observation probability P2, and the lane line type observation probability P3:

[0276] The lane number observation probability P2 and lane line type observation probability P3 can be obtained by training the Bayesian network. The total observation probability is: P o =P1*P2*P3

[0277] The cosine distance is used for the transfer probability, and the angle between the line connecting the center point of each lane at the previous moment and the center point of each lane at the current moment and the positioning displacement line segment is θ:

[0278] Therefore, in an embodiment of the present application, the probability of a vehicle transferring between lanes can be used so that the location to which the vehicle may transfer can be predicted based on the probability.

[0279] 1011. Sequence estimation.

[0280] The method proposed in this application combines vehicle motion data, lane perception data, and navigation data to transform the single-frame lane positioning problem into an optimal probability problem under continuous spatiotemporal observation. The N navigation lanes are treated as N hidden state variables. Based on a hidden Markov process, the Viterbi algorithm is used to calculate the vehicle's current optimal lane-level positioning and output the results in a probabilistic form.

[0281] Typically, lane-level positioning outputs an absolute lane number. However, in the embodiment of the present application, a probabilistic output is used to provide a refined representation of the lane information, thereby achieving high accuracy in lane change decisions.

[0282] For example, as shown in Figure 15, the single process of lane-level matching and positioning can be divided into several steps: high-confidence region initialization, SD route expansion mapping, probabilistic matching and positioning, and optimal estimation of state sequences. When outside the credible interval, lane-level positioning information can be obtained by means of perception and other means. Based on the hidden Markov process, the embodiment of the present application transforms the single-frame lane line perception and positioning problem into an optimal probability problem under continuous spatiotemporal observation. The perception, navigation, and positioning results jointly determine the state transition probability, reduce the possibility of erroneous positioning caused by false detection, missed detection, and occlusion, and obtain the optimal estimate of the specific state sequence, that is, obtain a better estimate of the vehicle's lane.

[0283] 1012. Determine whether to change lanes. If so, execute step 1013; if not, execute step 1014.

[0284] Combined with the lane traffic attributes (left turn, straight ahead, right turn, etc.) provided by the pedestrian navigation data, the virtual and real line information provided by the lane perception data, and the mid-lane-level positioning result obtained in the aforementioned step 1011, a traffic regulation check is performed to make a decision on whether the vehicle currently needs to change lanes.

[0285] 1013. Generate lane change guide lines.

[0286] When the lane change decision is to change lanes, the lane perception data and navigation data in the environmental perception data are combined, and the driving points of the vehicle along the current lane, the lane center point of the target lane, and the remote navigation points are integrated to generate guide line control points, including but not limited to using curve models such as B-spline curves and Bezier curves to obtain a smooth lane change guide line.

[0287] 1014. Generate a straight guide line.

[0288] When the lane change decision is to maintain the current lane, the vehicle's current position is projected onto the navigation route, the navigation line points in front of the vehicle are taken, and the lane perception data is weighted and fused as guide line control points, including but not limited to using curve models such as B-spline curves and Bezier curves to generate a smooth straight guide line.

[0289] 1015. Display navigation instructions.

[0290] After the navigation instructions are generated, they can be displayed. Specifically, the navigation instructions can be displayed in the AR-HUD or on the display screen.

[0291] For example, for intersection guidance, the AR-HUD display interface for turning right guidance can be as shown in Figure 16, and the AR-HUD display interface for turning left guidance can be as shown in Figure 17.

[0292] For another example, for lane change guidance, the guidance AR-HUD display interface for changing lanes to the right can be as shown in FIG18 , and the guidance AR-HUD display interface for changing lanes to the left can be as shown in FIG19 .

[0293] The method provided in this application addresses the issue of insufficient SD map accuracy to support high-precision navigation through trajectory prediction and local guideline generation. It can also provide stable intersection and lane change guidance in scenarios such as those with no lane lines or obscured lane lines. For example, for intersection guidance, it overcomes the difficulty of placing guide lines at complex intersections without lane line constraints, which can lead to off-roading. For lane change guidance, it overcomes the mismatch between lane-level perception and navigation information, which can lead to lane change errors. Furthermore, when applied to AR-HUDs, the limited viewing angle of the central control requires real-time construction and generation of guide lines tailored to the vehicle's position, rather than directly projecting the SD map.

[0294] The above describes the method flow provided by the present application. The following describes the device for executing the method flow provided by the present application.

[0295] The present application also provides an electronic device, comprising a display device, a memory, and one or more processors. The memory stores code for a graphical user interface of an application. The one or more processors are configured to execute the code for the graphical user interface (GUI) stored in the memory to display the graphical user interface on the display device. The graphical user interface includes:

[0296] Navigation guidance is displayed, wherein the navigation guidance is generated based on navigation data, map data and scene data, the map data includes data of a map of the vehicle's driving area, the navigation data includes a navigation path for the vehicle when driving on the map, and may include a path generated based on the vehicle's driving starting point and destination, and the scene data is information collected in the environment where the vehicle is located, and may specifically include environmental perception data collected by sensors in the vehicle; the navigation guidance can be used to identify a drivable lane corresponding to the navigation path in the environment, the corresponding drivable lane in the environment includes the vehicle's driving path obtained by perception based on the scene data, or a lane with divided lane lines or a lane without divided lane lines that the vehicle can actually travel, and the navigation guidance is displayed superimposed on the perceived lane.

[0297] Therefore, in the embodiment of the present application, navigation guidance can be superimposed and displayed in the lane in the GUI interface, thereby achieving more accurate lane-level guidance.

[0298] The aforementioned electronic device may specifically be a vehicle, a vehicle-mounted terminal, or other navigation device.

[0299] Optionally, the aforementioned display device may specifically include a HUD or a display screen.

[0300] In a possible embodiment, if the aforementioned display device includes a HUD, such as a conventional HUD or an AR-HUD, the projection position of the vehicle's drivable lane on the windshield can be determined based on the user's sight point and scene data, and the navigation guidance can be displayed at the projection position, thereby improving the usability of the navigation guidance. For example, as shown in Figures 16 to 19 above, the navigation guidance can be projected on the windshield of the vehicle. Referring to Figures 3 to 4 above, when projecting, the position of the lane corresponding to the navigation guidance projected on the windshield can be determined in combination with the user's sight point, that is, the position of the lane observed by the user through the windshield, and the navigation guidance is superimposed and displayed at this position. This is equivalent to the method provided in the present application, which projects more accurate navigation guidance obtained by the method corresponding to Figures 5 to 15 of the present application into the actual lane through the HUD, thereby improving the user's viewing experience of the navigation guidance and improving the user experience.

[0301] In a possible embodiment, if the aforementioned display device includes a display screen, a scene image, that is, an image of the vehicle's drivable area, can be displayed on the display screen, and navigation guidance can be superimposed on the lane in the scene image, thereby improving the usability of the navigation guidance.

[0302] In addition, the method for generating the navigation guide displayed in the GUI can refer to the corresponding descriptions of Figures 5 to 15 above, and will not be repeated here.

[0303] Referring to FIG. 20 , a schematic diagram of the structure of a navigation device provided by the present application may include:

[0304] Acquisition module 2001 is responsible for acquiring navigation data, map data, and lane perception data. Map data includes data on a map of the vehicle's driving area. Navigation data includes the navigation path of the vehicle as it travels on the map. Lane perception data is information collected from the vehicle's environment.

[0305] Processing module 2002, for generating navigation guidance based on navigation data, map data and lane perception data;

[0306] Display module 2003 is used to display navigation instructions. The navigation instructions are used to instruct the vehicle to travel according to the navigation path. The navigation instructions are used to identify the drivable lanes corresponding to the navigation path in the environment. The corresponding drivable lanes in the environment include the drivable paths of the vehicle obtained by sensing based on the lane perception data.

[0307] In one possible implementation, the aforementioned processing module 2002 is further used to: perceive and determine lane perception data based on data collected by the vehicle's sensors, such as determining lane information based on lane lines included in the lane perception data, or outputting lane perception data by using the lane perception data as input to a pre-trained lane perception model, etc. The lane perception data may specifically include information about at least one lane in the vehicle's environment, and may specifically include information such as lane lines, lane positioning, or the number of lanes of at least one lane. The at least one lane may specifically include the lane the vehicle is currently traveling in, the lane adjacent to the vehicle, or the lane in front of the vehicle, etc. Therefore, in the method provided in the embodiment of the present application, lane information in the vehicle's environment can be perceived based on the perception of the vehicle's environment.

[0308] In one possible implementation, the lane perception data may specifically include data collected by sensors in the vehicle. For example, such sensors may include, but are not limited to, image sensors, radars, infrared sensors, or depth sensors. In other words, the lane perception data may also include, but is not limited to, environmental data perceived by sensors such as image sensors, radars, infrared sensors, or depth sensors. Therefore, in the implementation of this application, sensor-perceived data can be used to understand the specific conditions of the vehicle's environment, thereby determining lane-level navigation guidance based on the perception results, without relying on HD maps.

[0309] In one possible implementation, the aforementioned navigation data may specifically include map data and a navigation route. The map data refers to data on a map of the vehicle's driving area, and the navigation route refers to the route planned by the vehicle within the corresponding map area. This navigation data may be generated based on user triggering or based on the vehicle's autonomous driving function, depending on the actual application scenario.

[0310] In one possible implementation, the aforementioned processing module 2002 is specifically used to: use map data and historical movement information of the vehicle as input to a prediction model to output a predicted path of the vehicle, the prediction model being used to output the vehicle's driving path based on the input data; and generate navigation guidance based on the predicted path, navigation data, and lane perception data.

[0311] In one possible implementation, the processing module 2002 is specifically configured to: if the navigation path in the navigation data includes a path for the vehicle to change lanes, obtain a first path and a second path of the navigation path, where the first path is the path for the vehicle to enter a target lane from a current lane. For example, when the vehicle needs to turn, the first path is the path of the vehicle in the navigation path when entering a curve, or when the vehicle needs to change lanes, the first path may be the path of the vehicle when transferring from the current lane to the target lane, where the target lane is one of the at least one lanes mentioned above; and the second path includes a path planned for the vehicle after the vehicle enters the target lane. For example, when the vehicle needs to turn, the second path is the path of the vehicle in the navigation path when exiting a curve, or when the vehicle needs to change lanes, the second path may be the path of the vehicle after transferring from the current lane to the target lane; determine a plurality of travel points based on the predicted path, the first path, and the second path, the plurality of travel points including a curve entry point and a curve exit point, where the curve entry point includes a point determined based on the first path, and the curve exit point includes a point determined based on the second path; and generate navigation guidance based on the plurality of travel points.

[0312] In a possible implementation, the aforementioned processing module 2002 is specifically configured to perform fitting on a plurality of row points to generate a steering guidance curve, where the steering guidance curve is used to indicate a lane in which the vehicle is traveling in an environment.

[0313] Optionally, when the processing module fits the steering guide curve, a curve model can be used to fit a smoother guide line to improve the user's viewing experience.

[0314] In a possible embodiment, when the vehicle turns, the processing module 2002 is specifically used to: obtain a set of broken line points, the broken line point set including the current position of the vehicle, points on the untraveled path in the predicted path, and broken lines formed between multiple row points that the vehicle has not reached; fit the row points in the broken line point set to generate a steering guide curve.

[0315] In one possible implementation, the aforementioned processing module 2002 is specifically used to: when the vehicle reaches a turning point, fit the row points in the first path to generate a steering guidance curve pointing to the turning point; when the vehicle reaches a turning point, fit the row points in the second path to generate a steering guidance curve pointing from the vehicle to the end of the predicted path.

[0316] In a possible implementation, the aforementioned prediction model specifically includes:

[0317] A map encoder is used to extract features from map data to obtain map features;

[0318] A motion encoder is used to extract features from the historical motion information of the vehicle to obtain motion features;

[0319] Fusion encoder, used to extract features from map features and motion features to obtain global features;

[0320] A reference path generator is used to generate a reference path for the vehicle based on the predicted path of the previous frame;

[0321] Path decoder, used to generate a predicted path based on global features and reference path.

[0322] In a possible implementation, the aforementioned prediction model further includes:

[0323] A visual encoder is used to extract features from the input lane perception data to obtain visual features;

[0324] The path decoder is also used to generate a predicted path based on visual features, global features and path features.

[0325] In a possible embodiment, the aforementioned reference path generator is also used to obtain a reference path by combining the predicted path of the previous frame and the motion constraint path. The motion constraint path is a path generated based on the physical motion model of the vehicle. The motion constraint path is used as a constraint for the output reference path.

[0326] In one possible implementation, the aforementioned processing module 2002 is specifically used to: determine a lane change decision of the vehicle based on navigation data and lane perception data, where the lane change decision is used to indicate whether the vehicle changes lanes; and generate navigation guidance based on the lane change decision.

[0327] In one possible implementation, the aforementioned processing module 2002 is specifically used to: determine the lane in which the vehicle is currently traveling based on map data and lane perception data; obtain the probability of the vehicle transferring from the current lane to each of N lanes based on the lane perception data and navigation data, where N is a positive integer and the N lanes are determined from the lane perception data; determine the vehicle's transfer path based on the probability of the vehicle transferring from the current lane to each of the N lanes; and obtain a lane change decision based on the transfer path.

[0328] In one possible implementation, the aforementioned processing module 2002 is specifically configured to: use the map data and lane perception data as inputs to a classification network, and divide the map into multiple regions through the classification network, wherein the multiple regions include high-confidence regions, and the high-confidence regions include regions with confidence levels higher than a preset value; expand the map based on the high-confidence regions to obtain an expanded map; and obtain the probability of the vehicle shifting from the current lane to each of N lanes based on the expanded map and the navigation path.

[0329] In one possible implementation, the aforementioned processing module 2002 is specifically configured to: generate navigation guidance based on the transfer path if the lane change decision is to change lanes; and maintain the generated guide line indicating lane keeping as a navigation guidance if the lane change decision is to maintain lanes.

[0330] In a possible embodiment, the aforementioned vehicle is further provided with a head-up display HUD or display screen, a display module, which is specifically used to: display navigation instructions through the HUD, and the display position of the navigation instructions matches the environment; or, display a scene image of the vehicle's environment on the display screen, and superimpose the navigation instructions in the lane of the scene image.

[0331] Referring to Figure 21, there is shown a hardware structure diagram of a navigation device 210 provided in an embodiment of the present application. The navigation device can be deployed in a vehicle and can be used to execute the method steps shown in Figures 3 to 19. The navigation device can also be referred to as an electronic device.

[0332] The navigation device 210 shown in FIG21 may include: a processor 2101 , a memory 2102 , a communication interface 2103 , and a bus 2104 . The processor 2101 , the memory 2102 , and the communication interface 2103 may be connected via the bus 2104 .

[0333] The processor 2101 is the control center for generating the navigation device 210 and can be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0334] As an example, the processor 2101 may include one or more CPUs, such as CPU 0 and CPU 1 shown in FIG. 21 .

[0335] The memory 2102 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0336] In one possible implementation, the memory 2102 may exist independently of the processor 2101. The memory 2102 may be connected to the processor 2101 via a bus 2104 and used to store data, instructions, or program codes. When the processor 2101 calls and executes the instructions or program codes stored in the memory 2102, it can implement the display method provided in the embodiment of the present application or extract the GUI provided in the embodiment of the present application.

[0337] In another possible implementation, the memory 2102 may also be integrated with the processor 2101 .

[0338] The communication interface 2103 is used to connect the navigation device 210 to other devices via a communication network, which may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 2103 may include a receiving unit for receiving data and a sending unit for sending data.

[0339] Bus 2104 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, FIG21 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0340] It should be noted that the structure shown in FIG. 21 does not limit the navigation device 210. In addition to the components shown in FIG. 21, the navigation device 210 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0341] Optionally, the navigation device shown in the aforementioned FIG. 21 is a chip.

[0342] The present application also provides a digital processing chip. The digital processing chip integrates circuitry and one or more interfaces for implementing the aforementioned processor 2101 or the functions of processor 2101. When the digital processing chip integrates memory, it can perform the method steps of any one or more of the aforementioned embodiments. When the digital processing chip does not integrate memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the actions performed by the navigation device in the aforementioned embodiments based on program code stored in the external memory.

[0343] An embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the steps of the method described in the embodiment shown in FIG. 3 .

[0344] The navigation device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit, and the chip may perform the method described in the embodiment shown in FIG3 above. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0345] Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processing unit or processor may be used to execute the steps of the method corresponding to the aforementioned FIG. 3 .

[0346] A vehicle is also provided in an embodiment of the present application, which includes the device shown in Figure 12 or Figure 21 above.

[0347] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0348] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0349] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0350] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0351] The terms "first," "second," "third," "fourth," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0352] Finally, it should be noted that the above is only a specific implementation method of the present application, but the protection scope of the present application is not limited to this. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the protection scope of the present application.

Claims

1. A navigation method, characterized in that: include: Acquire navigation data and lane perception data, the navigation data including a navigation path of the vehicle when traveling, and the lane perception data including information of at least one lane determined based on information collected from an environment where the vehicle is located; Navigation guidance is generated based on the navigation data and the lane perception data, and the navigation guidance is displayed, the navigation guidance is used to identify a drivable lane corresponding to the navigation path in the environment, and the drivable lane includes a path of the vehicle obtained based on the lane perception data.

2. The method according to claim 1, characterized in that The generating navigation guidance according to the navigation data and the lane perception data includes: Using map data and historical movement information of the vehicle as inputs of a prediction model, and outputting a predicted path of the vehicle, the prediction model being used to output the driving path of the vehicle based on the input data, the historical movement information including information generated by the movement of the vehicle in a historical period, and the map data including data of a map of the driving area of ​​the vehicle in the navigation data; The navigation guidance is generated according to the predicted path, the navigation path and the lane perception data.

3. The method according to claim 2, characterized in that The generating the navigation guidance according to the predicted path, the navigation path and the lane perception data includes: If the navigation path includes a path for the vehicle to change lanes, a first path and a second path are acquired according to the navigation path and the predicted path, wherein the first path is a path for the vehicle to enter a target lane from a current lane, and the second path includes a path planned for the vehicle after the vehicle drives to the target lane, and the target lane is one of the at least one lane; Determine a plurality of points based on the first path and the second path, the plurality of points comprising a curve entry point and a curve exit point, the curve entry point comprising a point determined based on the first path, and the curve exit point comprising a point determined based on the second path; The navigation guidance is generated according to the plurality of travel points.

4. The method according to claim 3, characterized in that Generating the navigation guidance according to the plurality of travel points comprises: Fitting the multiple row points to generate a steering guidance curve, where the steering guidance curve is used to indicate a lane in which the vehicle is traveling in the environment; The navigation guidance is generated according to the steering guidance curve.

5. The method according to claim 4, characterized in that When the vehicle turns, fitting the plurality of line points according to the curve model to generate a steering guide curve includes: Acquire a set of broken line points, the set of broken line points including the current position of the vehicle, points on the untraveled path in the predicted path, and line points that the vehicle has not reached among the multiple line points; Fitting is performed on the set of broken line points to generate the steering guide curve.

6. The method according to claim 4 or 5, characterized in that: The fitting of the plurality of row points to generate a steering guide curve further includes: When the vehicle reaches the curve entry point, fitting is performed based on the points in the first path to generate the turning guide line, and the turning guide line points to the curve exit point; When the vehicle travels to the exit point, fitting is performed based on the line points in the second path to generate the steering guide curve, where the steering guide curve is a curve pointing from the vehicle to the end of the predicted path.

7. The method according to any one of claims 1 to 6, characterized in that The generating the navigation guidance lane perception data according to the navigation data and the lane perception data further includes: determining a lane change decision of the vehicle according to the navigation data and the lane perception data, the lane change decision being used to indicate whether the vehicle changes lanes; The navigation guidance is generated according to the lane change decision.

8. The method according to claim 7, characterized in that The determining, according to the navigation data and the lane perception data, a lane change decision of the vehicle includes: determining the lane in which the vehicle is currently traveling according to map data and the lane perception data, wherein the map data includes data of a map of the vehicle traveling area in the navigation data; Acquire a probability of the vehicle transferring from a current lane to each of N lanes according to the lane perception data and the navigation data, wherein N is a positive integer and the N lanes are determined from the lane perception data; Determining a transfer path of the vehicle according to the probability of the vehicle transferring from the current lane to each of the N lanes; The lane change decision is obtained according to the transition path.

9. The method according to claim 8, characterized in that The obtaining the probability of the vehicle transferring from the current lane to each of the N lanes according to the lane perception data and the navigation data includes: Using the map data and the lane perception data as inputs of a classification network, dividing the map into a plurality of regions through the classification network, wherein the plurality of regions include a high confidence region, and the high confidence region includes a region with a confidence level higher than a preset value; Expanding the map according to the high confidence region to obtain an expanded map; The probability of the vehicle transferring from the current lane to each of the N lanes is obtained according to the expanded map and the navigation path.

10. The method according to claim 8 or 9, characterized in that: The generating the navigation guidance according to the lane change decision includes: If the lane change decision is lane change driving, generating the navigation guidance according to the transfer path; If the lane change decision is to maintain the lane, the generated guide line indicating lane keeping is maintained as the navigation guidance.

11. The method according to any one of claims 1 to 10, characterized in that The process of acquiring the lane perception data includes: Acquire data collected by sensors in the vehicle to obtain scene data; The lane perception data is obtained by performing perception according to the scene data.

12. The method according to claim 11, characterized in that The vehicle's sensors include one or more of the following: Image sensor, radar, infrared sensor or depth sensor.

13. The method according to any one of claims 1 to 12, characterized in that The vehicle is also provided with a head-up display HUD or a display screen, and the display of the navigation guidance includes: Displaying the navigation guide through the HUD, and the display position of the navigation guide matches the environment; Alternatively, a scene image of the environment in which the vehicle is located is displayed on the display screen, and the navigation guidance is superimposed and displayed in a lane of the scene image.

14. A navigation device, characterized in that: include: An acquisition module, having the function of acquiring navigation data and lane perception data, wherein the navigation data includes a navigation path of the vehicle when the vehicle is traveling, and the lane perception data includes information of at least one lane determined based on information collected from an environment where the vehicle is located; A processing module, configured to generate navigation guidance according to the navigation data and the lane perception data; A display module is used to display the navigation guidance, wherein the navigation guidance is used to identify a drivable lane corresponding to the navigation path in the environment, and the drivable lane includes a path of the vehicle obtained according to the lane perception data.

15. The device according to claim 14, characterized in that The processing module is specifically used for: Using map data and historical movement information of the vehicle as inputs of a prediction model, and outputting a predicted path of the vehicle, the prediction model being used to output the driving path of the vehicle based on the input data, the historical movement information including information generated by the movement of the vehicle in a historical period, and the map data including data of a map of the driving area of ​​the vehicle in the navigation data; The navigation guidance is generated according to the predicted path, the navigation path and the lane perception data.

16. The device according to claim 15, characterized in that The processing module is specifically used for: If the navigation path includes a path for the vehicle to change lanes, a first path and a second path are acquired according to the navigation path and the predicted path, wherein the first path is a path for the vehicle to enter a target lane from a current lane, and the second path includes a path planned for the vehicle after the vehicle drives to the target lane, and the target lane is one of the at least one lane; Determine a plurality of points based on the predicted path, the first path, and the second path, wherein the plurality of points include a curve entry point and a curve exit point, wherein the curve entry point includes a point determined based on the first path, and the curve exit point includes a point determined based on the second path; The navigation guidance is generated according to the plurality of travel points.

17. The device according to claim 16, characterized in that The processing module is specifically used for: Fitting the multiple row points to generate a steering guidance curve, where the steering guidance curve is used to indicate a lane in which the vehicle is traveling in the environment; The navigation guidance is generated according to the steering guidance curve.

18. The device according to claim 17, characterized in that When the vehicle turns, the processing module is specifically used to: Acquire a set of broken line points, the broken line point set including a current position of the vehicle, points on a path not traveled in the predicted path, and broken lines formed between line points not reached by the vehicle among the plurality of line points; Fitting is performed on the set of broken line points to generate the steering guide curve.

19. The device according to claim 17 or 18, characterized in that The processing module is specifically used for: When the vehicle travels to the curve entry point, the line points in the first path are fitted to generate the turning guide line, and the turning guide line points to the curve exit point; When the vehicle reaches the exit point, the points in the second path are fitted to generate the turning guide line, which is a curve pointing from the vehicle to the end of the predicted path.

20. The device according to any one of claims 14 to 19, characterized in that The processing module is specifically used for: determining a lane change decision of the vehicle according to the navigation data and the lane perception data, the lane change decision being used to indicate whether the vehicle changes lanes; The navigation guidance is generated according to the lane change decision.

21. The device according to claim 20, characterized in that The processing module is specifically used for: determining the lane in which the vehicle is currently traveling according to map data and the lane perception data, wherein the map data includes data of a map of the vehicle traveling area in the navigation data; Acquire a probability of the vehicle transferring from a current lane to each of N lanes according to the lane perception data and the navigation data, where N is a positive integer and the N lanes are determined from the lane perception data; Determining a transfer path of the vehicle according to the probability of the vehicle transferring from the current lane to each of the N lanes; The lane change decision is obtained according to the transition path.

22. The device according to claim 21, characterized in that The processing module is specifically used for: Using the map data and the lane perception data as inputs of a classification network, dividing the map into a plurality of regions through the classification network, wherein the plurality of regions include a high confidence region, and the high confidence region includes a region with a confidence level higher than a preset value; Expanding the map according to the high confidence region to obtain an expanded map; The probability of the vehicle transferring from the current lane to each of the N lanes is obtained according to the expanded map and the navigation path.

23. The device according to claim 19 or 20, characterized in that The processing module is specifically used for: If the lane change decision is lane change driving, generating the navigation guidance according to the transfer path; If the lane change decision is to maintain the lane, the generated guide line indicating lane keeping is maintained as the navigation guidance.

24. The device according to any one of claims 14 to 23, characterized in that The processing module is specifically used for: Acquire data collected by sensors in the vehicle to obtain scene data; The lane perception data is obtained by performing perception according to the scene data.

25. The device according to claim 24, characterized in that The vehicle's sensors include one or more of the following: Image sensor, radar, infrared sensor or depth sensor.

26. The device according to any one of claims 14 to 25, characterized in that The vehicle is also provided with a head-up display HUD or a display screen, and the display module is specifically used for: Displaying the navigation guide through the HUD, and the display position of the navigation guide matches the environment; Alternatively, a scene image of the environment in which the vehicle is located is displayed on the display screen, and the navigation guidance is superimposed and displayed in a lane of the scene image.

27. An electronic device, characterized in that: The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 13 is implemented.

28. A vehicle, characterized in that: The device comprises an electronic device and at least one display device, wherein the electronic device is used to implement the method according to any one of claims 1 to 13, and the at least one display screen is used to display data when triggered by the electronic device.

29. The vehicle according to claim 28, characterized in that The at least one display device includes a heads-up display HUD or a display screen.

30. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the method according to any one of claims 1 to 13.

31. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images