Navigation method and corresponding device

Abstract

A navigation method comprises: acquiring a driving image and a pose of a vehicle at a first time, and a standard map for navigation (301); converting the target element and the vector direction of the road in the standard map to a vehicle body coordinate system of the vehicle, wherein the target element comprises a lane line or a road edge in the driving image (302); and according to the vector direction of the road, the target element, and the pose of the vehicle at the first moment and the pose of the vehicle at a second moment, constructing a lane-level navigation map at the first moment, the second moment being earlier than the first moment, the lane-level navigation map being used to provide a lane-level navigation service for the user (303). A lane-level navigation service can be provided for a user through a standard SD map without a high-precision sensor and too much computing power.

Description

A navigation method and corresponding device Technical Field

[0001] The present application relates to the field of navigation technology, and in particular to a navigation method and corresponding device. Background Art

[0002] With the rapid development of urban roads in recent years, many roads have become multi-lane. Traditional road-level navigation based on standard definition (SD) maps can no longer provide a good driving experience, and there is an urgent need to upgrade to lane-level navigation.

[0003] Current lane-level navigation systems (NOP) rely on high-precision (HD) maps for high-precision positioning. This also requires vehicles to have high-precision sensors and strong computing power to provide lane-level positioning and navigation services.

[0004] Currently, HD maps are expensive, have limited coverage, and place high demands on vehicles. Therefore, providing lane-level navigation services at a low cost has become an urgent challenge.

[0005] Summary of the Invention

[0006] The present application provides a navigation method for providing lane-level navigation services through a standard map. The present application also provides corresponding devices, computer-readable storage media, and computer program products, etc.

[0007] In a first aspect, the present application provides a navigation method, comprising: obtaining a driving image and position of a vehicle at a first moment, and a standard map for navigation; converting target elements and vector directions of roads in the standard map into a vehicle body coordinate system, wherein the target elements include lane lines or curbs in the driving image; constructing a lane-level navigation map at the first moment in the vehicle body coordinate system based on the vector directions of the road, the target elements, and the position of the vehicle at the first moment and the position of the vehicle at the second moment, wherein the second moment is earlier than the first moment, and the lane-level navigation map is used to provide lane-level navigation services to users.

[0008] The navigation method of the present application can be executed by a client, which can be a terminal device or an application. For example, the application runs on a terminal device to provide navigation services to the user. The terminal device can be a vehicle-mounted terminal or a handheld terminal, and the vehicle-mounted terminal can include a head-up display (HUD) device. Of course, the terminal device can also be a vehicle or other means of transportation that requires navigation.

[0009] In this application, the first moment and the second moment are different time points during the navigation process. The second moment and the first moment can be the time points corresponding to each frame of two consecutive driving images captured by the camera, or can also be the time points corresponding to two non-consecutive driving images. The camera used to capture the driving images can be a vehicle-mounted camera or a camera on a handheld terminal.

[0010] In this application, the vehicle's posture refers to the vehicle's position and attitude, such as the vehicle's coordinates, azimuth, etc. The vehicle's posture can be obtained through the latitude and longitude data of the global navigation satellite system (GNSS) and the data measured by the inertial measurement unit (IMU). Of course, in this application, wheel speed sensor (WSS) data can also be introduced when determining the vehicle's posture, which is not limited in this application.

[0011] In this application, a standard definition (SD) map refers to a map that provides road guidance during navigation and does not provide lane-level guidance. The vector direction of a road in a standard map refers to the road being represented by points in the standard map, each with a value and a direction. A standard map may also provide the number of lanes. The vector direction of the road can be converted from a global coordinate system to a vehicle coordinate system using vehicle dead reckoning (VDR).

[0012] In this application, the target element is lane lines or curbs and other lane-related information extracted from the driving image. The target element can be transformed into the vehicle body coordinate system through inverse perspective mapping (IPM).

[0013] Converting the road's vector direction and target elements into the vehicle's body coordinate system's bird's-eye view space can be understood as a heterogeneous data fusion process. This fusion process can be performed from the vehicle's bird's-eye view (BEV) perspective. The bird's-eye view perspective can be understood as a bird's-eye view. Of course, fusion can also be performed from other perspectives, such as the vehicle's front perspective. Data from different perspectives can be converted to each other.

[0014] The solution provided in this application can build a lane-level navigation map based on the SD map during the navigation process. It does not require high-precision (HD) maps, nor does it require high-precision sensors and too much computing power to obtain lane-level navigation services, thus providing users with high-quality navigation services at a lower cost.

[0015] In one possible implementation, the above steps of: constructing a lane-level navigation map at the first moment based on the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at the second moment, include: constructing the lane-level navigation map at the first moment based on the lane-level navigation map at the second moment and the frame estimation result at the first moment; wherein the frame estimation result at the first moment is used to indicate the confidence of the driving image at the first moment.

[0016] In this possible implementation, the lane-level navigation map of the current frame may be determined by using historical lane-level navigation maps and the frame estimation result of the current frame, thereby improving the generation efficiency of the lane-level navigation map.

[0017] In one possible implementation, the frame estimation result at the first moment is obtained through a target relationship formula, which includes the product of the first weight and the vector direction of the road, the product of the second weight and the inter-frame motion information, and the product of the third weight and the parameter value of the target element.

[0018] The target relationship can be expressed as:

[0019]

[0020] Among them, L(p t ) represents the frame estimation result at the first moment, W navi Represents the first weight, V t Indicates the vector direction of the road, W vdr represents the second weight, B represents inter-frame motion information, pos t Indicates the position of the vehicle at the first moment, pos t-1 represents the vehicle’s position at the second moment, L(p t-1 ) represents the frame estimation result at the second moment, W de represents the third weight, Represents the results of IPM of target elements i to j in the image frame at the first moment.

[0021] In this possible implementation, the frame estimation result at the first moment can be obtained quickly and accurately through the above target relationship, thereby improving the generation speed of the lane-level navigation map.

[0022] In one possible implementation, the inter-frame motion information is a result obtained by projecting a frame estimation result at the second moment onto the first moment based on a change between the vehicle's position at the first moment and the vehicle's position at the second moment.

[0023] In the present application, the frame estimation result at the second moment is used to indicate the confidence level of the driving image at the second moment.

[0024] In this possible implementation, the frame estimation result at the first moment can be quickly obtained by using the frame estimation result at the historical moment for projection.

[0025] In one possible implementation, before constructing the lane-level navigation map at the first moment, the method also includes: updating at least one of the first weight, the second weight or the third weight based on the result of the error estimation, the result of the error estimation is the difference estimation result between the target element in the driving image at the first moment and the target element in the driving image at the second moment, or the result of the error estimation is the difference estimation result between the frame estimation result at the first moment and the frame estimation result at the second moment.

[0026] In this possible implementation, before generating the lane-level navigation map at the first moment, the corresponding weights may be adjusted based on the error estimation result, thereby improving the accuracy of the lane-level navigation map at the first moment.

[0027] In a possible implementation, if the error estimation result indicates that the similarity between the target element in the driving image at the first moment and the target element in the driving image at the second moment is less than a first threshold, the third weight is adjusted down.

[0028] In this possible implementation, by comparing the target elements in the driving image at the first moment with the target elements in the driving image at the second moment, the similarity of the target elements at the two moments can be determined. If the similarity is greater than a first threshold, it indicates that the target elements in the driving images at the two moments are highly similar, because the two moments are generally close together. This high similarity indicates that the perception module that extracts the target elements from the driving images is functioning properly. If the similarity is less than the first threshold, it indicates that the target elements in the driving images at the two moments are less similar, possibly due to false detection or missed detection by the perception module. In this case, the credibility of the target elements in the driving image at the first moment is low, and the corresponding third weight needs to be reduced, so that the frame estimation result at the second moment is retained more.

[0029] In a possible implementation, if the error estimation result indicates that the similarity between the frame estimation result at the first moment and the frame estimation result at the second moment is less than a second threshold, the third weight is increased.

[0030] In this possible implementation, if the similarity between the frame estimation results at two moments is small, it means that the lane may have changed at the first moment relative to the second moment. In this case, as much information as possible from the driving image at the first moment should be used, so the third weight needs to be increased to improve the accuracy of the lane-level navigation map.

[0031] In one possible implementation, the method further includes: correcting the position parameters of lane lines of other lanes with the same orientation in the lane-level navigation map at the first moment based on the frame estimation result at the first moment and the orientation parameters and width of the lane where the vehicle is located.

[0032] In this possible implementation, lane orientation parameters may include an angle, a lane offset, and lane curvature. The angle can be the angle between the vehicle's front and lane orientation, or the angle between the lane and an axis of the vehicle's coordinate system. The lane offset is the offset between the lane and the origin of the vehicle coordinate system. The frame estimation result at the first moment is used to determine whether to correct the lane position parameters. Because the frame estimation result represents the confidence level of the prediction based on the target element, the vehicle's position, and the road vector direction in the SD map, if the confidence level is low and unusable, lane correction is not necessary. If the frame estimation result at the first moment is reasonable, the vehicle's lane is typically the clearest lane in the driving image. Therefore, the lane position parameters of other lanes in the navigation map with the same orientation can be corrected based on the lane's orientation and width. This facilitates more accurate navigation guidance.

[0033] In one possible implementation, the method further includes: performing semantic merging based on the lane in which the vehicle is located in the lane-level navigation map at the first moment and the navigation information input by the user in the standard map to output lane-level navigation guidance prompts, the lane-level navigation guidance prompts including maintaining the lane, changing lanes to the left, or changing lanes to the right.

[0034] In this possible implementation, the navigation information may include the navigation destination and the number of lanes on the current road. Semantic merging refers to directly giving clear navigation instructions without the user having to make another conversion. For example: entering the two left lanes in the SD map navigation scene is merged into keeping the lane or changing lanes to the left, and entering the right lane in the SD map navigation scene is merged into keeping the lane or changing lanes to the right. In this way, the user can perform the corresponding operations according to the lane-level navigation guidance prompts, and the user does not need to observe the current lane and then decide whether to change lanes. It can be seen that the solution provided by the present application can improve navigation efficiency.

[0035] In one possible implementation, if it is determined based on the lane in which the vehicle is located and the navigation information that the vehicle needs to keep going straight, is already in the leftmost lane in a scenario where a U-turn or left turn is required, or is already in the rightmost lane in a scenario where a right turn is required, the lane-level navigation guidance prompt is to maintain the lane.

[0036] In one possible implementation, if it is determined based on the lane in which the vehicle is located and navigation information that the vehicle needs to change lanes, the lane-level navigation guidance prompt is to change lanes to the left or to the right.

[0037] In one possible implementation, the method further includes: when the lane in which the vehicle is located cannot be determined through the driving image at the first moment, determining the lane in which the vehicle is located at the first moment based on the lane in which the vehicle is located in the driving image at the third moment and the driving behavior from the third moment to the first moment; wherein the third moment is before the first moment, and the distance between the third moment and the first moment is less than a third threshold, and the frame estimation result at the third moment is the most credible estimation result in the driving image generated from the third moment to the first moment.

[0038] In this possible implementation, when locating the vehicle's current lane during navigation guidance, if the vehicle's current lane cannot be determined from the driving image at the first moment, such as in intersection or congested scenarios due to occlusion by vehicles on both sides or front and rear measurements, the vehicle's current lane can be determined by querying historical driving images (which can determine the vehicle's lane). The vehicle's driving behavior between the third moment and the first moment of the historical driving image can then be combined to determine the vehicle's lane at the first moment. This can meet lane-level navigation guidance requirements in complex scenarios.

[0039] In one possible implementation, the method further includes: during the navigation process, adjusting the position of a virtual arrow used to indicate the navigation direction according to multi-dimensional parameters, where the multi-dimensional parameters include at least two of the heading angle, roll angle, horizontal offset of the vehicle system, or vertical offset of the vehicle system.

[0040] In this possible implementation, the position of the virtual arrow used to indicate the navigation direction may have multiple dimensions, so that the virtual arrow is more flexible during navigation.

[0041] In one possible implementation, during navigation, the direction of the virtual arrow is consistent with the direction of the lane.

[0042] In this possible implementation, the direction of the virtual arrow is consistent with the direction of the lane, which better meets the user's driving needs.

[0043] The second aspect of the present application provides a navigation method, including: responding to a user clicking a first button on a standard map used for navigation, starting a camera, the camera being used to capture driving images, the first button being an entrance for starting a lane-level navigation service; converting target elements and vector directions of roads in the standard map into a vehicle body coordinate system, the target elements including lane lines or curbs in the driving image at a first moment; constructing a lane-level navigation map at a first moment based on the vector direction of the road, the target elements, and the vehicle's position at the first moment and the vehicle's position at a second moment, the second moment being earlier than the first moment; and displaying the lane-level navigation map, the lane-level navigation map being used to provide lane-level navigation services to users.

[0044] In this second aspect, in the navigation interface of a standard map, clicking the entry for starting lane-level navigation service can initiate the generation of a lane-level navigation map, which is then used for navigation. Lane-level navigation service does not require HD maps, high-precision sensors, or excessive computing power, thus providing users with high-quality navigation services at a lower cost.

[0045] In a possible implementation, the method further includes:

[0046] In response to the user's switching operation of the perspective button on the lane-level navigation map, the lane-level navigation map switches from a first perspective to a second perspective. The first perspective includes a bird's-eye view of the vehicle or a front perspective of the vehicle, and the second perspective includes a front perspective of the vehicle or a bird's-eye view of the vehicle. The first perspective is different from the second perspective.

[0047] In this possible implementation, the user can switch between the top-down perspective and the front perspective. Of course, other perspectives can also be included. The user can switch between different perspectives according to needs, such as: left perspective, right perspective, rear perspective, etc., thereby displaying lane-level navigation maps from different perspectives.

[0048] In one possible implementation, the method also includes: responding to a user's switching operation of the navigation guidance mode on the lane-level navigation map, switching from a first navigation guidance mode to a second navigation guidance mode, the first navigation guidance mode including an enhanced lane rendering mode for the driving lane or a mode for rendering guidance arrows on the driving lane, and the first navigation guidance mode is different from the second navigation guidance mode.

[0049] In this possible implementation, users can choose the corresponding guidance mode according to their preferences, thereby enhancing user stickiness.

[0050] In one possible implementation, the method further includes: responding to a user's control operation of an information button on a lane-level navigation map, closing a navigation guidance prompt on the lane-level navigation map, or displaying a navigation guidance prompt on the lane-level navigation map, the navigation guidance prompt including maintaining lanes, changing lanes left, changing lanes right, intersection indications, or warning indications.

[0051] In this possible implementation, the user can choose to turn on or off the navigation guidance prompt according to his or her preferences, thereby meeting the user's personalized needs.

[0052] A third aspect of the present application provides a navigation device, comprising:

[0053] The acquisition unit is used to acquire the driving image and posture of the vehicle at the first moment, as well as a standard map for navigation.

[0054] The first processing unit is configured to convert target elements and vector directions of roads in a standard map into a vehicle body coordinate system, wherein the target elements include lane lines or curbs in a driving image.

[0055] The second processing unit is used to construct a lane-level navigation map at a first moment based on the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at a second moment, where the second moment is earlier than the first moment. The lane-level navigation map is used to provide lane-level navigation services to users.

[0056] In one possible implementation, the second processing unit is specifically used to construct a lane-level navigation map at the first moment based on the lane-level navigation map at the second moment and the frame estimation result at the first moment; wherein the frame estimation result at the first moment is used to indicate the confidence of the driving image at the first moment.

[0057] In one possible implementation, the frame estimation result at the first moment is obtained through a target relationship formula, which includes the product of the first weight and the vector direction of the road, the product of the second weight and the inter-frame motion information, and the product of the third weight and the parameter value of the target element.

[0058] In one possible implementation, the inter-frame motion information is a result obtained by projecting a frame estimation result at the second moment onto the first moment based on a change between the vehicle's position at the first moment and the vehicle's position at the second moment.

[0059] In one possible implementation, the second processing unit is further used to update at least one of the first weight, the second weight or the third weight according to the result of error estimation before constructing the lane-level navigation map at the first moment, where the result of error estimation is the difference estimation result between the target element in the driving image at the first moment and the target element in the driving image at the second moment, or the result of error estimation is the difference estimation result between the frame estimation result at the first moment and the frame estimation result at the second moment.

[0060] In a possible implementation, if the error estimation result indicates that the similarity between the target element in the driving image at the first moment and the target element in the driving image at the second moment is less than a first threshold, the third weight is adjusted down.

[0061] In a possible implementation, if the error estimation result indicates that the similarity between the frame estimation result at the first moment and the frame estimation result at the second moment is less than a second threshold, the third weight is increased.

[0062] In one possible implementation, the second processing unit is further used to correct the position parameters of lane lines of other lanes with the same direction in the lane-level navigation map at the first moment based on the frame estimation result at the first moment and the orientation parameters and width of the lane where the vehicle is located.

[0063] In one possible implementation, the second processing unit is further used to perform semantic merging based on the lane in which the vehicle is located in the lane-level navigation map at the first moment and the navigation information input by the user in the standard map to output lane-level navigation guidance prompts, where the lane-level navigation guidance prompts include maintaining the lane, changing lanes to the left, or changing lanes to the right.

[0064] In one possible implementation, if it is determined based on the lane in which the vehicle is located and the navigation information that the vehicle needs to keep going straight, is already in the leftmost lane in a scenario where a U-turn or left turn is required, or is already in the rightmost lane in a scenario where a right turn is required, the lane-level navigation guidance prompt is to maintain the lane.

[0065] In one possible implementation, if it is determined based on the lane in which the vehicle is located and navigation information that the vehicle needs to change lanes, the lane-level navigation guidance prompt is to change lanes to the left or to the right.

[0066] In one possible implementation, the second processing unit is further used to determine the lane in which the vehicle is located at the first moment based on the lane in which the vehicle is located in the driving image at the third moment and the driving behavior from the third moment to the first moment, when the lane in which the vehicle is located cannot be determined by the driving image at the first moment; wherein the third moment is before the first moment, and the distance between the third moment and the first moment is less than a third threshold, and the frame estimation result at the third moment is the most credible estimation result in the driving image generated from the third moment to the first moment.

[0067] In one possible implementation, the second processing unit is further used to adjust the position of the virtual arrow used to indicate the navigation direction according to multi-dimensional parameters during the navigation process, where the multi-dimensional parameters include at least two of the heading angle, roll angle, horizontal offset of the vehicle system, or vertical offset of the vehicle system.

[0068] In one possible implementation, during navigation, the direction of the virtual arrow is consistent with the direction of the lane.

[0069] A fourth aspect of the present application provides a navigation device, comprising:

[0070] The first processing unit is used to respond to a user clicking a first button on a standard map used for navigation, and start a camera for capturing driving images. The first button is an entry for starting a lane-level navigation service.

[0071] The second processing unit is used to convert the target elements and the vector direction of the road in the standard map into the vehicle body coordinate system, where the target elements include lane lines or curbs in the driving image at the first moment.

[0072] The third processing unit is used to construct a lane-level navigation map at a first moment according to the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at a second moment, where the second moment is earlier than the first moment.

[0073] The display unit is used to display a lane-level navigation map, and the lane-level navigation map is used to provide lane-level navigation services to users.

[0074] In one possible implementation, the first processing unit is also used to respond to the user's switching operation of the perspective button on the lane-level navigation map, switching from the lane-level navigation map of the first perspective to the lane-level navigation map of the second perspective, the first perspective includes the vehicle's top-down perspective or the vehicle's front perspective, the second perspective includes the vehicle's front perspective or the vehicle's top-down perspective, and the first perspective is different from the second perspective.

[0075] In one possible implementation, the first processing unit is also used to respond to the user's switching operation of the navigation guidance mode on the lane-level navigation map, switching from the first navigation guidance mode to the second navigation guidance mode. The first navigation guidance mode includes an enhanced lane rendering mode for the driving lane or a mode for rendering guidance arrows on the driving lane. The first navigation guidance mode is different from the second navigation guidance mode.

[0076] In one possible implementation, the first processing unit is also used to respond to the user's control operation of the information button on the lane-level navigation map, turn off the navigation guidance prompts on the lane-level navigation map, or display the navigation guidance prompts on the lane-level navigation map, and the navigation guidance prompts include keeping lane, changing lanes to the left, changing lanes to the right, intersection indications or warning indications.

[0077] In a fifth aspect of the present application, a client is provided for executing the method described in the first aspect or any possible implementation of the first aspect. Specifically, the client includes modules or units for executing the method described in the first aspect or any possible implementation of the first aspect, such as an acquisition unit, a first processing unit, and a second processing unit.

[0078] In a sixth aspect of the present application, a client is provided for executing the method described in the second aspect or any possible implementation of the second aspect. Specifically, the client includes modules or units for executing the method described in the second aspect or any possible implementation of the second aspect, such as a first processing unit, a second processing unit, a third processing unit, and a display unit.

[0079] In the seventh aspect of the present application, a client is provided, comprising a transceiver, a processor and a memory, wherein the transceiver and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the program or instructions are executed by the processor, the client executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0080] In an eighth aspect, the present application provides a client comprising a transceiver, a processor and a memory, wherein the transceiver and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the program or instructions are executed by the processor, the client executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

[0081] In the ninth aspect of the present application, a chip system is provided, which includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the client's memory and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the client executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0082] In the tenth aspect of the present application, a chip system is provided, which includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the client's memory and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the client executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

[0083] In the eleventh aspect of the present application, a computer-readable storage medium is provided, on which a computer program or instruction is stored. When the computer program or instruction is executed on a computer device, the computer device executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0084] The twelfth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is run on a computer device, the computer device executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

[0085] The thirteenth aspect of the present application provides a computer device program product, which includes a computer device program code. When the computer device program code is executed on a computer device, the computer device executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

[0086] In a fourteenth aspect, the present application provides a computer device program product, which includes a computer device program code. When the computer device program code is executed on a computer device, the computer device executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

[0087] The fifteenth aspect of the present application provides a vehicle, including: a camera, a sensor and an on-board terminal, the on-board terminal being communicatively connected to the camera and the sensor; the camera is used to capture driving images; the sensor is used to collect data related to the vehicle's posture; the on-board terminal includes the client described in the third or fourth aspect above, or includes the chip system described in the fifth or sixth aspect above.

[0088] Among them, the technical effects brought about by the third to fifteenth aspects or any possible implementation methods thereof can refer to the technical effects brought about by the first aspect or different possible implementation methods of the first aspect, or can refer to the technical effects brought about by the second aspect or different possible implementation methods of the second aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] FIG1 is a schematic structural diagram of a vehicle provided in an embodiment of the present application;

[0090] FIG2 is a schematic structural diagram of another vehicle provided in an embodiment of the present application;

[0091] FIG3 is a schematic diagram of an embodiment of a navigation method provided in an embodiment of the present application;

[0092] FIG4A is a standard map in a global coordinate system provided by an embodiment of the present application;

[0093] FIG4B and FIG4C are schematic diagrams of an example of a driving image provided by an embodiment of the present application;

[0094] FIG4D is a map in a vehicle coordinate system provided in an embodiment of the present application;

[0095] FIG4E is a schematic diagram of an example of a lane-level navigation map provided in an embodiment of the present application;

[0096] FIG4F is another exemplary schematic diagram of a lane-level navigation map provided by an embodiment of the present application;

[0097] FIG4G is another exemplary schematic diagram of a lane-level navigation map provided by an embodiment of the present application;

[0098] FIG5A is a schematic diagram illustrating an example of lane orientation parameters provided by an embodiment of the present application;

[0099] FIG5B is a schematic diagram of an example of lane line correction provided by an embodiment of the present application;

[0100] 6A to 6D are several exemplary schematic diagrams of navigation guidance provided by embodiments of the present application;

[0101] FIG7 is a schematic diagram of an example of lane positioning provided by an embodiment of the present application;

[0102] FIG8A is a schematic diagram illustrating an example of lane-level navigation with virtual arrows provided in an embodiment of the present application;

[0103] FIG8B is a schematic diagram of output curves of multiple dimensions of a virtual arrow provided in an embodiment of the present application;

[0104] FIG9A is a schematic diagram of another embodiment of a navigation method provided in an embodiment of the present application;

[0105] FIG9B is a schematic diagram of another embodiment of the navigation method provided in an embodiment of the present application;

[0106] FIG10A is a schematic diagram of another embodiment of a navigation method provided in an embodiment of the present application;

[0107] FIG10B is a schematic diagram of the interface of a standard map provided in an embodiment of the present application;

[0108] FIG11 is a schematic structural diagram of a navigation device provided in an embodiment of the present application;

[0109] FIG12 is another structural diagram of a navigation device provided in an embodiment of the present application;

[0110] FIG13 is a schematic structural diagram of a vehicle provided in an embodiment of the present application;

[0111] FIG14 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0112] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0113] The terms "first," "second," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0114] The present application provides a navigation method for providing lane-level navigation using a standard map, and also provides a corresponding apparatus, a computer-readable storage medium, and a computer program product, etc. These are described in detail below.

[0115] The navigation method provided by the present application can be executed by a client. The client can be a terminal device or an application, for example, the application runs on the terminal device to provide navigation services to the user. The terminal device can be a vehicle-mounted terminal or a handheld terminal, and the vehicle-mounted terminal can include a head-up display (HUD) device. Of course, the terminal device can also be a vehicle or other means of transportation that requires navigation.

[0116] To facilitate understanding of the present solution, the structure of the vehicle provided in the present application is first introduced in conjunction with Figure 1 in the embodiment of the present application. Please refer to Figure 1 first. Figure 1 is a structural schematic diagram of the vehicle provided in the embodiment of the present application. Figure 1 is a functional block diagram of the vehicle 100 provided in the embodiment of the present application. The vehicle 100 can be configured in an automatic driving mode or a human-vehicle co-navigation mode. For example: the vehicle 100 can obtain environmental information about its surroundings through the perception system 120, and obtain a navigation strategy based on the analysis of the surrounding environmental information to achieve fully automatic driving, or present a lane-level map for navigation to the user to achieve human-vehicle co-navigation.

[0117] Vehicle 100 may include various subsystems, such as an infotainment system 110, a perception system 120, a decision control system 130, a drive system 140, and a computing platform 150. Alternatively, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0118] In some embodiments, the infotainment system 110 may include a communication system 111 , an entertainment system 112 , and a navigation system 113 .

[0119] The communication system 111 may include a wireless communication system 111 that can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 111 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 111 may communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system 146 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee. The wireless communication system 111 may include one or more dedicated short range communications (DSRC) devices that may include public and / or private data communications between vehicles and / or roadside stations.

[0120] The entertainment system 112 may include a central control screen, a microphone, and speakers. Users can use the entertainment system 112 to listen to the radio and play music in the vehicle. Alternatively, they can connect their mobile phone to the vehicle and project their phone's screen onto the central control screen, which may be touch-sensitive and user-operated. In some cases, the microphone can capture the user's voice signal and, based on analysis of the voice signal, enable the user to control certain aspects of the vehicle 100, such as adjusting the vehicle's temperature. In other cases, the speakers can play music to the user.

[0121] Navigation system 113 may include map services provided by a map provider, thereby providing navigation for vehicle 100. Navigation system 113 may be used in conjunction with the vehicle's global positioning system 121 and inertial measurement unit 122. The map services provided by the map provider may be standard definition (SD) maps. The solution provided in this application can generate a local lane-level navigation map based on the SD map and data measured by the perception system 120 during navigation, providing lane-level navigation services to users.

[0122] The perception system 120 may include several types of sensors that sense information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a global positioning system 121 (the global positioning system may be a GPS system, or a BeiDou system, or other positioning systems), an inertial measurement unit (IMU) 122, a lidar 123, a millimeter-wave radar 124, an ultrasonic radar 125, and a camera 126. The perception system 120 may also include sensors of the internal systems of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the vehicle 100.

[0123] The global positioning system 121 may be used to determine the geographic location of the vehicle 100 .

[0124] The inertial measurement unit 122 is used to sense changes in position and orientation of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 may be a combination of an accelerometer and a gyroscope.

[0125] LiDAR 123 may utilize laser light to sense objects in the environment in which vehicle 100 is located. In some embodiments, LiDAR 123 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0126] The millimeter wave radar 124 can use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the millimeter wave radar 124 can also be used to sense the speed and / or heading of the objects.

[0127] The ultrasonic radar 125 may sense objects around the vehicle 100 using ultrasonic signals.

[0128] The camera device 126 may be used to capture image information of the surrounding environment of the vehicle 100. The camera device 126 may include a monocular camera, a binocular camera, a structured light camera, a panoramic camera, etc. The image information acquired by the camera device 126 may include static image information or video stream information.

[0129] The decision control system 130 includes a computing system 131 that analyzes and makes decisions based on the information obtained by the perception system 120. The decision control system 130 also includes a vehicle controller 132 that controls the power system of the vehicle 100, as well as a steering system 133, throttle 134 and braking system 135 for controlling the vehicle 100.

[0130] The computing system 131 can process and analyze the various information obtained by the perception system 120 in order to identify targets, objects and / or features in the environment surrounding the vehicle 100. The targets may include pedestrians or animals, and the objects and / or features may include traffic signals, road boundaries and obstacles. The computing system 131 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking and other technologies. In some embodiments, the computing system 131 can be used to map the environment, track objects, estimate the speed of objects, and so on. The computing system 131 can analyze the various information obtained and derive a control strategy for the vehicle.

[0131] The vehicle controller 132 can be used to coordinate and control the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100.

[0132] The steering system 133 may be used to adjust the forward direction of the vehicle 100. For example, in one embodiment, the steering system 133 may be a steering wheel system.

[0133] The throttle 134 is used to control the operating speed of the engine 141 and thus the speed of the vehicle 100 .

[0134] Braking system 135 is used to control the deceleration of vehicle 100. Braking system 135 can use friction to slow the rotation of wheels 144. In some embodiments, braking system 135 can convert the kinetic energy of wheels 144 into electric current. Braking system 135 can also take other forms to slow the rotation of wheels 144 and thus control the speed of vehicle 100.

[0135] Drive system 140 includes components that provide power to vehicle 100. In one embodiment, drive system 140 may include engine 141, energy source 142, transmission system 143, and wheels 144. Engine 141 may be an internal combustion engine, an electric motor, an air compression engine, or another combination of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. Engine 141 converts energy source 142 into mechanical energy.

[0136] Examples of energy source 142 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 142 may also provide energy to other systems of vehicle 100.

[0137] The transmission system 143 can transmit mechanical power from the engine 141 to the wheels 144. The transmission system 143 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission system 143 may also include other components, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 144.

[0138] Some or all functions of the vehicle 100 are controlled by a computing platform 150. The computing platform 150 may include at least one processor 151 that can execute instructions 153 stored in a non-transitory computer-readable medium such as a memory 152. In some embodiments, the computing platform 150 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0139] The processor 151 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor 151 may include a graphics processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application-specific integrated circuit (ASIC), or a combination thereof. The processor 151 may be located on a device remote from the vehicle and communicate wirelessly with the vehicle.

[0140] In some embodiments, memory 152 may include instructions 153 (e.g., program logic) that are executable by processor 151 to perform various functions of vehicle 100. Memory 152 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of infotainment system 110, perception system 120, decision control system 130, and drive system 140.

[0141] In addition to instructions 153, memory 152 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other similar vehicle data, as well as other information. This information may be used by vehicle 100 and computing platform 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0142] The computing platform 150 may control functions of the vehicle 100 based on input received from various subsystems, such as the drive system 140, the perception system 120, and the decision control system 130. For example, the computing platform 150 may utilize input from the decision control system 130 to control the steering system 133 to avoid an obstacle detected by the perception system 120. In some embodiments, the computing platform 150 may be operable to provide control over many aspects of the vehicle 100 and its subsystems.

[0143] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 100. For example, the memory 152 may be partially or completely separate from the vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0144] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 1 should not be understood as a limitation to the embodiments of the present application.

[0145] Optionally, when in autonomous driving mode, the vehicle 100 or a sensing and computing device associated with the vehicle 100 (e.g., computing system 131, computing platform 150) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so the behavior of all identified objects can be considered together to predict the behavior of a single identified object. The vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 on the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0146] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0147] The vehicle 100 may be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawn mower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and the embodiments of the present application do not impose any particular limitation.

[0148] Figure 2 shows a schematic diagram of the system architecture of a vehicle 100 based on some embodiments. The vehicle 100 includes multiple vehicle integration units (VIUs) 11, a telematic box (T-BOX) 12, a cockpit domain controller (CDC), a mobile data center (MDC) 14, and a vehicle domain controller (VDC) 15.

[0149] Vehicle 100 also includes various types of sensors installed on the vehicle body, including: a laser radar 21, a millimeter-wave radar 22, an ultrasonic radar 23, and a camera 24. Multiple sensors of each type may be provided. It should be understood that while FIG2 illustrates the layout of various sensors on vehicle 100, the number and layout of sensors shown in FIG2 are merely illustrative. Those skilled in the art will appreciate the appropriate selection of sensor types, numbers, and layouts as needed.

[0150] FIG2 shows four VIUs. It should be understood that the number and positions of the VIUs in FIG2 are only an example, and those skilled in the art may select appropriate number and positions of the VIUs according to actual needs.

[0151] The vehicle integrated unit VIU 11 provides a plurality of vehicle components with some or all of the data processing functions or control functions required by the vehicle components. The VIU may have one or more of the following functions.

[0152] 1. Electronic control functions: The VIU is used to implement the electronic control functions provided by the electronic control unit (ECU) within some or all vehicle components. For example, the control function required by a vehicle component or the data processing function required by a vehicle component.

[0153] 2. The same functions as the gateway, that is, the VIU can also have some or all of the same functions as the gateway, such as protocol conversion function, protocol encapsulation and forwarding function, and data format conversion function.

[0154] 3. The function of processing data across vehicle components, that is, processing and calculating data obtained from actuators of multiple vehicle components.

[0155] It should be noted that the data involved in the above functions may include operating data of actuators in vehicle components, such as actuator motion parameters, actuator operating status, etc. The data involved in the above functions may also be data collected by data acquisition units (e.g., sensors) of vehicle components, such as road information or weather information collected by vehicle sensors, which is not specifically limited in the present embodiment.

[0156] In the example of vehicle 100 in FIG2 , vehicle 100 may be divided into multiple domains, each of which has an independent domain controller. Specifically, FIG2 shows two domain controllers: a cockpit domain controller CDC 13 and a vehicle domain controller VDC 15 .

[0157] The cockpit domain controller CDC 13 can be used to implement functional control of the cockpit area of ​​the vehicle 100. Vehicle components in the cockpit area may include a head-up display (HUD), an instrument panel, a radio, a central control screen, a navigation system, a camera, etc.

[0158] The vehicle domain controller VDC 15 can be used to coordinate and control the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100. In some embodiments, the vehicle domain controller 132 in Figure 1 can implement various functions of the VDC.

[0159] FIG2 also shows the Internet of Vehicles device T-BOX 12 and the mobile data center MDC 13. T-BOX 12 can be used to achieve communication between the vehicle 100 and internal and external devices. T-BOX can obtain data from in-vehicle devices through the bus of the vehicle 100, and can also communicate with the user's mobile phone through a wireless network. In some embodiments, T-BOX 12 can be included in the communication system 111 of FIG1. ​​The mobile data center MDC 13 is used to output drive, transmission, steering, braking and other execution control instructions based on core control algorithms such as environmental perception positioning, intelligent planning and decision-making, and vehicle motion control to achieve automatic control of the vehicle 100. It can also realize human-computer interaction of vehicle driving information through a human-computer interaction interface. In some embodiments, the computing platform 150 in FIG1 can implement various functions of MDC 13.

[0160] In Figure 2, the four VIUs 11 form a ring topology network. Each VIU 11 is connected to sensors in its immediate vicinity. T-BOX 12, CDC 13, MDC 14, and VDC 15 are also connected to the VIU's ring topology network. The VIU 11 can obtain information from each sensor and report it to CDC 13, MDC 14, and VDC 15. The ring topology also enables communication between T-BOX 12, CDC 13, MDC 14, and VDC 15.

[0161] It should be understood that the above-mentioned ring topology connection is only an illustration, and those skilled in the art may select other appropriate VIU connection methods according to their needs.

[0162] The connection between VIUs can use, for example, Ethernet; the connection between VIU and T-BOX 12, CDC 13, MDC 14 and VDC 15 can use, for example, Ethernet or peripheral component interconnect express (PCIe) bus; the connection between VIU and sensors can use, for example, controller area network (CAN), local interconnect network (LIN), FlexRay, multimedia data transmission (MOST) and the like.

[0163] Vehicle AUTOSAR can be divided into AP AUTOSAR and CP AUTOSAR, with CP also known as Classic AUTOSAR. CP has been widely used in traditional automotive embedded ECUs, meeting the needs of scenarios with high functional safety and real-time requirements but low computing power requirements, such as engine and brake controllers. AP, on the other hand, is a new architecture designed to meet the high performance and high computing power requirements of next-generation intelligent connected vehicles, as well as the continuous connectivity and integration needs of complex scenarios such as autonomous driving and cloud computing. It will be widely used in autonomous vehicles in the future.

[0164] Of course, the above description uses the structure of a vehicle as an example. In practice, the device used to execute the navigation method provided in this application can be any device capable of performing navigation functions, such as an onboard terminal in a vehicle or a user's handheld terminal, such as a mobile phone or tablet computer. Other possible product forms will not be described one by one in the embodiments of this application.

[0165] The following describes the navigation method provided in the embodiment of the present application using the client as the execution entity.

[0166] As shown in FIG3 , an embodiment of the navigation method provided by the embodiment of the present application includes:

[0167] 301. Obtain the driving image and position of the vehicle at the first moment, as well as a standard map for navigation.

[0168] In the embodiments of the present application, the vehicle's posture refers to the vehicle's position and attitude, such as the vehicle's coordinates, azimuth, etc. The vehicle's posture can be obtained using latitude and longitude data from a global navigation satellite system (GNSS) and data measured by an inertial measurement unit (IMU). Of course, wheel speed sensor (WSS) data may also be used to determine the vehicle's posture, but this is not limited in the embodiments of the present application.

[0169] In the embodiments of this application, a standard definition (SD) map refers to a map that provides road guidance during navigation and does not provide lane-level guidance. The vector direction of a road in a standard definition map refers to the road being represented by points in the standard definition map, with each point having a value and a direction. A standard definition map may also provide the number of lanes.

[0170] 302. Convert target elements and vector directions of roads in a standard map into a vehicle body coordinate system, wherein the target elements include lane lines or curbs in a driving image.

[0171] In the embodiment of the present application, the vector direction of the road can be converted from the global coordinate system to the vehicle coordinate system through vehicle dead reckoning (VDR).

[0172] In the embodiment of the present application, the target element is lane lines or curbs and other lane-related information extracted from the driving image, and the target element can be transformed into the vehicle body coordinate system through inverse perspective mapping (IPM).

[0173] The above process of converting the vector direction of the road and the target element from the global coordinate system to the vehicle coordinate system can be understood by referring to FIG. 4A to FIG. 4D .

[0174] FIG4A is a simplified schematic diagram of the SD map in the global coordinate system, FIG4B and FIG4C are driving images, and FIG4D is a data schematic diagram in the vehicle body coordinate system.

[0175] As shown in FIG4A , the vector direction of the road in the SD map in the global coordinate system is indicated by 401 . As the curvature of the road changes, the vector direction of the road also changes.

[0176] FIG4B is a driving image captured by a camera. By extracting lane lines and curbs from the driving image, a schematic diagram shown in FIG4C can be obtained.

[0177] Figure 4D is a schematic diagram of converting the data in Figures 4A and 4C into a vehicle coordinate system. As shown in Figure 4D, the vehicle coordinate system is a right-handed coordinate system with the position of vehicle 402 as the origin. The road vector direction in Figure 4A and the lane line data in Figure 4C have both been converted to Figure 4D.

[0178] In the embodiments of the present application, converting the road vector direction and target elements into the vehicle coordinate system of the vehicle's position can be understood as a heterogeneous data fusion process. This fusion process can be performed from the vehicle's bird's-eye view (BEV) perspective. The bird's-eye view perspective can be understood as a top-down perspective. Of course, fusion can also be performed from other perspectives, such as the vehicle's front perspective. Data from different perspectives can be converted to each other.

[0179] 303. A lane-level navigation map at a first moment is constructed based on the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at a second moment, where the second moment is earlier than the first moment. The lane-level navigation map is used to provide lane-level navigation services to users.

[0180] The lane-level navigation map at the first moment can be understood by referring to FIG4E . As shown in FIG4E , the lane-level navigation map at the first moment can provide lane-level navigation services to the user. The lane-level navigation map shown in FIG4E can be understood as a lane-level navigation map from a bird's-eye view of the vehicle.

[0181] The lane-level navigation map at the first moment can also be understood by referring to FIG4F and FIG4G . FIG4F and FIG4G can be understood as the lane-level navigation map from the front perspective of the vehicle.

[0182] The navigation guidance modes in Figures 4F and 4G are different. The navigation guidance mode in Figure 4F is a lane rendering mode that enhances the lane in which the vehicle is traveling. As shown in Figure 4F, the lane in which the vehicle is traveling is enhanced and rendered. Within a certain distance from the vehicle, the lane in which the vehicle is traveling is highlighted or displayed in a color different from the lane, such as 403 in Figure 4F. The navigation guidance mode in Figure 4G is a mode that renders a guidance arrow on the lane in which the vehicle is traveling. As shown in Figure 4G, a guidance arrow is rendered in front of the vehicle, above the lane in which the vehicle is traveling, such as guidance arrow 404 in Figure 4G. The guidance arrow in Figure 4G is only an example, and the shape and color of the guidance arrow are not limited in this application.

[0183] The lane-level navigation map shown in Figures 4F and 4G can indicate the lane the user should drive in. Of course, Figures 4F and 4G are just two examples of display methods in this application. The lane the user should drive in can also be indicated in other forms, and this application does not limit this.

[0184] In the embodiment of the present application, the first moment and the second moment are different time points during the navigation process. The second moment and the first moment can be the time points corresponding to each frame of two consecutive driving images captured by the camera, or can be the time points corresponding to two non-consecutive driving images. The camera used to capture the driving images can be a vehicle-mounted camera or a camera on a handheld terminal.

[0185] The solution provided in this application can build a lane-level navigation map based on the SD map during the navigation process. It does not require high-precision (HD) maps, nor does it require high-precision sensors and too much computing power to obtain lane-level navigation services, thus providing users with high-quality navigation services at a lower cost.

[0186] The above-mentioned driving image can be understood as an image frame. The above-mentioned step 303 of constructing the lane-level navigation map at the first moment can be determined by the lane-level navigation map at the second moment and the frame estimation result of the image frame at the first moment. The process may include: constructing the lane-level navigation map at the first moment based on the lane-level navigation map at the second moment and the frame estimation result at the first moment; wherein, the frame estimation result at the first moment is used to indicate the confidence of the driving image at the first moment.

[0187] Optionally, the frame estimation result at the first moment is obtained through a target relationship formula, which includes the product of the first weight and the vector direction of the road, the product of the second weight and the inter-frame motion information, and the product of the third weight and the parameter value of the target element.

[0188] Optionally, the inter-frame motion information is a result obtained by projecting a frame estimation result at the second moment onto the first moment based on a change between the vehicle's position at the first moment and the vehicle's position at the second moment.

[0189] The target relationship can be expressed as:

[0190]

[0191] Among them, L(p t ) represents the frame estimation result at the first moment, W navi Represents the first weight, V t Indicates the vector direction of the road, W vdr represents the second weight, B represents inter-frame motion information, post Indicates the position of the vehicle at the first moment, pos t-1 represents the vehicle’s position at the second moment, L(p t-1 ) represents the frame estimation result at the second moment, W de represents the third weight, Represents the results of IPM of target elements i to j in the image frame at the first moment.

[0192] In this optional embodiment, the lane-level navigation map of the current frame may be determined by using historical lane-level navigation maps and the frame estimation result of the current frame, thereby improving the generation efficiency of the lane-level navigation map.

[0193] Optionally, the sizes of the first weight, the second weight and the third weight in the above target relationship equation can be adjusted according to needs. Usually, before constructing the lane-level navigation map at the first moment, the method also includes: updating at least one of the first weight, the second weight or the third weight according to the result of error estimation, the result of error estimation is the difference estimation result between the target element in the driving image at the first moment and the target element in the driving image at the second moment, or the result of error estimation is the difference estimation result between the frame estimation result at the first moment and the frame estimation result at the second moment.

[0194] In the embodiment of the present application, one possible approach is to determine the error estimation result by comparing the similarity of target elements in driving images at two moments.

[0195] By comparing the target elements in the driving image at the first moment with the target elements in the driving image at the second moment, the similarity of the target elements at the two moments can be determined. If the similarity is greater than the first threshold, it means that the target elements in the driving images at the two moments are very similar, because the two moments are usually close. The high similarity indicates that the perception module that extracts the target elements from the driving image is working properly. If the similarity is less than the first threshold, it means that the target elements in the driving images at the two moments are less similar, which may be due to false detection or missed detection by the perception module. In this case, the credibility of the target elements in the driving image at the first moment is low, and the corresponding third weight needs to be reduced, and the frame estimation result of the second moment is retained more. In this way, if the result of the error estimation indicates that the similarity between the target elements in the driving image at the first moment and the target elements in the driving image at the second moment is less than the first threshold, the third weight is adjusted down.

[0196] In an embodiment of the present application, another possible method is to determine the adjusted weight by comparing the frame estimation results at two moments. If the similarity of the frame estimation results at the two moments is small, it means that the lane may have changed at the first moment relative to the second moment. In this case, as much information as possible in the driving image at the first moment should be used, so the third weight needs to be increased. In this way, if the result of the error estimation indicates that the similarity between the frame estimation result at the first moment and the frame estimation result at the second moment is less than the second threshold, the third weight is increased.

[0197] Whether reducing or increasing the third weight, the accuracy of the lane-level navigation map can be improved.

[0198] In addition, in an embodiment of the present application, when generating a lane-level navigation map at the first moment, the position parameters of the lane lines of other lanes with the same direction in the lane-level navigation map at the first moment can also be corrected based on the frame estimation result at the first moment and the orientation parameters and width of the lane where the vehicle is located.

[0199] In an embodiment of the present application, if the frame estimation result at the first moment is too small, it means that the credibility of the prediction result determined based on the target element, the vehicle's posture, and the vector direction of the road in the SD map is very low and has no use value, then there is no need to correct the lane line. If the frame estimation result at the first moment is relatively reasonable, the lane where the vehicle is located is usually the clearest lane in the driving image, so the position parameters of the lane lines of other lanes in the same direction in the navigation map can be corrected according to the orientation parameters and width of the lane lines of the lane where the vehicle is located.

[0200] Lane orientation parameters include the angle, lane offset, and lane curvature. The angle can be described as the angle between the vehicle's frontal orientation and the lane's orientation, or as the angle between the lane and an axis of the vehicle coordinate system. The lane offset is the offset between the lane and the origin of the vehicle coordinate system. For more information on lane orientation parameters, refer to Figure 5A. As shown in Figure 5A, in the vehicle coordinate system, the lane is the result of a high-order fitting. The lane can be understood as a curve with a curvature radius R, a lane offset a0, and an angle θ with the X-axis.

[0201] The schematic diagram of lane correction can be understood by referring to FIG5B . As shown in FIG5B , five lanes are illustrated. At time t, the vehicle is traveling in the second lane on the left. At this time, the first lane on the left and the third lane on the left can be more clearly extracted from the driving image, while the two lanes on the right cannot be accurately restored from the driving image at time t. The two lanes on the right 501 can be corrected to lane 502 based on the width of the second lane on the left where the vehicle is located and the lane orientation parameters.

[0202] After the vehicle changes lanes to the third lane on the left at time t, at time t+1, the vehicle is traveling in the third lane on the left. At this time, the second lane on the left and the fourth lane on the left can be more clearly extracted from the driving image, while the first lane on the left and the first lane on the right cannot be accurately restored from the driving image at time t+1. The first lane on the left and the first lane on the right 501 can be corrected to lane 502 based on the width of the third lane on the left where the vehicle is located and the lane orientation parameters.

[0203] After the vehicle changes lanes to the second lane on the right at time t+1, at time t+2, the vehicle is traveling in the second lane on the right. At this time, the third lane on the left and the first lane on the right can be more clearly extracted from the driving image, while the first vehicle and the second lane on the left cannot be accurately restored from the driving image at time t+2. The first vehicle and the second lane on the left 501 can be corrected to lane 502 based on the width of the second lane on the right where the vehicle is located and the lane orientation parameters.

[0204] In an embodiment of the present application, lane-level positioning guidance can be provided via a lane-level navigation map. This process may include semantically merging the vehicle's lane as shown in the lane-level navigation map at a first moment and the navigation information entered by the user in the standard map to output lane-level navigation guidance prompts. The lane-level navigation guidance prompts may include maintaining the lane, changing lanes left, or changing lanes right.

[0205] In an embodiment of the present application, the navigation information may include the navigation destination and the number of lanes on the current road. Semantic merging refers to directly giving clear navigation instructions without the user having to make another conversion. For example: entering the two left lanes in the SD map navigation scene is merged into keeping the lane or changing lanes to the left, and entering the right lane in the SD map navigation scene is merged into keeping the lane or changing lanes to the right. In this way, the user can perform the corresponding operation according to the lane-level navigation guidance prompts, and the user does not need to observe the current lane and then decide whether to perform the lane change operation.

[0206] In an optional embodiment, if it is determined based on the lane in which the vehicle is located and the navigation information that the vehicle needs to keep going straight, is already in the leftmost lane in a scenario where a U-turn or left turn is required, or is already in the rightmost lane in a scenario where a right turn is required, the lane-level navigation guidance prompt is to maintain the lane.

[0207] In the embodiment of the present application, when keeping the lane, the display effect on the lane-level navigation map can be understood by referring to Figure 6A. As shown in Figure 6A, the navigation guidance on the lane-level navigation map indicates to keep going straight.

[0208] In another optional embodiment, if it is determined based on the lane in which the vehicle is located and the navigation information that the vehicle needs to change lanes, the lane-level navigation guidance prompt is to change lanes to the left or to the right.

[0209] In the embodiment of the present application, taking changing lanes to the left as an example, the display effect on the lane-level navigation map can be understood by referring to FIG6B . As shown in FIG6B , the navigation guidance on the lane-level navigation map indicates changing lanes to the left.

[0210] In an embodiment of the present application, intersection guidance can also be provided in the lane-level navigation map, as shown in FIG6C . For intersection scenarios, intersection guidance can be provided through intersection guidance prompts.

[0211] In multi-lane scenarios, lanes on either side can easily be obscured or missed due to a narrow field of view, making alignment with the navigation lane impossible. After semantic merging, as shown in Figure 6D, the four right lanes are all through lanes, and the semantic positioning result is for the through lanes, not the two lanes from the left. This significantly increases the fault tolerance and robustness of the positioning results without any loss of functionality in the guidance results.

[0212] Semantic-level positioning results undergo semantic post-verification, incorporating prior information about traffic regulations. Generally, left-turn lanes are on the left, and right-turn lanes are on the right. In positioning scenarios involving left or right turns, these are converted to left and right lane changes. Absolute positioning is based on the curb, rather than relative positioning based on left or right numbers, significantly improving accuracy.

[0213] In an embodiment of the present application, during navigation guidance, it is necessary to locate the current lane of the vehicle. If the current lane of the vehicle cannot be determined from the driving image at the first moment, such as in an intersection or congested scene, due to occlusion by vehicles on both sides or occlusion by front and rear measurements, the current lane of the vehicle cannot be determined. This can be done by querying historical driving images (which can determine the vehicle's lane), and then combining the driving behavior of the vehicle between the third moment and the first moment when the historical driving image was generated to determine the lane of the vehicle at the first moment. This process may include: when the lane of the vehicle cannot be determined from the driving image at the first moment, determining the lane of the vehicle at the first moment based on the lane of the vehicle in the driving image at the third moment and the driving behavior from the third moment to the first moment; wherein the third moment is before the first moment and the distance between the third moment and the first moment is less than a third threshold, and the frame estimation result at the third moment is the most reliable estimation result among the driving images generated from the third moment to the first moment. This can meet the requirements of lane-level navigation guidance in complex scenes.

[0214] This process can be understood by referring to Figure 7. As shown in Figure 7, the vehicle is obscured by obstacles around time t+1, making it impossible to locate its lane. This can be solved by querying historical driving images. For example, if the vehicle is not obscured in the driving image at time t, the vehicle can be located in the first lane on the left. In this way, based on the vehicle's location in the first lane at time t and the vehicle's single lane change to the right from time t to time t+1, the vehicle can be located in the second lane on the left at time t+1.

[0215] In addition, in the embodiment of the present application, a virtual arrow is provided in the lane-level navigation map to indicate the navigation direction, as shown in FIG8A , and the direction of the virtual arrow is consistent with the direction of the lane. This better meets the user's driving needs.

[0216] In an embodiment of the present application, during the navigation process, the position of the virtual arrow used to indicate the navigation direction can also be adjusted according to multi-dimensional parameters, where the multi-dimensional parameters include at least two of the heading angle, roll angle, horizontal offset of the vehicle system, or vertical offset of the vehicle system.

[0217] Taking the virtual arrow including the heading angle, roll angle, vehicle system horizontal offset or vehicle system vertical offset as an example, these four dimensions can be obtained by the following relationship: y = a (e -x +1)x

[0218] Among them, a represents the amplification factor, e -x Represents an exponential relationship, x represents the horizontal axis input, and y represents the output of the four dimensions corresponding to different x, where x can be the curvature of the lane line or the angle between the lanes.

[0219] As shown in FIG8B , line 801 represents the roll angle, line 802 represents the heading angle, line 803 represents the horizontal offset of the vehicle system, and line 804 represents the vertical offset of the vehicle system.

[0220] The effect of the virtual arrow obtained by the above relationship can be understood by referring to FIG4G . As shown in FIG4G , the virtual arrow has a more three-dimensional posture and a better presentation method, which can effectively improve the navigation guidance effect.

[0221] In the embodiment of the present application, in order to better understand the above navigation method, the above process can be described as a whole in conjunction with Figure 9A below.

[0222] As shown in Figure 9A, the process includes:

[0223] 901. The vehicle's latitude and longitude data are collected through the GNSS sensor, the corresponding IMU data are collected through the IMU sensor, and the vehicle's wheel speed data can also be collected through the WSS.

[0224] 902. Determine the vehicle's position in the vehicle coordinate system through the VDR module, as well as latitude and longitude data, IMU data and wheel speed data.

[0225] The wheel speed data may or may not be used.

[0226] 903. Obtain driving images through cameras.

[0227] 904. Extract target elements from driving images.

[0228] Target elements can include lane lines, curbs, landmarks and other visual element information.

[0229] 905. Convert the target element to the vehicle coordinate system through IPM.

[0230] 906. Obtain the vector direction of the road corresponding to the SD map through the posture in the world coordinate system provided by VDR, and convert the vector direction value from the world coordinate system to the vehicle coordinate system.

[0231] The above steps 905 and 906 are both converted to the vehicle coordinate system where the vehicle's posture is located, which can be understood as the fusion of data from three different domains, and can also be called heterogeneous data fusion.

[0232] 907. After accumulating multiple frames of heterogeneous data fusion results, filtering is performed.

[0233] The filtering method may be an extended Kalman filter (EKF).

[0234] 908.Constructing local lane-level maps from data in a period of time and space.

[0235] The local lane-level map can also be understood as the lane-level navigation map introduced earlier. The process of constructing the local lane-level map can be understood by referring to the target relationship formula introduced earlier.

[0236] 909. Error estimation.

[0237] 910. Weight update.

[0238] The relationship between the error estimation in step 909 and step 910, as well as the related processes, can be understood by referring to the above corresponding introduction.

[0239] 911. Lane-Level Semantic Merging.

[0240] When the constructed local lane-level map is consistent with the SD map navigation information, it is semantically merged with the lane traffic attributes (left turn, straight, right turn, etc.) provided by the current navigation to complete lane-level positioning.

[0241] This process can be understood by referring to the introduction of FIG. 6A to FIG. 6D .

[0242] 912. Historical trajectory prediction.

[0243] When the local lane-level map is incomplete at the current moment and inconsistent with the navigation lane information, historical trajectory prediction is completed by querying historical data and combining segmentation errors to estimate the current lane-level positioning result.

[0244] This process can be understood by referring to the introduction in Figure 7.

[0245] 913. Lane-level navigation guidance.

[0246] This process can be understood by referring to the introduction of FIG. 6A to FIG. 6D .

[0247] The processes of step 911 and step 912 can also be understood in conjunction with FIG. 9B .

[0248] As shown in FIG9B , the process includes:

[0249] 921. Are the local lane-level map and the SD map aligned? If so, execute step 922; if not, execute step 927.

[0250] 922. Merge lane attributes.

[0251] If the lane the vehicle is in can be determined, the lane traffic attributes (left turn, straight, right turn, etc.) provided by the current navigation can be combined to perform semantic merging and complete lane-level positioning.

[0252] 923. Is semantic alignment achieved? If so, go to step 924; if not, go to step 925.

[0253] Semantic alignment means that the navigation semantics in local lane-level navigation fully include the navigation meaning in the SD map.

[0254] 924. Lane keeping.

[0255] 925. Lane change guidance.

[0256] 926. Traffic regulations verification.

[0257] 927. Query historical data.

[0258] 928. Trajectory prediction.

[0259] If the lane in which the vehicle is located is determined once, steps 927 and 928 may be executed in a loop until the local lane-level map is aligned with the SD map.

[0260] After completing the above steps, the constructed local lane-level map information and navigation status can be used to provide guidance such as lane keeping, lane change guidance, and turn guidance. Furthermore, based on the actual lane line orientation data, dimensionality is increased to improve the fit of the virtual arrow guidance and user experience. This process can be understood by referring to the previous descriptions of Figures 8A, 8B, and 4G.

[0261] This embodiment of the present application further provides another navigation method, as shown in FIG10A . Another embodiment of the navigation method provided by this embodiment of the present application includes:

[0262] 1001. In response to a user clicking a first button on a standard map used for navigation, a camera is started, the camera is used to capture driving images, and the first button is an entry for starting a lane-level navigation service.

[0263] The first button may be an augmented reality (AR) button, as shown in FIG10B . When the user clicks the AR button on the standard map interface, the camera may be activated to capture driving images.

[0264] 1002. Convert the target elements and the vector direction of the road in the standard map to the vehicle body coordinate system, where the target elements include lane lines or curbs in the driving image at the first moment.

[0265] 1003. Construct a lane-level navigation map at a first moment based on the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at a second moment, where the second moment is earlier than the first moment.

[0266] Step 1002 and step 1003 can be understood by referring to steps 302 and 303 in the previous FIG. 3 , and will not be repeated here.

[0267] 1004. Display a lane-level navigation map, which is used to provide lane-level navigation services to users.

[0268] In this embodiment of the application, in the navigation interface of a standard map, by clicking the entry for starting lane-level navigation service, the generation of a lane-level navigation map can be initiated, and then the lane-level navigation map is obtained for user navigation. Lane-level navigation service does not require HD maps, high-precision sensors, or excessive computing power, thus providing users with high-quality navigation services at a lower cost.

[0269] In this possible embodiment, the processes shown in FIG. 4A to FIG. 9B may also be performed, and will not be repeated here.

[0270] Optionally, in an embodiment of the present application, this may include: responding to a user's switching operation of a perspective button on a lane-level navigation map, switching from a lane-level navigation map of a first perspective to a lane-level navigation map of a second perspective, the first perspective including a bird's-eye view of the vehicle or a front perspective of the vehicle, the second perspective including a front perspective of the vehicle or a bird's-eye view of the vehicle, and the first perspective is different from the second perspective.

[0271] This embodiment can be understood by referring to the previous Figures 4E and 4F. The user can click the perspective button on the lane-level navigation map to perform a switching operation, such as: switching from the lane-level navigation map of the top view perspective of Figure 4E to the lane-level navigation map of the front view perspective of Figure 4F. Of course, it can also be done vice versa, switching from the lane-level navigation map of the front view perspective of Figure 4F to the lane-level navigation map of the top view perspective of Figure 4E.

[0272] Of course, the above-mentioned perspective switching process can also be understood by referring to the previous Figures 4E and 4G, switching from the lane-level navigation map of the top view perspective of Figure 4E to the lane-level navigation map of the front view perspective of Figure 4G. Of course, it can also be reversed, switching from the lane-level navigation map of the front view perspective of Figure 4G to the lane-level navigation map of the top view perspective of Figure 4E.

[0273] In an embodiment of the present application, the user can switch between a top-down perspective and a front perspective. Of course, other perspectives can also be included. The user can switch between different perspectives according to needs, such as: left perspective, right perspective, rear perspective, etc., thereby displaying lane-level navigation maps from different perspectives.

[0274] Although the viewing angle button is not shown in FIG. 4E , FIG. 4F and FIG. 4G , it is understandable that the viewing angle button can be presented in various possible forms, which is not limited in this application.

[0275] Optionally, in an embodiment of the present application, this may include: responding to a user's switching operation of the navigation guidance mode on the lane-level navigation map, switching from a first navigation guidance mode to a second navigation guidance mode, the first navigation guidance mode including an enhanced lane rendering mode for the driving lane or a mode for rendering guidance arrows on the driving lane, and the first navigation guidance mode is different from the second navigation guidance mode.

[0276] This embodiment can be understood by referring to the previous Figures 4F and 4G. The user can click on the navigation guidance mode on the lane-level navigation map to switch operations, such as switching from the navigation guidance mode of Figure 4F, which enhances the lane rendering of the driving lane, to the mode of Figure 4G, which renders the guidance arrow on the driving lane. Of course, it is also possible to switch from the mode of Figure 4G, which renders the guidance arrow on the driving lane, to the navigation guidance mode of Figure 4F, which enhances the lane rendering of the driving lane. Of course, the embodiments of the present application may also include other forms of navigation guidance modes, and switching between different navigation guidance modes is possible.

[0277] Although Figures 4F and 4G do not illustrate the buttons related to switching the navigation guidance mode, it can be understood that the navigation guidance mode can be switched by buttons or other methods, and this is not limited in this application.

[0278] Optionally, in an embodiment of the present application, this may include: responding to the user's control operation of the information button on the lane-level navigation map, closing the navigation guidance prompts on the lane-level navigation map, or displaying the navigation guidance prompts on the lane-level navigation map, the navigation guidance prompts including keeping lane, changing lanes to the left, changing lanes to the right, intersection indications or warning indications.

[0279] This embodiment can be understood by referring to the previous Figures 6A to 6C. As shown in Figures 6A to 6C, navigation guidance prompts such as speed, lane change instructions, camera, and intersection guidance are displayed on the lane-level navigation map. The user can turn off the navigation guidance prompts or turn on the navigation guidance prompts by operating the information button.

[0280] Although the information button is not shown in FIG. 6A to FIG. 6C , it is understandable that the information button can be presented in various possible forms, which is not limited in this application.

[0281] Although the above description describes switching through buttons, it is understandable that switching between different interfaces can also be performed through voice control, which is not limited in this application.

[0282] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Please refer to Figure 11 for details. Figure 11 is a structural diagram of a navigation device provided by the embodiment of the present application. The navigation device 1100 may include an acquisition unit 1101, a first processing unit 1102 and a second processing unit 1103, wherein:

[0283] The acquisition unit 1101 is configured to acquire a driving image and position of the vehicle at a first moment, as well as a standard map for navigation. The acquisition unit 1101 may execute step 301 in the method embodiment corresponding to FIG. 3 .

[0284] The first processing unit 1102 is configured to convert target elements and vector directions of roads in the standard map into a vehicle body coordinate system. The target elements include lane lines or curbs in the driving image. The first processing unit 1102 may execute step 302 in the method embodiment corresponding to FIG. 3 .

[0285] The second processing unit 1103 is configured to construct a lane-level navigation map at the first moment based on the road vector direction, the target element, and the vehicle's position at the first moment and the vehicle's position at the second moment, where the second moment is earlier than the first moment. The lane-level navigation map is used to provide lane-level navigation services to the user. The second processing unit 1103 may execute step 303 in the method embodiment corresponding to FIG. 3 .

[0286] Optionally, the second processing unit 1103 is specifically used to construct a lane-level navigation map at the first moment based on the lane-level navigation map at the second moment and the frame estimation result at the first moment; wherein, the frame estimation result at the first moment is used to indicate the confidence of the driving image at the first moment.

[0287] Optionally, the frame estimation result at the first moment is obtained through a target relationship formula, which includes the product of the first weight and the vector direction of the road, the product of the second weight and the inter-frame motion information, and the product of the third weight and the parameter value of the target element.

[0288] Optionally, the inter-frame motion information is a result obtained by projecting a frame estimation result at the second moment onto the first moment based on a change between the vehicle's position at the first moment and the vehicle's position at the second moment.

[0289] Optionally, the second processing unit 1103 is further used to update at least one of the first weight, the second weight or the third weight according to the result of error estimation before constructing the lane-level navigation map at the first moment, where the result of error estimation is the difference estimation result between the target element in the driving image at the first moment and the target element in the driving image at the second moment, or the result of error estimation is the difference estimation result between the frame estimation result at the first moment and the frame estimation result at the second moment.

[0290] Optionally, if the error estimation result indicates that the similarity between the target element in the driving image at the first moment and the target element in the driving image at the second moment is less than a first threshold, the third weight is adjusted down.

[0291] Optionally, if the result of the error estimation indicates that the similarity between the frame estimation result at the first moment and the frame estimation result at the second moment is less than a second threshold, the third weight is increased.

[0292] Optionally, the second processing unit 1103 is further used to correct the position parameters of lane lines of other lanes with the same direction in the lane-level navigation map at the first moment based on the frame estimation result at the first moment and the orientation parameters and width of the lane where the vehicle is located.

[0293] Optionally, the second processing unit 1103 is further used to perform semantic merging based on the lane in which the vehicle is located in the lane-level navigation map at the first moment and the navigation information input by the user in the standard map to output lane-level navigation guidance prompts, where the lane-level navigation guidance prompts include maintaining the lane, changing lanes to the left, or changing lanes to the right.

[0294] Optionally, if it is determined based on the lane and navigation information that the vehicle needs to keep going straight, is already in the leftmost lane where a U-turn or left turn is required, or is already in the rightmost lane where a right turn is required, the lane-level navigation guidance prompt is to maintain the lane.

[0295] Optionally, if it is determined based on the lane in which the vehicle is located and navigation information that the vehicle needs to change lanes, the lane-level navigation guidance prompt is to change lanes to the left or to the right.

[0296] Optionally, the second processing unit 1103 is also used to determine the lane in which the vehicle is located at the first moment based on the lane in which the vehicle is located in the driving image at the third moment and the driving behavior from the third moment to the first moment, when the lane in which the vehicle is located cannot be determined by the driving image at the first moment; wherein the third moment is before the first moment, and the distance between the third moment and the first moment is less than a third threshold, and the frame estimation result at the third moment is the most credible estimation result in the driving image generated from the third moment to the first moment.

[0297] Optionally, the second processing unit 1103 is further used to adjust the posture of the virtual arrow used to indicate the navigation direction according to multi-dimensional parameters during the navigation process, where the multi-dimensional parameters include at least two of the heading angle, roll angle, horizontal offset of the vehicle system, or vertical offset of the vehicle system.

[0298] Optionally, during navigation, the direction of the virtual arrow is consistent with the direction of the lane.

[0299] It should be noted that the information interaction, execution process, etc. between the modules / units in the navigation device 1100 are based on the same concept as the various method embodiments corresponding to Figures 3 to 9B in this application. For specific contents, please refer to the description of the method embodiments shown in the aforementioned embodiment of this application, and will not be repeated here.

[0300] FIG12 is a schematic diagram of a structure of a navigation device provided in an embodiment of the present application. The navigation device 1200 may include a first processing unit 1201, a second processing unit 1202, a third processing unit 1203, and a display unit 1204, wherein:

[0301] The first processing unit 1201 is configured to respond to a user clicking a first button on a standard map for navigation, thereby activating a camera for capturing driving images. The first button is an entry for activating a lane-level navigation service.

[0302] The second processing unit 1202 is configured to convert target elements and vector directions of roads in the standard map into a vehicle body coordinate system, where the target elements include lane lines or curbs in the driving image at the first moment.

[0303] The third processing unit 1203 is used to construct a lane-level navigation map at a first moment according to the vector direction of the road, the target element, and the vehicle's position at the first moment and the vehicle's position at a second moment, where the second moment is earlier than the first moment.

[0304] The display unit 1204 is used to display a lane-level navigation map. The lane-level navigation map is used to provide lane-level navigation services to users.

[0305] Optionally, the first processing unit 1201 is also used to respond to the user's switching operation of the perspective button on the lane-level navigation map, switching from the lane-level navigation map of the first perspective to the lane-level navigation map of the second perspective, the first perspective includes the vehicle's top-down perspective or the vehicle's front perspective, the second perspective includes the vehicle's front perspective or the vehicle's top-down perspective, and the first perspective is different from the second perspective.

[0306] Optionally, the first processing unit 1201 is also used to respond to the user's switching operation of the navigation guidance mode on the lane-level navigation map, switching from the first navigation guidance mode to the second navigation guidance mode, the first navigation guidance mode includes an enhanced lane rendering mode for the driving lane or a mode for rendering guidance arrows on the driving lane, and the first navigation guidance mode is different from the second navigation guidance mode.

[0307] Optionally, the first processing unit 1201 is also used to respond to the user's control operation of the information button on the lane-level navigation map, close the navigation guidance prompts on the lane-level navigation map, or display the navigation guidance prompts on the lane-level navigation map, the navigation guidance prompts including keeping lane, changing lanes to the left, changing lanes to the right, intersection indications or warning indications.

[0308] It should be noted that the information interaction, execution process, etc. between the modules / units in the navigation device 1200 are based on the same concept as the various method embodiments corresponding to Figures 3 to 10B in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0309] The embodiment of the present application also provides a vehicle. In combination with the above description of Figure 1, please refer to Figure 13. Figure 13 is a structural schematic diagram of the vehicle provided in the embodiment of the present application, wherein the navigation device 1100 described in the embodiment corresponding to Figure 11 or the navigation device 1200 described in the embodiment corresponding to Figure 12 can be deployed on the vehicle 100 to implement the functions of the vehicle in the embodiments corresponding to Figures 3 to 10B. Since in some embodiments, the vehicle 100 can also include a communication function, the vehicle 100, in addition to the components shown in Figure 1, can also include: a receiver 1301 and a transmitter 1302, wherein the processor 133 can include an application processor 1331 and a communication processor 1332. In some embodiments of the present application, the receiver 1301, the transmitter 1302, the processor 133 and the memory 134 can be connected via a bus or other means.

[0310] Processor 133 controls vehicle operations. In specific applications, the various components of vehicle 100 are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, the various buses are collectively referred to as a bus system in the figure.

[0311] Receiver 1301 can be used to receive input digital or character information and generate signal input related to vehicle settings and function control. Transmitter 1302 can be used to output digital or character information through the first interface. Transmitter 1302 can also be used to send instructions to the disk pack through the first interface to modify the data in the disk pack. Transmitter 1302 can also include a display device such as a display screen.

[0312] In an embodiment of the present application, the application processor 1331 is used to execute the navigation method performed by the vehicle in the embodiment corresponding to Figure 2. Specifically, the application processor 1331 is used to perform the following steps: obtaining a driving image and position of the vehicle at a first moment, and a standard map for navigation; converting target elements and the vector direction of the road in the standard map to the vehicle body coordinate system, where the target elements include lane lines or curbs in the driving image; constructing a lane-level navigation map at the first moment based on the vector direction of the road, the target elements, and the position of the vehicle at the first moment and the position of the vehicle at the second moment, where the second moment is earlier than the first moment, and the lane-level navigation map is used to provide lane-level navigation services to users.

[0313] The application processor 1331 may also be configured to execute steps 1001 to 1003 in the embodiment corresponding to FIG. 10A .

[0314] Of course, the application processor 1331 can also perform other functions performed by the second processing unit 1103 in Figure 11 above, or it can also perform other functions performed by the first processor 1201, the second processing unit 1202 or the third processor 1203 in Figure 12 above, which are not listed one by one in this application.

[0315] It should be noted that for the specific implementation of the navigation method executed by the application processor 1331 and the beneficial effects brought about, reference can be made to the descriptions in the corresponding method embodiments of Figures 3 to 10B, and no further details will be given here.

[0316] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for generating a vehicle driving speed. When the program is running on a computer, the computer executes the steps performed in the method described in the embodiments shown in Figures 3 to 10B above.

[0317] An embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the steps performed in the method described in the embodiments shown in Figures 3 to 10B above.

[0318] An embodiment of the present application further provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the steps performed in the method described in the embodiments shown in Figures 3 to 10B above.

[0319] The navigation device or vehicle provided in the embodiments of the present application may specifically be a chip, which includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit, so that the chip in the server executes the vehicle speed generation method described in the embodiments shown in Figures 3 to 9B above. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0320] Specifically, see Figure 14 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip may be a neural network processor (NPU) 140. NPU 140 is mounted on a host CPU (host CPU) as a coprocessor, with tasks assigned by the host CPU. The core of the NPU is arithmetic circuit 140 , which controls arithmetic circuit 1403 via controller 1404 to extract matrix data from memory and perform multiplication operations.

[0321] In some implementations, the arithmetic circuit 1403 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1403 is a two-dimensional systolic array. The arithmetic circuit 1403 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.

[0322] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1402 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1401 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1408.

[0323] Unified memory 1406 is used to store input and output data. Weight data is directly transferred to weight memory 1402 through the Direct Memory Access Controller (DMAC) 1405. Input data is also transferred to unified memory 1406 through the DMAC.

[0324] BIU stands for Bus Interface Unit, i.e., bus interface unit 1410 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1409 .

[0325] The bus interface unit 1410 (BIU) is used for the instruction fetch memory 1409 to obtain instructions from the external memory, and is also used for the storage unit access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0326] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1406 or transfer weight data to the weight memory 1402 or transfer input data to the input memory 1401.

[0327] The vector calculation unit 1407 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0328] In some implementations, the vector calculation unit 1407 can store the processed output vector to the unified memory 1406. For example, the vector calculation unit 1407 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1403, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1407 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1403, for example, for use in a subsequent layer in a neural network.

[0329] An instruction fetch buffer 1409 connected to the controller 1404 is used to store instructions used by the controller 1404;

[0330] Unified memory 1406, input memory 1401, weight memory 1402, and instruction fetch memory 1409 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0331] The operations of each layer in the recurrent neural network can be performed by the operation circuit 1403 or the vector calculation unit 1407.

[0332] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0333] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0334] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CLUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0335] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0336] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive (SSD)).

Claims

1. A navigation method, characterized in that: include: Obtain the vehicle's driving image and posture at the first moment, as well as a standard map for navigation; Converting target elements and vector directions of roads in the standard map to a body coordinate system of the vehicle, wherein the target elements include lane lines or curbs in the driving image; A lane-level navigation map at the first moment is constructed based on the vector direction of the road, the target element, and the position of the vehicle at the first moment and the position of the vehicle at the second moment, where the second moment is earlier than the first moment, and the lane-level navigation map is used to provide lane-level navigation services to the user.

2. The method according to claim 1, characterized in that The constructing a lane-level navigation map at the first moment according to the vector direction of the road, the target element, and the position of the vehicle at the first moment and the position of the vehicle at the second moment includes: A lane-level navigation map at the first moment is constructed based on the lane-level navigation map at the second moment and the frame estimation result at the first moment; wherein the frame estimation result at the first moment is used to indicate the confidence of the driving image at the first moment.

3. The method according to claim 2, characterized in that The frame estimation result at the first moment is obtained through a target relationship formula, which includes the product of the first weight and the vector direction of the road, the product of the second weight and the inter-frame motion information, and the product of the third weight and the parameter value of the target element.

4. The method according to claim 3, characterized in that The inter-frame motion information is a result obtained by projecting a frame estimation result at the second moment onto the first moment based on a change between the position of the vehicle at the first moment and the position of the vehicle at the second moment.

5. The method according to claim 3 or 4, characterized in that Before constructing the lane-level navigation map at the first moment, the method further includes: At least one of the first weight, the second weight or the third weight is updated according to the result of the error estimation, wherein the result of the error estimation is the difference estimation result between the target element in the driving image at the first moment and the target element in the driving image at the second moment, or the result of the error estimation is the difference estimation result between the frame estimation result at the first moment and the frame estimation result at the second moment.

6. The method according to claim 5, characterized in that If the result of the error estimation indicates that the similarity between the target element in the driving image at the first moment and the target element in the driving image at the second moment is less than a first threshold, the third weight is reduced.

7. The method according to claim 5, characterized in that If the error estimation result indicates that the similarity between the frame estimation result at the first moment and the frame estimation result at the second moment is less than a second threshold, the third weight is increased.

8. The method according to any one of claims 2 to 7, characterized in that: The method further comprises: According to the frame estimation result at the first moment and the orientation parameter and width of the lane where the vehicle is located, the position parameters of the lane lines of other lanes with the same orientation in the lane-level navigation map at the first moment are corrected.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: Semantic merging is performed based on the lane in which the vehicle is located in the lane-level navigation map at the first moment and the navigation information input by the user in the standard map to output lane-level navigation guidance prompts, where the lane-level navigation guidance prompts include maintaining the lane, changing lanes to the left, or changing lanes to the right.

10. The method according to claim 9, characterized in that If it is determined based on the lane in which the vehicle is located and the navigation information that the vehicle needs to keep going straight, is already in the leftmost lane where a U-turn or left turn is required, or is already in the rightmost lane where a right turn is required, the lane-level navigation guidance prompt is to maintain the lane.

11. The method according to claim 9, characterized in that If it is determined that the vehicle needs to change lanes based on the lane in which the vehicle is located and the navigation information, the lane-level navigation guidance prompt is to change lanes to the left or to change lanes to the right.

12. The method according to any one of claims 9 to 11, characterized in that: The method further comprises: When the lane in which the vehicle is located cannot be determined through the driving image at the first moment, the lane in which the vehicle is located at the first moment is determined based on the lane in which the vehicle is located in the driving image at the third moment and the driving behavior from the third moment to the first moment; wherein the third moment is before the first moment, and the distance between the third moment and the first moment is less than a third threshold, and the frame estimation result at the third moment is the most credible estimation result in the driving image generated from the third moment to the first moment.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: During navigation, the position of a virtual arrow indicating the navigation direction is adjusted according to multi-dimensional parameters, wherein the multi-dimensional parameters include at least two of the heading angle, the roll angle, the horizontal offset of the vehicle system, or the vertical offset of the vehicle system.

14. The method according to claim 13, characterized in that During navigation, the direction of the virtual arrow is consistent with the direction of the lane.

15. A navigation method, characterized in that: include: In response to a user clicking a first button on a standard map for navigation, activating a camera for capturing driving images, wherein the first button is an entry for activating a lane-level navigation service; Converting target elements and vector directions of roads in the standard map to a body coordinate system of the vehicle, wherein the target elements include lane lines or curbs in the driving image at a first moment; Constructing a lane-level navigation map at the first moment according to the vector direction of the road, the target element, and the position of the vehicle at the first moment and the position of the vehicle at the second moment, where the second moment is earlier than the first moment; The lane-level navigation map is displayed, and the lane-level navigation map is used to provide lane-level navigation services for users.

16. The method according to claim 15, characterized in that The method further comprises: In response to the user's switching operation of the perspective button on the lane-level navigation map, the lane-level navigation map is switched from a first perspective to a second perspective, the first perspective includes a top-down perspective of the vehicle or a front perspective of the vehicle, the second perspective includes a front perspective of the vehicle or a top-down perspective of the vehicle, and the first perspective is different from the second perspective.

17. The method according to claim 15 or 16, characterized in that The method further comprises: In response to the user's switching operation of the navigation guidance mode on the lane-level navigation map, switching is performed from the first navigation guidance mode to the second navigation guidance mode, the first navigation guidance mode includes an enhanced lane rendering mode for the driving lane or a mode for rendering guidance arrows on the driving lane, and the first navigation guidance mode is different from the second navigation guidance mode.

18. The method according to any one of claims 15 to 17, characterized in that: The method further comprises: In response to the user's control operation of the information button on the lane-level navigation map, the navigation guidance prompt on the lane-level navigation map is closed, or the navigation guidance prompt is displayed on the lane-level navigation map, the navigation guidance prompt including keeping lane, changing lanes to the left, changing lanes to the right, intersection indication or warning indication.

19. A client, characterized in that: include: A transceiver, a processor and a memory, wherein the transceiver and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the programs or instructions are executed by the processor, the client executes the method according to any one of claims 1-14 or 15-18.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a computer device, the computer device is caused to execute the method according to any one of claims 1 to 14 or 15 to 18.

21. A computer program product, characterized in that The computer program product comprises a computer program code, which, when run on a computer device, causes the computer device to perform the method according to any one of claims 1 to 14 or 15 to 18.

22. A chip system, characterized in that: The chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the client's memory and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the client executes the method as described in any one of claims 1-14 or 15-18.

23. A vehicle, characterized in that: include: A camera, a sensor, and an on-board terminal, wherein the on-board terminal is communicatively connected with the camera and the sensor; The camera is used to capture driving images; The sensor is used to collect data related to the vehicle's posture; The vehicle-mounted terminal includes the client described in claim 19 or the chip system described in claim 22.

Images