Navigation path generation method, apparatus, electronic device, vehicle and storage medium

Abstract

The present disclosure belongs to the technical field of artificial intelligence, relates to a navigation technology in the technical field, and specifically provides a navigation path generation method, comprising: acquiring sensing data from at least one sensor, and performing feature extraction on the sensing data to obtain sensing features; acquiring map navigation data, and encoding the map navigation data to obtain road features, wherein the map navigation data comprises navigation path attribute data, the navigation path attribute data is used for indicating whether a road is a navigation path, and the road features comprise representations of the navigation path attribute data; and, on the basis of the sensing features and the road features, obtaining a navigation path. The technical solution provided in the present disclosure generates navigation paths independently of high-precision maps, and can determine the navigation paths in various complex road topology scenarios, thereby improving the accuracy and safety of path planning.

Description

Methods, devices, electronic equipment, vehicles, and storage media for generating navigation routes

[0001] This application claims priority to Chinese Patent Application No. 202510106098.7, filed on January 22, 2025, entitled “Method, Apparatus, Electronic Device, Vehicle and Storage Medium for Generating Navigation Paths”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, vehicle, and storage medium for generating navigation paths. Background Technology

[0003] Navigation technology can generate road and lane information—i.e., a navigation path—based on the user's selected destination using map information or a map engine. This allows the planning and control module to guide the autonomous vehicle to follow the navigation path constraints. However, ordinary electronic navigation maps are not accurate enough to be directly used for determining autonomous driving navigation paths. High-definition maps (HD maps), due to their high production costs, long update cycles, and untimely road topology updates, are prone to causing safety accidents and thus cannot meet the safety requirements of navigation or autonomous driving.

[0004] Therefore, the technology for generating navigation paths still needs further optimization and improvement. Summary of the Invention

[0005] In view of the above-mentioned problems of the prior art, this disclosure provides a method, device, electronic device, vehicle and storage medium for generating navigation paths, which integrates perception features with road features of map navigation, so that the perception features carry navigation path information, thereby generating navigation paths for autonomous driving devices without relying on high-precision maps, and solves the safety problems caused by the high cost, long update cycle and untimely update of road topology of high-precision maps.

[0006] To achieve the above objectives, the first aspect of this disclosure provides a method for generating a navigation path, the method comprising: acquiring perception data from at least one sensor; extracting features from the perception data to obtain perception features; acquiring map navigation data; encoding the map navigation data to obtain road features, wherein the map navigation data includes navigation path attribute data, the navigation path attribute data being used to indicate whether a road is a navigation path, and the road features including a representation of the navigation path attribute data; and obtaining a navigation path based on the perception features and the road features.

[0007] Therefore, the navigation path generation method provided by the first aspect of this disclosure firstly generates a navigator path based on perception features and road features. This involves the interaction between perception data and road features of map navigation data at the original feature level, so that the perception features carry navigation path information, thereby realizing the generation of navigation paths without relying on high-precision maps and solving the safety problems caused by relying on high-precision maps. Secondly, the perception information obtained from sensors and the guidance information obtained from navigation maps are processed to generate navigation paths. This fully utilizes the original features and guidance information, achieving higher and more accurate navigation path recognition and generation, and improving the accuracy and safety of path planning.

[0008] As one possible implementation of the first aspect, map navigation data includes road geometry data, road attribute data, and road topology data.

[0009] As one possible implementation of the first aspect, the map navigation data is encoded to obtain road features, including: encoding the map navigation data to obtain road features, including: encoding road geometry data and road attribute data to obtain basic road features, wherein the road attribute data includes navigation path attributes; encoding road geometry data and road topology data to obtain road relationship features; and fusing the basic road features and road relationship features to obtain road features.

[0010] As described above, by encoding and interacting road geometric data, attribute data, geometric relationship data, and road topology data, road features can carry not only road-related geometric and attribute information, but also approximate relationships between road and navigation path attributes, as well as left-right and front-back topological relationships between navigation and non-navigation paths at intersections or forks. This allows for better correlation between the representation of map navigation data and the representation of perception data, and can solve the problem of map geometric deviation in real-world applications.

[0011] As one possible implementation of the first aspect, road geometry data and road attribute data include point-level data, lane-level data, and road-level data.

[0012] As one possible implementation of the first aspect, road geometry data and road attribute data include point-level data and road-level data.

[0013] As one possible implementation of the first aspect, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data, lane-level data and road-level data to obtain features of point-level data, lane-level data and road-level data; and splicing or fusing the features of point-level data, lane-level data and road-level data to obtain basic road features.

[0014] As one possible implementation of the first aspect, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data and road-level data to obtain features of point-level data and features of road-level data; and splicing or fusing the features of point-level data and features of road-level data to obtain basic road features.

[0015] As described above, by encoding the geometric and attribute information of roads in a hierarchical manner, the comprehensive and rich expression of road information is ensured, and the ability to represent road features is enhanced.

[0016] As one possible implementation of the first aspect, the map navigation data also includes lane data. The map navigation data is encoded to obtain road features, which include lane features.

[0017] As one possible implementation of the first aspect, the map navigation data also includes element data, which is encoded to obtain road features, including: where the road features also include element features.

[0018] Therefore, this disclosure can be applied not only to road-level navigation map scenarios but also to lane-level navigation map scenarios, demonstrating strong versatility and generalizability.

[0019] As one possible implementation of the first aspect, a navigation path is obtained based on perception features and road features, including: fusing perception features and road features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information. The geometric information is an ordered set of coordinate points describing the lane centerline and the corresponding left and right lane lines, as well as the ordered set of coordinate points describing the road centerline and the corresponding left and right boundary lines. The attribute information is information describing the attributes of the lane centerline, the left and right lane lines, the road centerline, and the left and right boundary lines. The navigation path information indicates whether a road is a navigation path. The navigation path information for each road is obtained, and the navigation path information for each road is tracked to obtain the navigation path.

[0020] As one possible implementation of the first aspect, a navigation path is obtained based on perception features and road features, including: fusing road features and road query features to obtain basic fused features; fusing basic fused features and perception features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road, and tracking the navigation path information for each road to obtain the navigation path.

[0021] As one possible implementation of the first aspect, a navigation path is obtained based on perception features and road features, including: decoding the perception features to obtain the embedding of the perception features; fusing the embedding of the perception features and the road features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain the navigation path.

[0022] As described above, by fusing perceptual features and road features, and determining whether the current road has navigation path information when outputting road information, the perceptual information is fused and interacted with the road features from the map at the original feature level. This enables the decoding neural network to pay attention to and distinguish the underlying features of the perceptual world corresponding to the map navigation route / non-navigation route, thereby realizing the recognition of navigation paths for real-world road structures.

[0023] As one possible implementation of the first aspect, a navigation path is obtained based on perception features and road features, including: fusing perception features and road features to obtain fused features; and decoding the fused features to obtain the navigation path.

[0024] Therefore, the decoding neural network directly decodes and outputs the navigation path on a separate path. This way, the navigation path output is not limited by road information output, enabling navigation paths to be provided even in special scenarios such as intersections, and in scenarios without road awareness, such as crossing large intersections. It is applicable to various traffic road conditions, such as ordinary roads, forks in the road, intersections, entrances and exits of main and auxiliary roads, ramps, and multi-fork roads. It can correctly select the real-world navigation path even in these complex topological scenarios, effectively improving the safety of autonomous driving.

[0025] A second aspect of this disclosure provides a navigation path generation apparatus. It includes: a first acquisition module configured to acquire perception data from at least one sensor; a first neural network configured to extract features from the perception data to obtain perception features; a second acquisition module configured to acquire map navigation data; a second neural network configured to encode the map navigation data to obtain road features, wherein the map navigation data includes navigation path attribute data indicating whether a road is a navigation path, and the road features include representations of the navigation path attribute data; and a third neural network configured to obtain a navigation path based on the perception features and the road features.

[0026] As one possible implementation of the second aspect, map navigation data includes road geometry data, road attribute data, and road topology data.

[0027] As one possible implementation of the second aspect, the second neural network is configured to encode map navigation data to obtain road features, including: encoding road geometry data and road attribute data to obtain basic road features, wherein the road attribute data includes navigation path attributes; encoding road geometry data and road topology data to obtain road relationship features; and fusing the basic road features and road relationship features to obtain road features.

[0028] As one possible implementation of the second aspect, road geometry data and road attribute data include point-level data, lane-level data, and road-level data.

[0029] As one possible implementation of the second aspect, road geometry data and road attribute data include point-level data and road-level data.

[0030] As a possible implementation of the second aspect, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data, lane-level data and road-level data to obtain features of point-level data, lane-level data and road-level data; and splicing or fusing the features of point-level data, lane-level data and road-level data to obtain basic road features.

[0031] As a possible implementation of the second aspect, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data and road-level data to obtain features of point-level data and features of road-level data; and splicing or fusing the features of point-level data and features of road-level data to obtain basic road features.

[0032] As one possible implementation of the second aspect, the map navigation data also includes lane data. The map navigation data is encoded to obtain road features, which include lane features.

[0033] As one possible implementation of the second aspect, the map navigation data also includes element data. The map navigation data is encoded to obtain road features, which include element features.

[0034] As a possible implementation of the second aspect, the third neural network is configured to: obtain a navigation path based on perceptual features and road features, including: fusing the perceptual features and road features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, the geometric information being an ordered set of coordinate points describing the lane centerline and the corresponding left and right lane lines, as well as an ordered set of coordinate points describing the road centerline and the corresponding left and right boundary lines, the attribute information being information describing the attributes of the lane centerline, the left and right lane lines, the road centerline, and the left and right boundary lines, and the navigation path information indicating whether a road is a navigation path; acquiring the navigation path information for each road, and tracking the navigation path information for each road to obtain a navigation path.

[0035] As a possible implementation of the second aspect, the third neural network is configured to: obtain a navigation path based on perceptual features and road features, including: fusing road features and road query features to obtain basic fused features; fusing basic fused features and perceptual features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain a navigation path.

[0036] As a possible implementation of the second aspect, the third neural network is configured to: obtain a navigation path based on perceptual features and road features, including: decoding the perceptual features to obtain an embedding of the perceptual features; fusing the embedding of the perceptual features and the road features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain a navigation path.

[0037] As a possible implementation of the second aspect, the third neural network is configured to: obtain a navigation path based on perceptual features and road features, including: fusing the perceptual features and road features to obtain fused features; and decoding the fused features to obtain the navigation path.

[0038] A third aspect of this disclosure provides an electronic device, including: a processor and a memory; the memory storing program instructions thereon, which, when executed by the processor, cause the processor to perform the navigation path generation method of any of the first aspects described above.

[0039] A fourth aspect of this disclosure provides a vehicle, including a body and electronic equipment provided in the third aspect.

[0040] The fifth aspect of this disclosure provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a computer, cause the computer to perform the navigation path generation method of any of the first aspects described above.

[0041] A sixth aspect of this disclosure provides a computer program product comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads and executes the computer instructions from the computer-readable medium, causing the computer device to implement the navigation path generation method of any of the first aspects described above.

[0042] These and other aspects of this disclosure will become more readily apparent in the description of the following embodiments(s). Attached Figure Description

[0043] Figure 1 is a schematic diagram of a scenario in which a navigation path generation method provided in an embodiment of this disclosure is applied;

[0044] Figure 2 is a schematic diagram of another application scenario of the navigation path generation method provided in the embodiments of this disclosure;

[0045] Figures 3A and 3B are schematic diagrams of another application scenario of the navigation path generation method provided in the embodiments of this disclosure;

[0046] Figure 4 is a schematic diagram of the structure of the navigation path generation method provided in the embodiment of this disclosure;

[0047] Figure 5 is a flowchart of a navigation path generation method provided in an embodiment of this disclosure;

[0048] Figure 6 is a schematic diagram of the map navigation data encoding structure provided in an embodiment of this disclosure;

[0049] Figure 7 is a schematic diagram of a scenario for an encoding method provided in an embodiment of this disclosure;

[0050] Figure 8 is a schematic diagram of the structure of a navigation path generation device provided in an embodiment of this disclosure;

[0051] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0052] The terms "first, second, third, etc." or similar terms such as module A, module B, module C, etc., used in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that a specific order or sequence may be interchanged where permitted so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0053] In the following description, the labels of the steps involved, such as S110, S120, etc., are described only as an exemplary implementation. The order of the steps can be interchanged or performed simultaneously where permitted.

[0054] The term "comprising" as used in the specification and claims should not be construed as limiting itself to what follows; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the mentioned feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components, or groups thereof. Thus, the statement "device comprising means A and B" should not be limited to a device consisting solely of components A and B.

[0055] The terms "an embodiment" or "an embodiment" as used in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in at least one embodiment of this disclosure. Therefore, the terms "in an embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, in one or more embodiments, the particular features, structures, or characteristics can be combined in any suitable manner, as will be apparent to those skilled in the art from this disclosure.

[0056] The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to limit this disclosure.

[0057] First, the relevant background of the embodiments of this application will be introduced.

[0058] Navigation technology calculates how a vehicle should drive in a given environment to reach a predetermined destination. In-vehicle infotainment systems or navigation software can generate road and lane information along the vehicle's path based on map data or a map engine, according to the user-selected destination. This information is then used by the planning and control module to guide the vehicle according to the navigation path constraints. Navigation technology also has wide applications in autonomous driving.

[0059] Current solutions include navigation route determination based on high-definition maps. After obtaining the planned route from the navigation map, the high-definition map uses high-precision positioning to acquire road structure information around the vehicle in the high-definition map, and matches it with the navigation map. Alternatively, it directly utilizes the correspondence between the high-definition map and the navigation map, loading the detailed road and lane information from the high-definition map onto the navigation map as a reference navigation route for the vehicle. However, high-definition maps are costly to produce, have long update cycles, and untimely road topology updates can easily lead to safety accidents. Furthermore, high-definition maps require extremely high positioning accuracy; when satellite signals are blocked or multipath effects occur, positioning deviations or degradations can occur, leading to safety issues.

[0060] Another approach is to perform geometric similarity matching between the standard definition map (SD MAP) navigation route and the vehicle's perception results, using the matched perception results as the navigation path. However, this approach has a high dependence on geometric points and poor tolerance for positioning / navigation map offsets, leading to navigation path errors / jumps. In addition, this approach lacks road information and is not suitable for complex topological scenarios such as road forks, intersections, and main and auxiliary roads, which can easily result in unstable matching.

[0061] Neither of the above two solutions can meet the safety requirements of navigation or autonomous driving. Therefore, the navigation path generation technology still needs further optimization and improvement.

[0062] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. First, a scenario in which the navigation path generation method provided by the embodiments of this disclosure is applied will be introduced. As shown in Figure 1, this is a schematic diagram of an application scenario to which the embodiments of this disclosure are applicable. In this application scenario, an autonomous vehicle 100 may be included.

[0063] The autonomous vehicle 100 includes a sensor system 110, a perception and planning system 111, a control system 112, a positioning system 113, a user interface system 114, and a communication system 115. The sensor system 110 may be one or more sensors, such as lidar, cameras, millimeter-wave radar, etc., used to acquire images of the environment surrounding the autonomous vehicle. Some or all of the functions of the autonomous vehicle 100 may be controlled or managed by the perception and planning system 110, especially when operating in autonomous driving mode. The perception and planning system 111 includes the necessary hardware (e.g., processor, memory, storage devices) and software (e.g., operating system, applications, etc.) to receive information from the sensor system 110, control system 112, positioning system 113, user interface system 114, and / or communication system 115, process the received information, determine a navigation path from a starting point to a destination, and then autonomously drive the vehicle 100 based on the navigation path information. Alternatively, the perception and planning system 111 may be integrated with the control system 112. The perception and planning system 111 includes a processor, which can be an electronic device, such as a processor in an in-vehicle processing unit like a vehicle infotainment system, domain controller, mobile data center (MDC), or vehicle computer, or a conventional chip such as a central processing unit (CPU), graphics processing unit (GPU), microcontroller unit (MCU), or AI (Artificial Intelligence) processor. The controller 112 is used to control the actuators in the vehicle to perform corresponding actions. The control system 112 includes, but is not limited to, a steering unit, a throttle unit (also known as an acceleration unit), and a braking unit. The steering unit is used to adjust the vehicle's direction or forward trajectory. The throttle unit is used to control the speed of the electric motor or engine, which in turn controls the vehicle's speed and acceleration. The braking unit decelerates the vehicle by providing friction to slow down the wheels or tires. The positioning system 113 can receive data from an inertial measurement unit (IMU), a wheel speed sensor (WSS), a global navigation satellite system (GNSS) antenna, a global positioning system (GPS), and / or a satellite antenna to determine the current position of the autonomous vehicle 100 and manage any data related to the user's trip or route.User interface system 114 may be part of a peripheral device implemented within vehicle 100, including, for example, a keyboard, touchscreen display, microphone, and speaker. A passenger can specify the start and destination of their trip, for example, via user interface system 114. Communication system 115 allows communication between autonomous vehicle 100 and external systems such as devices, sensors, other vehicles, and servers. For example, communication system 115 can communicate wirelessly with one or more devices directly, or wirelessly via a communication network, such as communicating with a server over a network. Communication system 115 can use any cellular communication network or wireless local area network (WLAN), for example, using WiFi, to communicate with another component or system. Communication system 115 can communicate directly with devices (e.g., passenger's mobile device, display device, speakers within vehicle 100), for example, using infrared links, Bluetooth, etc.

[0064] Components 110 to 115 may be communicatively connected to each other via interconnects, buses, networks, or combinations thereof. For example, components 110 to 115 may be communicatively connected to each other via a Controller Area Network (CAN) bus. For example, components 110 to 115 may also be communicatively connected to each other via an in-vehicle Ethernet network, and this disclosure is not limited thereto.

[0065] For example, a passenger can specify the start and destination of their trip via user interface system 114. Perception and planning system 111 acquires trip-related data. For example, perception and planning system 111 acquires map navigation data such as location and navigation route. Perception and planning system 111 can acquire perception data from sensor system 110, encode and process the perception data and map navigation data to generate a navigation path, and then autonomously drive vehicle 100 based on the navigation path so that autonomous vehicle 100 travels according to the generated navigation path.

[0066] This paper introduces another scenario in which the navigation path generation method provided by this disclosure is applied, as shown in Figure 2, which is a schematic diagram of another application scenario to which this disclosure is applicable. In this scenario, a client 201 and a server 202 may be included.

[0067] Client 201 is a program running on an electronic device that provides services to the user. In this embodiment of the disclosure, client 201 has navigation functions, such as map navigation applications, travel applications, etc. The electronic device may be a vehicle, a mobile robot, or other mobile electronic device, and this disclosure is not limited thereto.

[0068] The server 202 runs on a server or server cluster, such as a network or cloud server, application server, map server, traffic information server, location server, data analysis server, content server, backend server, or a combination thereof. For example, the client 201 acquires sensor data from an electronic device, receives map navigation data from the server 202, and uses a navigation path generation method provided in this embodiment to generate a navigation path so that the electronic device containing the client 201 travels according to the generated navigation path.

[0069] For example, server 202 receives perception data from client, acquires map navigation data, and generates a navigation path using a navigation path generation method provided in this embodiment. Server 202 then sends the navigation path data to client 201. Client 201 receives the navigation path data, enabling the electronic device containing client 201 to travel according to the generated navigation path.

[0070] Figures 3A and 3B show another scenario illustrating the application of the navigation path generation method provided in this embodiment of the present disclosure.

[0071] This scenario uses a ramp area as an example. Figure 3A shows the globally planned map navigation route for vehicle A within a certain range. The map navigation route is m1→m2→m3 along the main road on the left, and the non-map navigation route branch is m4. Map navigation routes may have geometric deviations and angular offsets, posing significant safety hazards. They are generally not suitable for direct use as local path planning in downstream autonomous driving processes (such as planning and control), and cannot meet the route planning requirements of autonomous vehicles. Therefore, obtaining a navigation route consistent with the vehicle's perception of its surrounding environment is extremely important for the development of autonomous driving technology. It should be understood that navigation routes can be road-level, lane-level, or path-level (connection of road segments or lane segments) and combinations thereof. Figure 3B shows the roads and lanes of the navigation and non-navigation paths under the vehicle system. ABCDEF constitutes the road area, ABCD constitutes road R1, BEFG constitutes road R2, and GHIC constitutes road R3. R1 and R2 are roads used for autonomous driving navigation routes, and R3 is a road for non-navigation paths. abc, def, ghi, jkm, and jln are lane centerlines, constituting the road lanes. abc, def, and ghi are lanes used for navigation paths downstream of autonomous driving, while jkm and jln are lanes not used for navigation paths. The navigation path generation method provided in this disclosure can generate navigation paths consistent with the vehicle's perception information in real time. It can be applied to all road segments, including but not limited to ordinary roads, forks in the road, intersections, entrances and exits of main and auxiliary roads, ramps, and multi-fork roads, providing safe and reliable planning and control information for the vehicle's autonomous driving.

[0072] Referring to the figures below, a method for generating a navigation path provided by an embodiment of this disclosure will be described in detail.

[0073] Figure 4 shows a schematic diagram of the structure of the navigation path generation method provided in this embodiment. The structure consists of three parts: a first neural network, a second neural network, and a third neural network. Sensors (e.g., cameras and radar) input sensing data into the first neural network, which extracts features from the sensing data to obtain sensing features. Map navigation data, after preprocessing, is input into the second neural network to obtain road features. The sensing features and road features are then input into the third neural network for processing to obtain the navigation path.

[0074] Figure 5 shows a flowchart of a navigation path generation method provided in this embodiment of the present disclosure. The implementation process of this method mainly includes steps S510-S530, which are described below:

[0075] S510: Acquire sensing data from at least one sensor, and extract features from the sensing data to obtain sensing features.

[0076] The perceived data originates from at least one sensor. For example, the sensor could be a vision sensor, such as a camera, in which case the acquired perceived data is image data. Alternatively, the sensor could be a lidar sensor, in which case the acquired perceived data is a lidar point cloud. Or, the sensor could be a millimeter-wave radar sensor, in which case the acquired data is millimeter-wave radar detection points. The acquired perceived data can come from a combination of one or more sensors; that is, the perceived data can be a combination of one or more types of data, such as image data from one or more cameras, lidar point clouds from one or more lidar sensors, or at least one millimeter-wave radar sensor. Of course, this disclosure is not limited to these categories.

[0077] Perceptual features can be representations in the perspective view (PV) space, the bird's-eye view (BEV) space, or other data that can represent environmental information; this disclosure is not limited thereto. Furthermore, representations in PV space (i.e., PV features) can be transformed into representations in BEV space (i.e., BEV features). It should be understood that the transformation from PV features to BEV features is not a mandatory operation; PV features can also serve as the basis for subsequent processing.

[0078] Step S510 can be implemented by a first neural network. The first neural network can be a residual network (ResNet), a visual transformer (ViT), an Alex convolutional neural network (AlexNet), a visual geometry group (VGG), or other neural networks, and this disclosure is not limited thereto.

[0079] In some embodiments, the first neural network extracts features from the perceived data to obtain perceived features. In one example, the first neural network extracts features from the perceived data and transforms them into the BEV space to obtain BEV features. In another example, the first neural network extracts features from the perceived data to obtain PV features.

[0080] In some other embodiments, the perceived data includes multiple features. The first neural network extracts features from the perceived data and fuses these multiple features to obtain perceived features. Feature fusion can be achieved through feature addition, feature concatenation, attention mechanisms, etc., and this disclosure is not limited thereto. In one example, the first neural network extracts features from the perceived data, fuses multiple features, and then transforms them into the BEV space to obtain perceived features. For example, features can be extracted from perceived data from multiple cameras, and then the data features from the multiple cameras can be fused and transformed into the BEV space to obtain perceived features. In another example, the first neural network extracts features from the perceived data, transforms them into the BEV space respectively, and then fuses the multiple BEV features to obtain perceived features. For example, features can be extracted from perceived data from multiple cameras, transformed into the BEV space to obtain BEV features, and then the BEV features from the multiple cameras can be fused to obtain perceived features. As another example, features can be extracted from image data from one or more cameras and point cloud data from a LiDAR to obtain image features and point cloud features, and then the image features and point cloud features can be fused in the BEV space to obtain perceived features.

[0081] It should be understood that the embodiments listed above are merely exemplary descriptions, and this disclosure is not limited to the methods for determining perceived features. The above description only provides some possible implementation methods.

[0082] S520: Acquire map navigation data and encode the map navigation data to obtain road features. The map navigation data includes navigation path attribute data, which is used to indicate whether a road is a navigation path. The road features include the representation of the navigation path attribute data.

[0083] Map navigation data includes road geometry data, road attribute data, and road topology data. Road geometry data describes the geometric information of a road. It includes road geometric point data and road geometric relationship data. Road geometric point data describes the geometric points of a road. Road geometric relationship data describes the relative geometric relationships between roads. The basic unit of a map is the road segment, and road geometry data is the geometric information of a road segment. Road geometry includes the geometric points of a road segment and the relative geometric relationships between road segments. A road segment's geometric points are generally composed of at least one coordinate point in sequence. Road geometric point data can include the coordinates of road points, the starting coordinates of the road segment, the ending coordinates of the road segment, the direction of the road segment, etc., or other geometric information calculated based on coordinates; this disclosure is not limited to these. Road geometric relationship data can be obtained by calculating the relative distances and relative angles between road geometric points. Road attribute data includes, but is not limited to, navigation path attribute data and road type data. Navigation path attribute data indicates whether the current road is a navigation path. Specifically, navigation path attribute data includes, but is not limited to, whether the current road is a navigation path, navigation action information (left or right turn, which fork in the road, etc.), road end connection type (intersection, merging, merging out, etc.), lane-level turning information within the road, lane number information, and lane-level turning information within the road that leads to the navigation path. Road type data includes, but is not limited to, main and auxiliary roads, road level, whether it is a ramp, whether it is an elevated road, whether it is a road within an intersection, whether it is an entrance road, and whether it is controlled by traffic lights. Road topology data is used to represent the connection relationships between road segments, the directional relationships between roads, and the hierarchical relationships between roads.

[0084] Map navigation data can be obtained from a map. Specifically, obtaining map navigation data may include retrieving map navigation data from a map based on location information and the user-defined destination. The map can be a standard definition map (SD MAP) or a lane-level navigation map. Optionally, obtaining map navigation data may also include retrieving map navigation routes from the map based on location information and the user-defined destination, and preprocessing the map navigation routes to obtain vehicle-specific map navigation data. Preprocessing involves converting the map navigation routes from a global system to a vehicle-specific system, including global-to-vehicle system conversion and vehicle-specific range cropping.

[0085] Optionally, step 520 can be implemented by a second neural network. The second neural network can be a fully connected network, a graph neural network, a convolutional neural network, an attention mechanism, or other network forms, and this disclosure is not limited thereto. The second neural network can encode map navigation data to obtain road features.

[0086] The following will describe in detail a specific implementation of the navigation path generation method of this embodiment based on the encoding structure. Figure 6 shows a schematic diagram of the map navigation data encoding structure provided in this embodiment. Road geometry data and road attribute data for each road are obtained from the map navigation data. The road geometry data and road attribute data for each road are encoded to obtain basic road features. Specifically, the road geometric point data in the road geometry data and the navigation path attribute data and road type data in the road attribute data are encoded to obtain basic road features, wherein the basic road features include the representation of the navigation path attribute data. Road topology data and road geometric relationship data in the road geometry data are obtained from the map navigation data. The road topology data and road geometric relationship data are encoded to obtain road relationship features. Encoding the road topology data and road geometric relationship data allows for the interaction of information between different roads in the map, enabling each road code to carry information distinguishing and connecting it to other map roads. Finally, the basic road features and road relationship features are fused to obtain the road features for each road, which include the representation of the navigation path attribute data. Feature fusion can employ feature addition, feature concatenation, attention mechanisms, etc., and this disclosure is not limited to these methods.

[0087] Therefore, road features not only include their own geometric and attribute information, but also the spatial and topological relationships between the road and other roads, as well as information on the similarities and differences in attributes. Road features containing rich information can be better correlated with real-time perception features, and have the ability to resist map geometric deviations that exist in real-world applications.

[0088] Furthermore, when the map is a lane-level map, that is, when the map navigation data includes lane data, the road geometry data in the map navigation data includes lane geometry data, the road attribute data includes lane attribute data, and the road topology data includes lane topology data. Lane geometry data includes lane geometric point data and lane geometric relationship data. Lane attribute data includes navigation path attribute data (e.g., whether it is the lane to be driven in or the recommended lane on the navigation path), the left-to-right order of lanes, and the lane turning type, etc., representing lane attributes. It should be understood that a road includes lanes. Encoding the road geometry data and road attribute data includes encoding the lane geometry data and lane attribute data to obtain basic road features, where basic road features include basic lane features. Encoding the road geometry data and road topology data includes encoding the lane geometric relationship data and lane topology data to obtain road relationship features, where road relationship features include lane relationship features. The basic road features and road relationship features are fused to obtain road features, where road features also include lane features.

[0089] As described above, encoding the map navigation data of lane-level maps includes encoding information such as the geometric points of the lanes, navigation path attributes, lane attributes, and road attributes where the lanes are located. Lane topology information and relative geometric relationships between lanes are also introduced into the feature fusion between lanes to enhance the representation ability of lane features.

[0090] Furthermore, when the map also includes elements such as intersections, sidewalks, and stop lines, the map navigation data also includes element data. Element data can be of different types; specifically, it can include intersection data, sidewalk data, stop line data, and other data belonging to intersections, as well as guide line data, no-stopping lines, and other data belonging to roads. Road geometry data also includes geometric point data and geometric relationship data of the same type of elements, as well as geometric point data and geometric relationship data of different types of elements. Road topology data also includes topology data of the same type of elements and topology data of different types of elements. Encoding road geometry data and road attribute data includes encoding road geometry data, road attribute data, and element data to obtain basic road features. Road geometry data includes lane geometry data, and road attribute data includes lane attribute data. Encoding road geometry data and road topology data includes encoding lane geometric relationship data, lane topology data, geometric relationship data of the same type of elements, topology data of the same type of elements, geometric relationship data of different types of elements, and topology data of different elements to obtain road relationship features. The basic road features and road relationship features are fused to obtain road features, which include lane features and element features. Element features are representations of element data.

[0091] As a result, incorporating richer elements from the map into road coding further enhances the ability to represent road features. Of course, this disclosure is not limited to this.

[0092] In some embodiments, road geometry data and road attribute data include point-level data, lane-level data, and road-level data, or point-level data and road-level data. Point-level data refers to the data of sampling points to be encoded, lane-level data refers to the data of lanes to be encoded, and road-level data refers to the data of roads to be encoded. That is, road geometry data and road attribute data can be divided into three levels: point-level data, lane-level data, and road-level data, or two levels: point-level data and road-level data. Sampling points are obtained by sampling the road. Point-level data, lane-level data, and road-level data can be set according to requirements, and this disclosure is not limited thereto. For example, Table 1 shows the point-level data, lane-level data, and road-level data. Point-level data includes data such as the x-coordinate of a road point, the y-coordinate of a road point, and the change in x-coordinate between the road point and the previous point. Road-level data includes data such as whether the current road is a navigation path, whether the road is a ramp, the type of road end connection, and the road grade. Lane-level data includes the order of lanes on the road from left to right, the turning type of the lanes on the road, and whether the lane should be used or recommended on the navigation path.

[0093] Table 1

[0094] It should be understood that the above data is merely an exemplary description, and other road coding data may be included in other embodiments, which are not limited thereto.

[0095] In some embodiments, road geometric data and road attribute data are encoded to obtain basic road features, including: encoding point-level data, lane-level data, and road-level data to obtain features of point-level data (i.e., point-level features), features of lane-level data (i.e., lane-level features), and features of road-level data (i.e., road-level features); and fusing the features of point-level data, lane-level data, and road-level data to obtain basic road features. When encoding geometric data at each level, geometric numerical encoding can be used; preferably, the geometric data encoding can also be normalized. When encoding attribute data at each level, attribute data can be one-hot encoded or directly numerically encoded. Of course, this disclosure is not limited to these methods. Feature fusion can employ feature addition, feature concatenation, attention mechanisms, etc.

[0096] Figure 7 illustrates a scenario of an encoding method provided in this embodiment. Taking this scenario as an example, one implementation method for obtaining road sampling points is introduced. As shown in Figure 7, the navigation map consists of five road segments: m0, m1, m2, m3, and m4. Each road segment is composed of... The original map contains multiple road segments, each with a different number of points. This increases the complexity of neural network processing. Therefore, to facilitate parallel processing by the neural network, an equal number of road points need to be generated for each road segment during the encoding process. Assuming the number of road points in each road segment is uniformly set to N, for... The original road polyline, consisting of points, is divided into N-1 equal parts to obtain N road sampling points on a single road segment. The polyline represents the road's geometry and is typically represented by line segments connecting multiple points. Alternatively, sampling can be performed at fixed intervals of 'a' meters (e.g., a = 1). After obtaining the sampling points, for each road sampling point p... i (i = 0, ..., N-1), perform feature encoding.

[0097] Taking the scenario shown in Figure 7 as an example, the point-level feature representation of sampling point p2 on road segment m0 ​​is as follows: The lane-level characteristics of lane L2 in road segment m0 ​​are characterized as follows: The road-level characteristics of road segment m0 ​​are characterized as follows: Point-level features are represented as (0≤i≤K, where i is an integer), after combination, a K×D dimensional matrix is ​​formed, where K is the number of road segments in the current map, M is the maximum number of lanes per road segment, N is the maximum number of sampling points per road segment, and D is the dimension of the feature. Lane-level features are represented as follows: After combination, a K×D dimensional matrix is ​​formed. Road-level features are represented as follows: After combination, a K×D dimensional matrix is ​​formed. After obtaining the features at three levels, the features at these three levels are fused to obtain the basic road features. Of course, this disclosure is not limited to this.

[0098] In other embodiments, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data, lane-level data, and road-level data to obtain features of point-level data, lane-level data, and road-level data; and concatenating the features of point-level data, lane-level data, and road-level data to obtain basic road features. Specifically, the features of lane-level data and road-level data can be concatenated onto the features of point-level data.

[0099] Taking the scenario shown in Figure 7 as an example, the point-level feature representation of sampling point p2 on road segment m0 ​​is as follows: The lane-level characteristics of lane L0 in road segment m0 ​​are characterized as follows: Lane-level characteristics of lane L1 are characterized as follows Lane-level characteristics of lane L2 are characterized as follows The road-level characteristics of road segment m0 ​​are characterized as follows: By concatenating lane-level and road-level features onto point-level features, the comprehensive point-level feature representation of sampling point p2 (m0) is obtained as [1,0.2,1,0,0.5,0.0,0,0,0.1,1,0,0.2,3,1,1,0,2,5,0,2,0]. That is, road segment m... i sampling point p j Point-level comprehensive features are characterized as (0≤i≤K, 0≤j≤N, where i and j are integers), where K is the number of road segments in the current map, and N is the maximum number of sampling points per road segment. Therefore, the point-level integrated feature dimension is D. PLR =D P +M×D L +D R The comprehensive feature dimension of a road segment is D. TR =N×D PLR D TR After point-level local fusion, a feature dimension D is formed. Here, M is the maximum number of lanes in each road segment, and D... P It is the point feature dimension, D L It is the lane feature dimension, D R This refers to the road feature dimension. For K road segments, the point-level comprehensive feature representation is as follows: After combination, a K×D dimensional matrix is ​​formed. After obtaining the point-level comprehensive feature representation, this feature is fused to obtain the basic road features. Of course, this disclosure is not limited to this.

[0100] In some other embodiments, road geometry data and road attribute data are encoded to obtain basic road features, including: encoding point-level data and road-level data to obtain features of point-level data and features of road-level data; and splicing or fusing the features of point-level data and features of road-level data to obtain basic road features.

[0101] As described above, by encoding the geometric and attribute information of roads in a hierarchical manner, the comprehensive and rich expression of road information is ensured, and the ability to represent road features is enhanced.

[0102] In some embodiments, encoding road geometry data and road topology data includes encoding road geometric relationship data and road topology data. Optionally, an adjacency matrix is ​​used to represent the topology data, and a matrix is ​​used to represent the road geometric relationship data. The matrix elements are relational values. Taking the scenario shown in Figure 7 as an example, for navigation maps m0, m1, m2, m3, and m4, road segments m0, m2, m3, and m4 all belong to navigation paths, road segment m1 belongs to a non-navigation path, and both m1 and m2 branch off from m0, with m1 located to the left of m2. To characterize the road topology relationships between road segments m0, m1, m2, m3, and m4, the topology data is encoded as an N×N adjacency matrix, with matrix elements taking values ​​of [0,1], and N being a positive integer. Road geometric relationship data is obtained based on the relative distances and relative angles between road geometric points, and this data is encoded as an N×N×D matrix, where D is the number of relative geometric relationships, matrix elements are relational values, and N and D are positive integers. The adjacency matrix representing topological data and the matrix representing road geometric relationships are fused to obtain road relationship features. Feature fusion can employ fully connected layer transformations, weighting, modulation, GNNs, or attention mechanisms, etc. Of course, this disclosure is not limited to these methods.

[0103] S530: Based on perception features and road features, a navigation path is obtained.

[0104] Step S530 can be implemented by a third neural network. The third neural network can adopt architectures such as Transformer, improved Transformer, and Mamba, and can process perception features and road features to output a navigation path.

[0105] In some embodiments, perceived features and road features can be fused to obtain fused features. A third neural network decodes the fused features to obtain road information for multiple roads. The road information includes geometric information, attribute information, and navigation path information. Geometric information describes the ordered coordinate points of the lane centerline and the corresponding left and right lane lines, as well as the ordered coordinate points of the road centerline and the corresponding left and right boundary lines. Attribute information describes the attributes of the lane centerline, left and right lane lines, road centerline, and road left and right boundary lines. For example, lane centerline attribute information includes lane type (motor vehicle, non-motor vehicle, bus, etc.), turning attributes (straight, left turn, right turn, U-turn, and their combinations), and topological relationship attributes (such as the sequential relationship between lanes, the left and right relationship between lanes, etc.). The attributes of the left and right lane lines include solid / dashed / dashed-solid lines, color, and left and right lane line types (solid lines, curbs, etc.). Navigation path information indicates whether the current road is a navigation path. It should be understood that navigation path information is obtained by encoding, fusing, and decoding perceived data, road attribute data, road geometric data, and road topological data. Road attribute data includes navigation attribute data. The third neural network outputs road information for multiple roads, extracts navigation path information from each road's road information, and tracks this navigation path information to obtain a navigation path. The tracking method can employ voting, Bayesian estimation, or other methods, and this disclosure is not limited to these. Taking voting as an example, the third neural network outputs road information for multiple roads. When the navigation path information in the road information indicates that the current road is a navigation path, points are added to the current road; when the navigation path information in the road information indicates that the current road is not a navigation path, points are deducted from the current road. The total score for each road is calculated, and roads with a total score exceeding a threshold are determined as navigation paths. This threshold can be set according to requirements, and this disclosure is not limited to this.

[0106] In other embodiments, road features and road query features (Query) can be fused first to obtain basic fused features, and then further fused with perceptual features to obtain fused features. Here, the road query feature (Query) is the feature of interest in the road. Thus, the fused features can include richer and more accurate road information. A third neural network decodes the fused features to obtain road information for multiple roads. Road information includes geometric information, attribute information, and navigation path information, indicating whether the current road is a navigation path. The third neural network outputs road information for multiple roads, obtains navigation path information from the road information of each road, and tracks the navigation path information in the road information of each road to obtain the navigation path.

[0107] In some other embodiments, the third neural network can first decode the perceptual features to obtain the embeddings of the perceptual features. The embeddings are then fused with road features to output road information for multiple roads. This road information includes geometric information, attribute information, and navigation path information, indicating whether the current road is a navigation path. The third neural network outputs road information for multiple roads, obtains navigation path information from the road information of each road, and tracks the navigation path information in the road information of each road to obtain the navigation path. In some other embodiments, the perceptual features and road features can be fused to obtain fused features. The third neural network decodes the fused features to generate the navigation path. The navigation path includes geometric information and Logits, where Logits are the raw information output by the third neural network. The navigation path can be generated in a rasterized form or in a regression coordinate form. When the navigation path is generated in a regression coordinate form, it can be a single navigation path guide line or a form with left and right boundary lines. A navigation path with left and right boundary lines can facilitate providing the driving area of ​​the navigation path. Therefore, the third neural network can directly output navigation paths without being limited by road information output, enabling autonomous vehicles to still output navigation paths in scenarios without road perception, such as intersections, thus improving the safety of path planning. It should be understood that, in addition to outputting the geometric information and logits of the navigation path, the third neural network can also output multiple road information, including geometric and attribute information. Furthermore, the navigation path generated by the third neural network can be further fused with road features to obtain the final navigation path, thereby enhancing the consistency between the navigation path and the road.

[0108] The navigation path generation method provided in this disclosure adopts a method of interaction between perceived data and road features of map navigation data at the original feature level, so that the perceived features carry rich navigation path information, realizing navigation path generation. It solves the problem of high-precision map dependence and solves the problem of inaccurate matching when traditional methods use rules to explicitly match the perceived road aggregation points with the geometric points of the navigation map. It is also applicable to complex road topology scenarios, has strong versatility and generalization, and improves the accuracy and security of planning.

[0109] Another embodiment of this disclosure provides a navigation path generation apparatus, which can be implemented by a software system, a hardware device, or a combination of a software system and a hardware device.

[0110] Figure 8 shows a schematic diagram of the structure of a navigation path generation device provided in an embodiment of this disclosure.

[0111] Optionally, the navigation path generation device is used to perform the steps S510-S530 shown in FIG5. Specifically, it can be: a first acquisition module 810, configured to acquire perception data from at least one sensor; a first neural network 820, configured to extract features from the perception data to obtain perception features; a second acquisition module 830, configured to acquire map navigation data; a second neural network 840, configured to encode the map navigation data to obtain road features, wherein the map navigation data includes navigation path attribute data, the navigation path attribute data is used to indicate whether a road is a navigation path, and the road features include a representation of the navigation path attribute data; and a third neural network 850, configured to obtain a navigation path based on the perception features and the road features.

[0112] In some embodiments, the map navigation data includes road geometry data, road attribute data, and road topology data. The second neural network 840 is configured to encode the map navigation data to obtain road features, including: encoding the road geometry data and road attribute data to obtain basic road features, wherein the road attribute data includes navigation path attributes; encoding the road geometry data and road topology data to obtain road relationship features; and fusing the basic road features and road relationship features to obtain road features.

[0113] Furthermore, the road geometry data and road attribute data include point-level data, lane-level data, and road-level data. The second neural network 840 is configured to encode the road geometry data and road attribute data to obtain basic road features, including: encoding the point-level data, lane-level data, and road-level data to obtain features of the point-level data, lane-level data, and road-level data; and concatenating or fusing the features of the point-level data, lane-level data, and road-level data to obtain the basic road features.

[0114] Optionally, the road geometry data and road attribute data include point-level data and road-level data. The second neural network 840 is configured to encode the road geometry data and road attribute data to obtain basic road features, including: encoding the point-level data and road-level data to obtain features of the point-level data and features of the road-level data; and concatenating or fusing the features of the point-level data and the features of the road-level data to obtain basic road features.

[0115] In some embodiments, the map navigation data further includes lane data, and the road features obtained by encoding the map navigation data include: wherein the road features further include lane features.

[0116] Furthermore, map navigation data also includes element data. Encoding the map navigation data yields road features, which include element features.

[0117] In some embodiments, the third neural network 850 is configured to: obtain a navigation path based on perception features and road features, including: fusing the perception features and road features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain a navigation path.

[0118] In some other embodiments, the third neural network 850 is configured to: obtain a navigation path based on perceptual features and road features, including: fusing road features and road query features to obtain basic fused features; fusing basic fused features and perceptual features to obtain fused features; decoding the fused features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain a navigation path.

[0119] In some other embodiments, the third neural network 850 is configured to: obtain a navigation path based on perception features and road features, including: decoding the perception features to obtain an embedding of the perception features; fusing the embedding of the perception features and the road features to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether a road is a navigation path; acquiring the navigation path information for each road; and tracking the navigation path information for each road to obtain a navigation path.

[0120] In some other embodiments, the third neural network 850 is configured to: obtain a navigation path based on perception features and road features, including: fusing the perception features and road features to obtain fused features; and decoding the fused features to obtain the navigation path.

[0121] It should be noted that the specific implementation of each functional module in this embodiment can be found in the description of the above method embodiment, and will not be repeated in this embodiment.

[0122] According to yet another embodiment of this disclosure, an electronic device is also provided for implementing the method according to an embodiment of this disclosure. FIG9 shows a schematic structural diagram of an electronic device 900 according to an embodiment of this disclosure. The electronic device 900 includes: a processor 910 and a memory 920.

[0123] It should be understood that the electronic device 900 shown in Figure 9 may also include a communication interface 930, which can be used to communicate with other devices.

[0124] The processor 910 can be connected to the memory 920. The memory 920 can be used to store the program code and data. Therefore, the memory 920 can be a storage unit inside the processor 910, an external storage unit independent of the processor 910, or a component that includes both the storage unit inside the processor 910 and the external storage unit independent of the processor 910.

[0125] Optionally, the electronic device 900 may also include a bus. The memory 920 and communication interface 930 can be connected to the processor 910 via the bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0126] It should be understood that in the embodiments of this disclosure, the processor 910 may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a graphics processing unit (GPU), a microcontroller unit (MCU), an AI (Artificial Intelligence) processor, or any conventional processor. Alternatively, the processor 910 may employ one or more integrated circuits to execute related programs to implement the technical solutions provided in the embodiments of this disclosure.

[0127] The memory 920 may include read-only memory and random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include non-volatile random access memory. For example, the processor 910 may also store device type information.

[0128] When the electronic device 900 is running, the processor 910 executes the computer execution instructions in the memory 920 to perform the operation steps of the above method.

[0129] It should be understood that the electronic device 900 according to the disclosed embodiments can correspond to the corresponding subject executing the methods according to the various embodiments of this disclosure, and the above and other operations and / or functions of each module in the electronic device 900 are respectively for implementing the corresponding processes of the methods of this embodiment. For the sake of brevity, they will not be described in detail here.

[0130] This disclosure also provides another electronic device, including a processor and an interface circuit. The processor accesses a memory through the interface circuit. The memory stores program instructions, which, when executed by the processor, cause the processor to perform the method of the embodiment corresponding to FIG5. Additionally, the electronic device may also include a communication interface, a bus, etc., as described in the embodiment shown in FIG9, and will not be repeated here. For example, the interface circuit can be a CAN bus or a LIN bus.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0132] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0136] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs a method for generating a navigation path, the method including at least one of the schemes described in the foregoing embodiments.

[0138] The computer storage medium of this disclosure can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0139] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0140] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0141] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] This disclosure also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a method according to an embodiment of this disclosure.

[0143] This disclosure can be applied not only to autonomous driving but also to robotics and embodied intelligence. When a robot needs to follow a navigation route, the solution provided in this disclosure can utilize the perception data and map navigation data output by the robot's environmental sensor module to generate a navigation path in the real environment in real time.

[0144] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0146] Note that the above are merely preferred embodiments and the technical principles employed in this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this disclosure, all of which fall within the scope of protection of this disclosure.

Claims

1. A method for generating a navigation path, characterized in that, include: Acquire sensing data from at least one sensor, and extract features from the sensing data to obtain sensing features; Acquire map navigation data and encode the map navigation data to obtain road features, wherein the map navigation data includes navigation path attribute data, the navigation path attribute data is used to indicate whether a road is a navigation path, and the road features include the representation of the navigation path attribute data; The navigation path is obtained based on the perceived features and the road features.

2. The method as described in claim 1, characterized in that, include: The map navigation data includes road geometry data, road attribute data, and road topology data.

3. The method as described in claim 2, characterized in that, The process of encoding the map navigation data to obtain road features includes: The road geometry data and the road attribute data are encoded to obtain basic road features, wherein the road attribute data includes navigation path attributes; The road geometry data and the road topology data are encoded to obtain road relationship features; The basic road features and the road relationship features are fused to obtain the road features.

4. The method as described in claim 2 or 3, characterized in that, include: The road geometry data and the road attribute data include point-level data, lane-level data, and road-level data.

5. The method as described in claim 2 or 3, characterized in that, include: The road geometry data and the road attribute data include the point-level data and the road-level data.

6. The method as described in claim 4, characterized in that, The process of encoding the road geometry data and the road attribute data to obtain basic road features includes: The point-level data, lane-level data, and road-level data are encoded to obtain the features of the point-level data, the lane-level data, and the road-level data; The features of the point-level data, the lane-level data, and the road-level data are spliced ​​or fused to obtain the basic road features.

7. The method as described in claim 5, characterized in that, The process of encoding the road geometry data and the road attribute data to obtain basic road features includes: The point-level data and the road-level data are encoded to obtain the features of the point-level data and the features of the road-level data; The features of the point-level data and the features of the road-level data are spliced ​​or fused to obtain the basic road features.

8. The method according to any one of claims 1-7, characterized in that, The map navigation data also includes lane data. The map navigation data is encoded to obtain road features, including lane features.

9. The method as described in claim 8, characterized in that, The map navigation data also includes element data. The map navigation data is encoded to obtain road features, which include element features.

10. The method according to any one of claims 1-9, characterized in that, The process of obtaining the navigation path based on the perceived features and the road features includes: The perceived features and the road features are fused to obtain fused features; The fused features are decoded to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether the road is the navigation path; Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

11. The method according to any one of claims 1-9, characterized in that, The process of obtaining the navigation path based on the perceived features and the road features includes: The road features and road query features are fused to obtain basic fused features; The basic fusion features and the perceptual features are fused to obtain fusion features; The fused features are decoded to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether the road is the navigation path; Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

12. The method according to any one of claims 1-9, characterized in that, The process of obtaining the navigation path based on the perceived features and the road features includes: The perceptual features are decoded to obtain the embedding of the perceptual features; The embedding of the perceived features and the road features are fused to obtain road information for multiple roads. The road information includes geometric information, attribute information, and navigation path information. The navigation path information indicates whether the road is the navigation path. Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

13. The method according to any one of claims 1-9, characterized in that, The process of obtaining the navigation path based on the perceived features and the road features includes: The perceived features and the road features are fused to obtain fused features; The fused features are decoded to obtain the navigation path.

14. A navigation path generation device, characterized in that, include: The first acquisition module is configured to acquire sensing data from at least one sensor; The first neural network is configured to: extract features from the perceived data to obtain perceived features; The second acquisition module is configured to acquire map navigation data. The second neural network is configured to encode the map navigation data to obtain road features, wherein the map navigation data includes navigation path attribute data, the navigation path attribute data is used to indicate whether a road is a navigation path, and the road features include a representation of the navigation path attribute data; The third neural network is configured to obtain the navigation path based on the perceived features and the road features.

15. The apparatus as claimed in claim 14, characterized in that, The map navigation data includes road geometry data, road attribute data, and road topology data.

16. The apparatus as claimed in claim 15, characterized in that, The second neural network, configured to encode the map navigation data to obtain road features, includes: The road geometry data and the road attribute data are encoded to obtain basic road features, wherein the road attribute data includes navigation path attributes; The road geometry data and the road topology data are encoded to obtain road relationship features; The basic road features and the road relationship features are fused to obtain the road features.

17. The apparatus as claimed in claim 15 or 16, characterized in that, include: The road geometry data and the road attribute data include point-level data, lane-level data, and road-level data.

18. The apparatus as claimed in claim 15 or 16, characterized in that, include: The road geometry data and the road attribute data include the point-level data and the road-level data.

19. The apparatus as claimed in claim 17, characterized in that, The second neural network, configured to encode the map navigation data to obtain road features, includes: The point-level data, lane-level data, and road-level data are encoded to obtain the features of the point-level data, the lane-level data, and the road-level data; The basic road features are obtained by splicing or fusing the features of the point-level data, the lane-level data, and the road-level data.

20. The apparatus as claimed in claim 18, characterized in that, The second neural network, configured to encode the map navigation data to obtain road features, includes: The point-level data and the road-level data are encoded to obtain the features of the point-level data and the features of the road-level data; The features of the point-level data and the features of the road-level data are spliced ​​or fused to obtain the basic road features.

21. The apparatus according to any one of claims 14-20, characterized in that, The map navigation data also includes lane data. The map navigation data is encoded to obtain road features, including lane features.

22. The apparatus as claimed in claim 21, characterized in that, The map navigation data also includes element data. The map navigation data is encoded to obtain road features, which include element features.

23. The apparatus as claimed in any one of claims 14-22, characterized in that, The third neural network is configured to obtain the navigation path based on the perceived features and the road features, including: The perceived features and the road features are fused to obtain fused features; The fused features are decoded to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether the road is the navigation path; Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

24. The apparatus according to any one of claims 14-22, characterized in that, The third neural network is configured to obtain the navigation path based on the perceived features and the road features, including: The road features and road query features are fused to obtain basic fused features; The basic fusion features and the perceptual features are fused to obtain fusion features; The fused features are decoded to obtain road information for multiple roads, wherein the road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether the road is the navigation path; Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

25. The apparatus according to any one of claims 14-22, characterized in that, The third neural network is configured to obtain the navigation path based on the perceived features and the road features, including: The perceptual features are decoded to obtain the embedding of the perceptual features; The embedding of the perceived features and the road features are fused to obtain road information for multiple roads. The road information includes geometric information, attribute information, and navigation path information, and the navigation path information indicates whether the road is the navigation path. Obtain the navigation path information for each of the roads, track the navigation path information for each of the roads, and obtain the navigation path.

26. The apparatus according to any one of claims 14-22, characterized in that, The third neural network is configured to obtain the navigation path based on the perceived features and the road features, including: The perceived features and the road features are fused to obtain fused features; The fused features are decoded to obtain the navigation path.

27. An electronic device, characterized in that, include: Processor, and memory; The memory stores program instructions that, when executed by the processor, cause the processor to perform the navigation path generation method according to any one of claims 1-13.

28. A vehicle, characterized in that, Includes the vehicle body and the electronic device as described in claim 27.

29. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by a computer, the computer performs the method for generating a navigation path according to any one of claims 1-13.

30. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads from the computer-readable medium and executes the computer instructions, causing the computer device to implement the navigation path generation method as described in any one of claims 1-13.

Images