Road information recognition method, device, electronic equipment, vehicle and medium

Abstract

The present application belongs to the field of artificial intelligence technology, and relates to road recognition technology under this technical field. Specifically, a method for identifying road information is provided, including: receiving road environment data of multiple modes; wherein the road environment includes the environment of the lane area and the environment of the lane-free area. Topological analysis is performed based on the road environment data of multiple modes to obtain the lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes in the road and the connection status of the lanes. The road information of the road is determined based on the lane-level topological connection relationship of the road. Based on the technical solution provided by the present application, not only the topological structure information of the relative position between the lanes where the vehicle is located can be identified in real time, but also the topological structure information of the lane connection can be identified, providing guidance for the subsequent planning and control of the automatic driving route of the vehicle and the prediction of the driving route of other vehicles.

Description

[0001] This application is a divisional application with an application date of October 14, 2021, application number 20211197941.5, and application name “Road information identification method, device, electronic device, vehicle and medium”. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic equipment, vehicle and medium for identifying road information. Background Art

[0003] Autonomous driving technology requires planning and controlling the vehicle route based on the topological structure of the road. Therefore, accurately obtaining the topological structure of the road is extremely important for the development of autonomous driving technology.

[0004] The current technical solutions mainly focus on obtaining the topological structure of roads through high-precision maps. However, when the topological structure of the road changes, the production of high-precision maps is extremely complex, which often leads to untimely updates of the topological structure; in addition, the method of obtaining road information based on high-precision maps relies on high-precision positioning information. When the positioning information is slightly deviated, it will cause serious accidents. In addition, the output of roads in existing technical solutions mostly focuses on semantic-level output, and does not involve the output of lane-level topological connection relationships, especially in laneless areas such as intersections. In such laneless areas, it is even impossible to obtain the semantic-level output of the road lane level, let alone the output of the road lane-level topological connection relationship. This will result in the output road information cannot be directly used to provide driving route guidance for the vehicle, and cannot meet the needs of autonomous driving. Summary of the invention

[0005] In view of the above problems in the prior art, the present application provides a road information recognition method, device, electronic device, vehicle and medium, which can not only obtain the topological structure information between lanes in the road where the vehicle is located in real time, but also obtain the topological structure information of the lane connections in the road, thereby providing guidance for the subsequent planning and control of the autonomous driving route of the vehicle and the prediction of the driving routes of other vehicles.

[0006] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a method for identifying road information, the method comprising: receiving road environment data of multiple modalities, wherein the road environment includes the environment of the lane area and the environment of the lane-free area. Performing topological analysis based on the road environment data of multiple modalities to obtain the lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes in the road and the connection status of the lanes. Determine the road information based on the lane-level topological connection relationship of the road. The road environment data of multiple modalities includes: the original data of the road environment and / or the perception data of the road environment, and the prior data of the road environment.

[0007] From the above, the first aspect of the present application provides a method for identifying road information. First, by receiving road environment data of multiple modes, the source of road environment data is richer and the road state can be more comprehensively described. Secondly, the technical solution provided by this aspect can make the topological connection relationship of the lane level of the road obtained by the analysis by topological analysis of road environment data of multiple modes, and the topological connection relationship of the lane level is closer to the topological connection relationship of the real road, and more accurately describes the topology of the road. The road information determined based on the lane-level topology of the road in the technical solution of the present application can provide guidance for the planning and control of the automatic driving route of the vehicle and the prediction of the driving route of other vehicles. In addition, the road environment of this aspect includes the environment of the lane area and the environment of the non-lane area. Therefore, when performing topological analysis, the topological connection relationship of the lane level in the non-lane area can be analyzed to obtain the topological connection relationship, and further guide the driving route of the unmanned vehicle in an orderly manner.

[0008] As a possible implementation of the first aspect, the lane-level topological connection relationship of the road includes: the topological connection relationship between lanes in the lane area, the topological connection relationship between virtual lanes in the lane-free area, and the topological connection relationship between lanes in the lane area and virtual lanes in the lane-free area.

[0009] From the above, the technical solution based on this aspect can not only obtain the topological connection relationship between lanes in the lane area, but also obtain the topological connection relationship between virtual lanes in the lane-free area, and even obtain the topological connection relationship between lanes in the lane area and virtual lanes in the lane-free area. In this way, the entire road can form a complete lane-level structure, so that even if the driving route of the unmanned vehicle includes a lane-free area, its driving route can still be orderly.

[0010] As a possible implementation of the first aspect, before obtaining the topological connection relationship between virtual lanes in the lane-free area and before obtaining the topological connection relationship between lanes in the lane area and the virtual lanes in the lane-free area, it also includes: determining the virtual lanes in the lane-free area based on a road environment of multiple modalities.

[0011] From the above, even in areas without lanes, this aspect will still provide virtual lanes to guide the vehicle's driving route, so that the autonomous driving vehicle can drive in an orderly manner and improve the safety of autonomous driving.

[0012] From the above, the original data of the road environment can come from the original data collected by various vehicle-mounted sensors. The perception data of the road environment can come from the perception results of the perception module installed on the vehicle. The prior data of the road environment can come from the standard definition map of the lane. The various road environment data provided in this application are mutually verified objects, so as to improve the accuracy of the road environment data and obtain accurate road information.

[0013] As a possible implementation of the first aspect, topological analysis is performed based on road environment data of multiple modes to obtain a lane-level topological connection relationship of the road, including: fusing the road environment data of multiple modes to obtain fused data. Topological analysis is performed on the fused data to obtain a lane-level topological connection relationship of the road.

[0014] From the above, by fusing road environment data of multiple modalities and then performing topological analysis, the lane-level topological connection relationship of the analyzed road can be made closer to the topological connection relationship of the real road.

[0015] As a possible implementation method of the first aspect, road information is determined based on the topological connection relationship of the road lane level, including: performing semantic analysis based on road environment data of multiple modalities to obtain lane-level semantic information of the road; combining the lane-level topological connection relationship of the road with the lane-level semantic information of the road to obtain road information.

[0016] From the above, the lane-level semantic information of the road can include lane edge lines, lane center lines, lane areas, areas outside lanes, etc. It can also include the forward and reverse conditions of the lanes, and the association between related lanes and traffic rules, such as: traffic light information, speed limit information, etc.

[0017] As a possible implementation manner of the first aspect, the road information is obtained using a road information model; wherein the road information model is obtained based on neural network training.

[0018] As a possible implementation of the first aspect, the road information model is obtained based on neural network training, including: obtaining road environment data in training samples and road information corresponding to the road environment data; wherein the road information is obtained by pre-labeling. The road environment data in the training samples is used as input data when training the road information model, and the road information corresponding to the road environment data is used as expected output data when training the road information model, and the road information model is trained to obtain the road information model.

[0019] From the above, a method for obtaining road information based on a neural network model is provided, and a training method for the model is provided. The model is a pre-trained model and can also support online correction. Through the neural network model, road information can be determined in real time, providing real-time guidance for vehicle driving.

[0020] The second aspect of the present application provides a road information recognition device, including: a receiving module, a parsing module and a determination module. Specifically, the receiving module is used to receive road environment data of multiple modalities, wherein the road environment includes the environment of the lane area and the environment of the lane-free area. The parsing module is used to perform topological parsing based on the road environment data of multiple modalities to obtain the lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes in the road and the connection status of the lanes. The determination module is used to determine the road information of the road based on the lane-level topological connection relationship of the road. The road environment data of multiple modalities include: the original data of the road environment and / or the perception data of the road environment, and the prior data of the road environment.

[0021] As a possible implementation method of the second aspect, the lane-level topological connection relationship of the road in the lane parsing module specifically includes: the topological connection relationship between lanes in the lane area, the topological connection relationship between virtual lanes in the lane-free area, and the topological connection relationship between lanes in the lane area and virtual lanes in the lane-free area.

[0022] As a possible implementation method of the second aspect, before obtaining the topological connection relationship between virtual lanes in the lane-free area and before obtaining the topological connection relationship between lanes in the lane area and the virtual lanes in the lane-free area, it also includes: determining the virtual lanes in the lane-free area based on a road environment of multiple modalities.

[0023] As a possible implementation of the second aspect, the parsing module is specifically used to: fuse the road environment data of multiple modes to obtain fused data, and perform topological parsing on the fused data to obtain a lane-level topological connection relationship of the road.

[0024] As a possible implementation method of the second aspect, the lane determination module is specifically used to: perform semantic analysis based on road environment data of multiple modalities to obtain lane-level semantic information of the road; combine the lane-level topological connection relationship of the road with the lane-level semantic information of the road to obtain road information.

[0025] As a possible implementation manner of the second aspect, road information of the road is obtained using a road information model; wherein the road information model is obtained based on neural network training.

[0026] As a possible implementation method of the second aspect, the road information model is obtained based on neural network training, including: obtaining road environment data in training samples and lane structures corresponding to the road environment data; wherein the road information is obtained by pre-labeling; using the road environment data in the training samples as input data when training the road information model, using the road information corresponding to the road environment data as expected output data when training the road information model, and training the road information model to obtain the road information model.

[0027] The third aspect of the present application provides an electronic device, comprising: a processor, and a memory; a lane memory having program instructions stored thereon, wherein the lane program instructions, when executed by the lane processor, enable the processor to execute any one of the road information recognition methods of the first aspect described above.

[0028] The fourth aspect of the present application provides an electronic device, comprising: a processor, and an interface circuit; wherein the processor is accessed through the interface circuit, and the processor is configured to call program instructions stored in a memory, and when the program instructions are executed, the processor executes any one of the road information recognition methods of the first aspect mentioned above.

[0029] A fifth aspect of the present application provides a vehicle, comprising a vehicle body and the electronic device described in the third aspect or the fourth aspect.

[0030] The sixth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, characterized in that when the lane program instructions are executed by a computer, the computer executes any one of the road information recognition methods of the first aspect described above.

[0031] These and other aspects of the present application will become more apparent from the following description of the embodiment(s). BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The following further illustrates the various features of the present application and the connections between the various features with reference to the accompanying drawings. The accompanying drawings are all exemplary, and some features are not shown in actual proportion. Some drawings may omit features that are conventional in the field involved in the present application and are not necessary for the present application, or additional features that are not necessary for the present application may be shown. The combination of the various features shown in the accompanying drawings is not intended to limit the present application. In addition, throughout this specification, the same figure numerals refer to the same content. The specific description of the drawings is as follows:

[0033] Figure 1 A schematic diagram of a scenario in which a lane structure recognition method provided in an embodiment of the present application is applied;

[0034] Figure 2 A flowchart of a road information recognition method provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a lane structure recognition network based on lane data of multiple modes provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of feature vectors corresponding to various modal data provided in the embodiments of the present application;

[0037] Figure 5 A schematic diagram of feature vectors included in the graph nodes F1-F7 provided in the embodiment of the present application;

[0038] Figure 6 A data fusion schematic diagram provided for an embodiment of the present application;

[0039] Figure 7 A schematic diagram of a bifurcated road scenario provided in an embodiment of the present application;

[0040] Figure 8 A structural schematic diagram of a road information recognition device provided in an embodiment of the present application;

[0041] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0042] Fig.10 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The words "first, second, third, etc." or module A, module B, module C and other similar terms in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0044] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0045] The term "comprising" as used in the description and claims should not be interpreted as being limited to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the features, integers, steps or components mentioned, but does not exclude the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "a device comprising means A and B" should not be limited to a device consisting of components A and B only.

[0046] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics can be combined in any appropriate manner, as would be apparent to one of ordinary skill in the art from this disclosure.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present application. In case of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0048] The following is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings. First, a scenario in which a method for identifying road information provided by an embodiment of the present application is applied is introduced.

[0049] A method for identifying road information provided in an embodiment of the present application can be applied to an autonomous vehicle (AV). When the embodiment of the present application is applied to a vehicle, the vehicle may include a data acquisition device and a processor. The data acquisition device is used to obtain road environmental data. For example, the data acquisition device may be a sensor for collecting various road environmental data, such as a lidar, a camera, etc. The data acquisition device may also be an electronic device for receiving road environmental data, such as a data transmission chip, a data transmission chip such as a bus data transceiver chip, a network interface chip, etc. The data transmission chip may also be a wireless transmission chip, such as a Bluetooth chip or a WiFi chip, etc. In other embodiments, the data acquisition device may also be integrated into the processor to become an interface circuit or a data transmission module integrated into the processor. The processor may receive the road environmental data in real time and fuse and topologically analyze the road environmental data to obtain the topological connection relationship at the lane level of the road; then determine the topological connection relationship at the lane level of the road based on the topological connection relationship at the lane level of the road. Figure 1 When the embodiment of the present application is applied to a vehicle, the processor may be an electronic device, for example, a processor of a vehicle-mounted processing device such as a vehicle computer, a domain controller, a mobile data center (MDC) or a vehicle-mounted computer, or a conventional chip such as a central processing unit (CPU) or a microcontroller (MCU). The road information recognition method provided in the present application is used to obtain road information in real time, which can provide auxiliary functions for the planning and control of the automatic driving route of the vehicle.

[0050] like Figure 1 As shown in FIG. 1 , a specific scenario in which a road information recognition method provided in an embodiment of the present application is applied is shown. This scenario takes an intersection area (a mixed road section with real lane areas and without real lane areas) as an example. Vehicle A receives road environment data in real time and performs topological analysis on the road environment data to obtain the topological connection relationship of the lane level of the road; then, based on the topological connection relationship of the lane level of the road, the following is determined: Figure 1 The road structure is shown.

[0051] The road information includes the actual lane lines of the road ( Figure 1 The solid line in the middle), the virtual lane line of the road ( Figure 1 The dotted line part in the figure), the relationship between the red and green lights and the lanes at the intersection (not shown), etc. Based on the road information, the autonomous driving vehicle can be guided to drive in an orderly manner to ensure the safety of the vehicle.

[0052] In the embodiments of the present application, the real lane area can be considered as an area with real lane lines, and the area without real lanes can be considered as an area without real lane lines (generally intersections, vehicle bifurcations, vehicle confluences, etc.). Generally, vehicles generally travel along the real lane lines in the real lane area; in the area without real lanes, vehicles can generally travel freely under the premise of satisfying the traffic regulations of the corresponding area. Based on the technical solution provided in the embodiments of the present application, not only the real lane lines of the real lane area can be obtained, but also the virtual lane lines of the area without real lanes can be obtained, and the virtual lane lines can be connected with the corresponding real lane lines to obtain a complete road connection structure. For example Figure 1 The real lane 1 and the real lane 2 shown are connected by the virtual lane 3, and the route for the vehicle to enter the lane 2 from the lane 1 is planned, so that the orderly driving of the autonomous driving vehicle is realized.

[0053] It should be understood that the road information consists of multiple lanes and the driving rules of the corresponding lanes. The lane-level topological connection relationship of the road in the embodiment of the present application includes the connection of each lane in the road and its corresponding driving rules. Among them, the driving rules include the association rules between the red light and the lane, the speed limit, etc.

[0054] In one example of the present application, a semantic map of the road can be determined based on the collected road image data, and then the structural information of the road can be determined based on the semantic map. The structural information of the road includes lane grouping information. However, in this case, the lane grouping information is obtained, which is based on the grouping of the lane in which the vehicle is currently located, and does not provide the connection relationship between the front and rear lanes. Therefore, when providing opinions for the vehicle's driving trajectory, there will often be delays in opinions and even inaccurate planned routes.

[0055] In one example of this application, the laser point cloud of the vehicle's surroundings can be encoded and projected onto the image plane captured by the vehicle's camera, and then the road segmentation of the two-dimensional image can be completed through deep learning. However, the road segmentation in this case is only semantic-level segmentation, and there is no output of relevant road topological structure attributes, so it cannot fully describe the road information and cannot meet the needs of autonomous driving vehicle route planning.

[0056] Referring to the figures below, a method for identifying road information provided in an embodiment of the present application is described in detail.

[0057] like Figure 2 The figure is a flow chart of the road information recognition method provided by the embodiment of the present application. The implementation process of the method mainly includes steps S110-S130, and each step is introduced in turn below:

[0058] S110: Receive road environment data of multiple modes, wherein the road environment includes an environment of a lane area and an environment of a non-lane area.

[0059] In this embodiment, one of the modal road environment data may be the raw data of the lanes on the road, and the raw data of the road may be collected by various sensors installed on the vehicle, that is, the raw data collected by the sensors. For example, the sensor may be a laser radar, and the raw data of the lanes collected is laser point data. For another example, the sensor may be a camera, and the raw data of the lanes on the road collected is image data. For another example, the sensor may also be an ultrasonic sensor, and the raw data of the lanes on the road collected is ultrasonic data.

[0060] In this embodiment, another modality of road environment data may be perception data of lanes on the road, and the perception data of the road may be obtained through various perception modules installed on the vehicle, that is, the perception result of the perception module is the perception data. For example, when the perception data is the detection result of the lane line, then the perception result of the perception module is the lane line result obtained by parsing the laser data or the image data. For another example: when the perception data is the positioning result of the vehicle, then the perception result of the perception module is the positioning information of the vehicle obtained by the positioning module. For another example: when the perception data is the detection result of obstacles in the surrounding environment, then the perception result of the perception module is the detection result of surrounding obstacles (for example, vehicles, pedestrians, etc.) obtained by parsing the laser data or the image data.

[0061] In this embodiment, another modality of road environment data may also be the prior data of lanes in the road environment, and the prior data of the road comes from a standard definition map (SD map), which may also be called a navigation map. A standard definition map is an electronic map with lower accuracy (its accuracy is generally around 10 meters) and fewer data dimensions (generally only recording dimensions such as the shape, slope, curvature, and direction of the road), and its users are generally drivers. A standard definition map is different from a high definition map (HD map). Simply put, a high definition map refers to an electronic map with higher accuracy and more data dimensions, and its accuracy generally needs to reach 20 centimeters. Its dimensions include not only the dimensions of the standard definition map, but also data related to lane attributes, such as the type of lane line, lane width, etc. Its users are generally autonomous driving vehicles.

[0062] Among them, the original data, perception data and prior data of the road environment can be used to verify each other to improve robustness.

[0063] As an implementation manner, the road environment data may include a priori data of the road environment, and any one of original data of the road and perceived data of the road environment.

[0064] As another implementation, the road environment data may also include a priori data of the road environment, original data of the road, and perceived data of the road environment. It should be understood that the three modalities of road environment data listed above are only exemplary descriptions. In other embodiments, other modalities of road environment data may also be included. The embodiments of the present application do not limit them. In addition, the embodiments of the present application do not limit the means for obtaining various modalities of road environment data, and the above description only provides some possible implementations.

[0065] S120: Fusing road environment data of multiple modes to obtain fused data.

[0066] Among them, the fusion of road environment data of multiple modes can adopt the method of probability theory and statistics, such as weighted average method, Bayesian estimation method, etc.; the fusion of road environment data of multiple modes can also adopt the method of logical reasoning, such as fuzzy reasoning, etc.; the fusion of road environment data of multiple modes can also adopt the search-based fusion algorithm, such as genetic algorithm (GA), particle swarm optimization algorithm (PSO), etc.; the fusion of road environment data of multiple modes can also adopt the method of neural network, such as graph neural network, etc. In the embodiments of the present application, the fusion of road environment data of multiple modes is illustrated by graph neural network. For the specific fusion process, please refer to the description of the feature fusion part in the following embodiments using graph neural network.

[0067] S120: Perform topological analysis based on the road environment data of the multiple modes to obtain a lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes on the road and the connection status of the lanes.

[0068] In this embodiment, the topological connection relationship of the lane level of the road includes the topological connection relationship between the real lanes in the lane area, the topological connection relationship between the virtual lanes in the laneless area, and the topological connection relationship between the lanes in the lane area and the virtual lanes in the laneless area. Among them, typical representatives of the laneless area are intersection areas, traffic bifurcation areas, traffic merging areas, etc. The topological connection relationship in this embodiment can provide the virtual lanes in the laneless area and the connection topology of the virtual lanes for the autonomous driving vehicle to use for route planning.

[0069] It should be understood that before providing the topological connection relationship between the virtual lanes in the lane-free area, it is first necessary to determine the virtual lanes in the lane-free area. As an optional implementation, the virtual lanes in the lane-free area can be obtained based on a neural network. Optionally, the neural network can be one or more of a convolutional neural network, a BP neural network, a deep neural network, or other neural networks, which are not limited in the embodiments of the present application.

[0070] Similarly, before providing the topological connection relationship between the lanes in the lane area and the virtual lanes in the lane-free area, it is also necessary to first determine the virtual lanes in the lane-free area. The method of obtaining the virtual lanes in the lane-free area here can also be obtained by using a neural network. Similarly, the neural network here can also be one or more of a convolutional neural network, a BP neural network, a deep neural network, or other neural networks, which are not limited in the embodiments of the present application.

[0071] As an optional implementation method, the road environment data of multiple modes are first fused, and then the fused data is topologically analyzed to obtain the lane-level topological connection relationship of the road.

[0072] Among them, the fusion of road environment data of multiple modes can adopt the method of probability theory and statistics, such as weighted average method, Bayesian estimation method, etc.; the fusion of road environment data of multiple modes can also adopt the method of logical reasoning, such as fuzzy reasoning, etc.; the fusion of road environment data of multiple modes can also adopt the search-based fusion algorithm, such as genetic algorithm (GA), particle swarm optimization algorithm (PSO), etc.; the fusion of road environment data of multiple modes can also adopt the method of neural network, such as graph neural network, etc. In the embodiments of the present application, the fusion of road environment data of multiple modes is illustrated by graph neural network. For the specific fusion process, please refer to the description of the feature fusion part in the following embodiments using graph neural network.

[0073] As an optional implementation, the virtual lanes in the lane-free areas in the training samples can be obtained by pre-marking the training samples. In this embodiment, the marking of the training samples here can generally be done by manual marking.

[0074] It should be understood that if the road environment data is of one modality, the topological connection relationship at the lane level of the road can be obtained by directly performing topological analysis on the road environment data, and there is no need to perform the data fusion step.

[0075] In this embodiment, the road environment data can be topologically parsed based on the road information model, and the road information model is obtained based on neural network training. Optionally, the road information model in this embodiment can be pre-trained and / or trained online. That is, the road information model can be obtained through offline training, or it can also be obtained through online training, or it can also be obtained through a combination of offline training and online training.

[0076] S130: Determine the road information of the road based on the lane-level topological connection relationship of the road.

[0077] In this step, semantic analysis is first performed based on the road environment data of the multiple modalities to obtain the lane-level semantic information of the road. Then, the lane-level topological connection relationship of the road and the lane-level semantic information of the road are combined to obtain the road information of the road.

[0078] In this step, the road environment data may be semantically parsed based on a road information model. Optionally, the road information model is obtained based on neural network training.

[0079] like Figure 3 As shown, a schematic diagram of a road information recognition network based on multi-modal lane data (i.e., a schematic diagram of a network structure of a road information model) provided in an embodiment of the present application is provided. A specific implementation method of road information recognition in this embodiment will be described in detail below based on the recognition network.

[0080] First, multi-modal road environment data are used as input data of the network.

[0081] In this embodiment, the road environment data includes the original data of the road environment, such as Figure 3 The laser radar data U1 and image data U2 are shown; the road environment data also includes the perception data of the road environment, such as Figure 3 The lane line U3 and the segmented image U4 are shown; the road environment data also includes the prior data of the road environment, such as Figure 3 The shape of the lane U5 and the direction of the lane U6 are shown. The data of the above three modes are respectively input into the corresponding encoding networks for feature extraction. For example, the encoding network can adopt a neural network. It should be understood that in this embodiment, the input of each mode will correspond to a different encoding network, and the embodiment of the present application is not limited thereto.

[0082] Then, the output results of the encoding network are subjected to feature fusion. As an optional implementation method, feature fusion can be implemented in the form of a graph neural network. The graph neural network needs to obtain the initial values ​​of the graph nodes and edge matrices.

[0083] The following first introduces the process of obtaining graph nodes in the graph neural network. Specifically, each modal data extracted from the encoding network is encoded into a feature vector, for example, Figure 4 As shown, the first modal data (original data of the road environment) is encoded as a feature vector f1, the second modal data (perception data of the road environment) is encoded as a feature vector f2, and the third modal data (prior data of the road environment) is encoded as a feature vector f3. Then, the feature vectors are combined to generate graph nodes. In this embodiment, the feature vectors f1, f2, and f3 are combined to obtain seven graph nodes F1-F7, where, when a feature vector of a specific modality is missing in a certain combination, it can be supplemented by an empty feature vector, such as Figure 5 The figure shows the eigenvectors included in the graph nodes F1-F7. According to the above principles, it can be concluded that the eigenvectors included in the graph node F1 are eigenvector f1 and two empty eigenvectors; the eigenvectors included in the graph node F2 are eigenvector f2 and two empty eigenvectors; the eigenvectors included in the graph node F3 are eigenvector f3 and two empty eigenvectors; the eigenvectors included in the graph node F4 are eigenvector f1, eigenvector f2 and an empty eigenvector; the eigenvectors included in the graph node F5 are eigenvector f1, an empty eigenvector and eigenvector f3; the eigenvectors included in the graph node F6 are an empty eigenvector, eigenvector f2 and eigenvector f3; the eigenvectors included in the graph node F7 are eigenvector f1, eigenvector f2 and eigenvector f3.

[0084] Next, the process of obtaining the initial value of the edge matrix in the graph neural network is introduced. The initial value of the edge matrix in this embodiment can be set by the ratio of the number of intersections of the modes contained in the relevant graph nodes to the number of unions of the modes contained. The following table shows the initial value of the edge matrix between the above 7 graph nodes F1-F7.

[0085] Table 1

[0086]

[0087] Specifically, for example: the eigenvectors included in graph node F2 are eigenvector f2 and two empty eigenvectors, and the eigenvectors included in graph node F4 are eigenvector f1, eigenvector f2, and an empty eigenvector. Therefore, the intersection of the modes contained in these two graph nodes is eigenvector f2, the number of intersections is 1, and the union of the modes contained in these two graph nodes is eigenvector f1 and eigenvector f2. Therefore, the initial value of the corresponding edge matrix between graph node F2 and graph node F4 is 1 / 2=0.5. Another example: the eigenvectors included in graph node F6 are an empty eigenvector, eigenvector f2, and eigenvector f3, and the eigenvectors included in graph node F7 are eigenvector f1, eigenvector f2, and eigenvector f3. Therefore, the intersection of the modes contained in these two graph nodes is the eigenvector f2 and the eigenvector f3, the number of intersections is 2, and the union of the modes contained in these two graph nodes is the eigenvector f1, the eigenvector f2 and the eigenvector f3. Therefore, the initial value of the edge matrix corresponding to the graph node F6 and the graph node F7 is 2 / 3≈0.67. The calculation process of the initial value of the edge matrix between other graph nodes is similar and will not be repeated here.

[0088] Next, the graph nodes (graph nodes F1, graph nodes F2, and graph nodes F3) obtained through the graph neural network that contain only one modality of data are used as features and convolution operations are performed on them using convolution kernels to obtain fused data. Figure 6 As shown, the corresponding Figure 4 The convolution kernels of the three modes shown (3*3*1) ( Figure 6 The left dashed box) and the feature vectors (1*9) of the nodes corresponding to the three modal data after the graph neural network processing are given ( Figure 6 The corresponding fused feature vector is (3*3*9) ( Figure 6 As an optional implementation, the convolution kernel can be obtained by encoding each modality data extracted from the encoding network into a matrix, and the value in the matrix represents the sensitivity of the modality at the pixel. It should be understood that Figure 6 The values ​​in the shown matrices are merely example values ​​and can be arbitrarily set according to actual conditions.

[0089] For example, Figure 6The calculation process of A1 value is A1=1*1+2*0.1+9*0.7=7.5, that is, the product of the first value (1) in the eigenvector of the node corresponding to the first modal data and the corresponding value (1) in the first row and first column of the convolution kernel corresponding to the first modal data + the product of the first value (2) in the eigenvector of the node corresponding to the second modal data and the corresponding value (0.1) in the first row and first column of the convolution kernel corresponding to the second modal data + the product of the first value (9) in the eigenvector of the node corresponding to the third modal data and the corresponding value (0.7) in the first row and first column of the convolution kernel corresponding to the third modal data. Figure 6 The calculation process of the B1 value is B1=1*0.5+2*0.6+9*0.4=5.3, that is, the product of the first value (1) in the eigenvector of the node corresponding to the first modal data and the corresponding value (0.5) in the first row and second column of the convolution kernel corresponding to the first modal data + the product of the first value (2) in the eigenvector of the node corresponding to the second modal data and the corresponding value (0.6) in the first row and second column of the convolution kernel corresponding to the second modal data + the product of the first value (9) in the eigenvector of the node corresponding to the third modal data and the corresponding value (0.4) in the first row and second column of the convolution kernel corresponding to the third modal data. Figure 6 The calculation process of the A2 value is A2=2*1+1*0.1+6*0.7=6.3, that is, the product of the second value (2) in the eigenvector of the node corresponding to the first modal data and the corresponding value (1) in the first row and first column of the convolution kernel corresponding to the first modal data + the product of the second value (1) in the eigenvector of the node corresponding to the second modal data and the corresponding value (0.1) in the first row and first column of the convolution kernel corresponding to the second modal data + the product of the second value (6) in the eigenvector of the node corresponding to the third modal data and the corresponding value (0.7) in the first row and first column of the convolution kernel corresponding to the third modal data. Figure 6 The calculation process of the B2 value in is B2=2*0.5+1*0.6+6*0.4=4, that is, the product of the second value (2) in the eigenvector of the node corresponding to the first modal data and the corresponding value (0.5) in the first row and second column of the convolution kernel corresponding to the first modal data + the product of the second value (1) in the eigenvector of the node corresponding to the second modal data and the corresponding value (0.6) in the first row and second column of the convolution kernel corresponding to the second modal data + the product of the second value (6) in the eigenvector of the node corresponding to the third modal data and the corresponding value (0.4) in the first row and second column of the convolution kernel corresponding to the third modal data. In addition, the calculation process of other positions of other channels is the same as that of A1 and B1 shown in the first channel and A2 and B2 shown in the second channel. It is only necessary to perform convolution operations on the corresponding positions, which will not be repeated here.

[0090] After the feature fusion is completed, the fused data needs to be parsed. In the recognition network of this embodiment, there are two parts: topological parsing and semantic parsing.

[0091] like Figure 7 As shown, a schematic diagram of a lane bifurcation scene is provided, wherein the solid line portion represents the lane boundary line, and the dotted line portion represents the lane center line. In this embodiment, the corresponding relationship between the semantic information of the lane and the semantic classification number is first defined, for example: the semantic classification number of the lane boundary line is defined as 2, the semantic classification number of the lane center line is defined as 1, and the semantic classification number of other areas is defined as 0, as shown in Table 2 below:

[0092] Table 2

[0093] Semantic Information Semantic Classification Number Lane Boundary 2 Lane Center 1 Other Areas(Empty) 0

[0094] right Figure 7 By performing semantic analysis on the scene shown in the figure, the lane semantic correspondence table shown in Table 3 can be obtained:

[0095] Table 3

[0096]

[0097] right Figure 7 By performing topological analysis on the scene shown in FIG. 1 , the topological connection relationship of the lanes in the top-down perspective of the scene can be obtained as shown in Table 4 below, where the number stored in each position in Table 4 represents the index number in the corresponding lane semantics corresponding table (Table 3). It should be understood that the index number can be understood as an intermediate quantity used to establish the corresponding relationship between lane semantics and lane topology, that is, the semantics corresponding to the lane topology and each position in the actual lane area can be obtained through the index number.

[0098] Table 4

[0099] 1 0 3 0 1 2 3 4 5 6 6 1 0 3 0 1 2 3 4 5 6 6 1 0 3 0 1 2 3 4 5 6 6 0 1 0 3 1 2 3 4 5 6 6 0 0 1 0 1 2 3 4 5 6 6 0 0 0 1 1 3 3 4 5 6 6 0 0 0 0 1 2 3 4 5 6 6 0 0 0 0 1 2 3 4 5 6 6 0 0 0 0 1 2 3 4 5 6 6 0 0 0 0 1 2 3 4 5 6 6 0 0 0 0 1 2 3 4 5 6 6

[0100] After completing the semantic analysis and topological analysis, the two are combined, that is, the index numbers at each position in Table 4 are used to search for the corresponding in Table 3 to obtain the corresponding road information. For example: the semantic classification number corresponding to the positions with index numbers 0, 2, 4, and 6 in Table 4 is 0, which represents other areas, the semantic classification number corresponding to the positions with index numbers 1 and 5 in Table 4 is 2, which represents the lane boundary line, and the semantic classification number corresponding to the position with index number 3 in Table 4 is 1, which represents the lane center line. As shown in Table 5A below, it is the topological information relationship of the lane boundary line, Table 5B below is the topological information relationship of the lane center line, and Table 5C below is the road information obtained by combining the information in Table 5A and Table 5B.

[0101] Table 5A

[0102] 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

[0103] Table 5B

[0104]

[0105]

[0106] Table 5C

[0107] 2 1 2 1 2 2 1 2 1 2 2 1 2 1 2 2 1 2 1 2 2 2 1 2 2 2 1 1 2 2 1 2 2 1 2 2 1 2 2 1 2 2 1 2

[0108] It should be understood that the above-mentioned semantic information is only an exemplary description, and in other embodiments, other semantics may also be included, such as red light signs, lane signs (eg, speed limit signs), etc.

[0109] Based on the road information recognition method provided in the embodiment of the present application, the lane-level topology and corresponding semantic information of the road can be output, which greatly facilitates autonomous driving planning, control and prediction, and realizes the output of virtual lanes in lane-free areas, such as intersections, lane bifurcation areas, lane merging areas, etc., which can better support high-level autonomous driving functions.

[0110] Another embodiment of the present application provides a training process for the above-mentioned road information model. The training process is divided into a data labeling process and a training process. The data labeling process is to label the data in the training sample so that a set of road environment data corresponds to its specific road information.

[0111] After completing the sample labeling, the road environment data in the training sample is used as the input data for training the road information model, and the corresponding road information is used as the expected output data for training the road information model. The road information model is obtained by continuously adjusting the model parameters.

[0112] As an optional implementation, the training samples can be divided into multiple training sub-samples according to different road scenes to train the road information model. For example, the training samples can be divided into training sub-samples of intersection areas, training sub-samples of forked road areas, training sub-samples of merging areas, etc. For each training sub-sample, the corresponding road information sub-model is trained respectively, and then the various road information sub-models are associated to obtain the final road information model. As another optional implementation, the training samples can be trained directly to directly obtain the final required road information model.

[0113] In the training process of the above model, because we hope that the output of the model is as close to the real scene (i.e. the result of the annotation) as possible, we can use the loss function or objective function to obtain the difference between the output of the current model and the annotation result, and then adjust and update the weight vectors of each layer of the model network according to the difference between the two. By continuously adjusting the weight vectors of each layer, the model training is terminated until the probability that the output of the model is the result of the annotation meets the requirements. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference between the output of the model and the annotation result, so the training process of the model becomes a process of minimizing the loss as much as possible.

[0114] Another embodiment of the present application provides a road information recognition device, which can be implemented by a software system, a hardware device, or a combination of a software system and a hardware device.

[0115] It should be understood that Figure 8 This is only a schematic diagram of a structure of a road information recognition device, and the present application does not limit the division of functional modules in the lane structure recognition device. Figure 8 As shown, the road information recognition device can be logically divided into multiple modules, each module can have different functions, and the function of each module can be realized by a processor in an electronic device reading and executing instructions in a memory. Exemplarily, the road information recognition device includes a receiving module 810, a parsing module 820 and a determining module 830.

[0116] In an optional implementation, the lane structure recognition device is used to perform Figure 2 The contents described in steps S110-S130 shown. Specifically, it can be: a receiving module 810, used to receive road environment data of multiple modes, wherein the road environment includes the environment of the lane area and the environment of the non-lane area. The parsing module 820 is used to perform topological parsing according to the road environment data of multiple modes to obtain the lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between the lanes in the road and the connection status of the lanes. The determination module 830 is used to determine the road information of the road based on the lane-level topological connection relationship of the road.

[0117] In this embodiment, the road environment data of the multiple modes include: the original data of the road environment and / or the perception data of the road environment, and the prior data of the road environment. The specific method of obtaining the road environment data of various modes can be referred to the detailed description in the above embodiment, and will not be described in detail in this embodiment.

[0118] In this embodiment, the lane-level topological connection relationship of the road in the analysis module 820 specifically includes: the topological connection relationship between lanes in the lane area, the topological connection relationship between virtual lanes in the lane-free area, and the topological connection relationship between lanes in the lane area and virtual lanes in the lane-free area.

[0119] In this embodiment, before obtaining the topological connection relationship between the virtual lanes in the lane-free area, the method further includes: determining the virtual lanes in the lane-free area according to the road environment of the multiple modes.

[0120] In addition, before obtaining the topological connection relationship between the lanes in the lane area and the virtual lanes in the lane-free area, the method further includes: determining the virtual lanes in the lane-free area according to the road environment of the multiple modes.

[0121] In this embodiment, the analysis module 820 is used to: fuse the road environment data of the multiple modes to obtain fused data, and perform topological analysis on the fused data to obtain the lane-level topological connection relationship of the road. The specific topological analysis process can be referred to the detailed description in the above embodiment, and will not be described in detail in this embodiment.

[0122] Specifically, the determination module 830 is specifically used to: perform semantic analysis based on the road environment data of the multiple modes to obtain the semantic information of the lane; combine the topological connection relationship of the lane with the semantic information of the lane to obtain the lane-level road information of the road. The specific semantic analysis process can be found in the detailed description of the above embodiment, and will not be described in detail in this embodiment.

[0123] As an optional implementation, the road information is obtained using a road information model; wherein the road information model is obtained based on neural network training. The specific training process of the road information model can refer to the description of the road information model training in the above embodiment, and this embodiment will not repeat it.

[0124] It should be noted that the specific implementation of each functional module in this embodiment can be found in the introduction of the above method embodiment, and this embodiment will not describe it in detail.

[0125] The present application also provides an electronic device, including a processor and a memory. The memory stores program instructions, which, when executed by the processor, cause the processor to execute Figure 2 The method of the corresponding embodiment, or each optional embodiment thereof.

[0126] Fig. 91 is a schematic structural diagram of an electronic device 900 provided in an embodiment of the present application. The electronic device 900 includes: a processor 910 and a memory 920.

[0127] It should be understood that Fig. 9 The electronic device 900 shown in FIG. 1 may further include a communication interface 930 , which may be used for communicating with other devices.

[0128] The processor 910 may be connected to a memory 920. The memory 920 may be used to store the program code and data. Therefore, the memory 920 may be a storage unit inside the processor 910, or an external storage unit independent of the processor 910, or a component including a storage unit inside the processor 910 and an external storage unit independent of the processor 910.

[0129] Optionally, the electronic device 900 may further include a bus. The memory 920 and the communication interface 930 may be connected to the processor 910 via a bus. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0130] It should be understood that in the embodiment of the present application, the processor 910 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 910 may adopt one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present application.

[0131] The memory 920 may include a read-only memory and a random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include a nonvolatile random access memory. For example, the processor 910 may also store information on the device type.

[0132] When the electronic device 900 is running, the processor 910 executes the computer execution instructions in the memory 920 to perform the operation steps of the above method.

[0133] It should be understood that the electronic device 900 according to the embodiment of the present application can correspond to the corresponding subject in the method according to each embodiment of the present application, and the above-mentioned and other operations and / or functions of each module in the electronic device 900 are respectively for realizing the corresponding processes of each method of the present embodiment. For the sake of brevity, they will not be repeated here.

[0134] The present application also provides another electronic device, such as Fig.10 The structure schematic diagram of another electronic device 1000 provided by this embodiment includes: a processor 1010 and an interface circuit 1020, wherein the processor 1010 accesses a memory through the interface circuit 1020, and the memory stores program instructions. When the program instructions are executed by the processor, the processor executes Figure 2 In addition, the electronic device may also include a communication interface, a bus, etc. For details, see Fig. 9 The description of the embodiment shown is omitted. Exemplarily, the interface circuit 1020 may be a CAN bus or a LIN bus.

[0135] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program is used to execute a method for identifying road information, which method includes at least one of the solutions described in the above embodiments.

[0142] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive lists) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.

[0143] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0144] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0145] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0146] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, all of which belong to the protection scope of the present application.

Claims

1. A method for identifying road information, characterized in that: include: Receiving road environment data of multiple modes; wherein the road environment includes an environment of a lane area and an environment of a non-lane area; Performing topological analysis based on the road environment data of the multiple modalities to obtain a lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes in the road and the connection status of the lanes; the lane-level topological connection relationship of the road includes: a topological connection relationship between lanes in a lane area, a topological connection relationship between virtual lanes in a lane-free area, and a topological connection relationship between lanes in a lane area and virtual lanes in a lane-free area; Determining road information of the road based on a lane-level topological connection relationship of the road; The multiple modal road environment data include: The original data of the road environment and / or the perception data of the road environment, and the prior data of the road environment.

2. The method according to claim 1, characterized in that Before obtaining the topological connection relationship between the virtual lanes in the laneless area, and before obtaining the topological connection relationship between the lanes in the lane area and the virtual lanes in the laneless area, the method further includes: A virtual lane in the lane-free area is determined according to the road environment of the multiple modalities.

3. The method according to claim 1, characterized in that The performing topological analysis according to the road environment data of the multiple modes to obtain a lane-level topological connection relationship of the road includes: Fusing the road environment data of the multiple modes to obtain fused data; The fused data is topologically parsed to obtain a lane-level topological connection relationship of the road.

4. The method according to claim 1, characterized in that: The determining the road information of the road based on the lane-level topological connection relationship of the road includes: Performing semantic analysis based on the road environment data of the multiple modalities to obtain lane-level semantic information of the road; The lane-level topological connection relationship of the road and the lane-level semantic information of the road are combined to obtain the road information of the road.

5. The method according to claim 4, characterized in that The road information of the road is obtained using a road information model; wherein the road information model is obtained based on neural network training.

6. The method according to claim 5, characterized in that The road information model is obtained based on neural network training, and includes: Acquire the road environment data in the training sample and the road information corresponding to the road environment data; wherein the road information is obtained by pre-marking; The road environment data in the training sample is used as input data when training the road information model, and the road information corresponding to the road environment data is used as expected output data when training the road information model, and the road information model is trained to obtain the road information model.

7. A road information recognition device, characterized in that: include: A receiving module, used to receive road environment data of multiple modes; wherein the road environment includes an environment of a lane area and an environment of a non-lane area; A parsing module, configured to perform topological parsing according to the road environment data of the multiple modes to obtain a lane-level topological connection relationship of the road, wherein the lane-level topological connection relationship of the road is used to represent the relative position relationship between lanes in the road and the connection status of the lanes; the lane-level topological connection relationship of the road specifically includes: a topological connection relationship between lanes in a lane area, a topological connection relationship between virtual lanes in a lane-free area, and a topological connection relationship between lanes in a lane area and virtual lanes in a lane-free area; A determination module, configured to determine the road information of the road based on the lane-level topological connection relationship of the road; The multiple modal road environment data include: The original data of the road environment and / or the perception data of the road environment, and the prior data of the road environment.

8. The device according to claim 7, characterized in that Before obtaining the topological connection relationship between the virtual lanes in the laneless area, and before obtaining the topological connection relationship between the lanes in the lane area and the virtual lanes in the laneless area, the method further includes: A virtual lane in the lane-free area is determined according to the road environment of the multiple modalities.

9. The device according to claim 7, characterized in that The analysis module is specifically used for: Fusing the road environment data of the multiple modes to obtain fused data; The fused data is topologically parsed to obtain a lane-level topological connection relationship of the road.

10. The device according to claim 7, characterized in that The determination module is specifically used for: Performing semantic analysis based on the road environment data of the multiple modalities to obtain lane-level semantic information of the road; The lane-level topological connection relationship of the road and the lane-level semantic information of the road are combined to obtain the road information of the road.

11. The device according to claim 10, characterized in that The road information of the road is obtained using a road information model; wherein the road information model is obtained based on neural network training.

12. The device according to claim 11, characterized in that The road information model is obtained based on neural network training, and includes: Acquire the road environment data in the training sample and the road information corresponding to the road environment data; wherein the road information is obtained by pre-marking; The road environment data in the training sample is used as input data when training the road information model, and the road information corresponding to the road environment data is used as expected output data when training the road information model, and the road information model is trained to obtain the road information model.

13. An electronic device, characterized in that: include: processor, and memory; The memory stores program instructions thereon, and when the program instructions are executed by the processor, the processor executes the road information recognition method according to any one of claims 1-6.

14. An electronic device, characterized in that: include: processor, and interface circuit; The processor is accessed through the interface circuit, and the processor is configured to call program instructions stored in a memory, and when the program instructions are executed, the processor executes the road information recognition method according to any one of claims 1 to 6.

15. A vehicle, characterized in that: The device comprises a vehicle body and the electronic device according to claim 13 or 14.

16. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a computer, the computer is caused to execute the road information recognition method according to any one of claims 1 to 6.

Images