Map construction method based on large-scale models, vehicle control method, apparatus, electronic equipment, storage medium, and computer program

Abstract

To provide a method for constructing a map based on a large model, a vehicle control method, an apparatus, an electronic device, a storage medium, and a computer program which are applicable to scenes of automatic driving, unmanned driving, etc.SOLUTION: A method for constructing a map based on a large model, includes acquiring an associated-area lane attribute and an image to be detected which is collected by a vehicle-side sensor. The image to be detected represents a road area to be detected. The associated-area lane attribute corresponds to an associated road area. The associated road area and the road area to be detected meet a predetermined similarity condition. The method further includes constructing target prompt information based on the associated-area lane attribute, and processing the target prompt information and the image to be detected by using the large model to obtain an area road map for the road area to be detected.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to the fields of computer vision, deep learning, large-scale models, and generative model technology, and is applicable to scenarios such as autonomous driving and driverless driving. Specifically, it relates to a map construction method, a vehicle control method, an apparatus, an electronic device, a storage medium, and a computer program based on a large-scale model.

Background Art

[0002] With the rapid development of science and technology, the number of vehicles running on the road shows a relatively rapid increasing trend. Vehicles can realize the autonomous driving function through road maps, and drivers can also assist in the driving of vehicles based on relatively accurate road maps, improving driving safety.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present disclosure provides a map construction method, a vehicle control method, an apparatus, an electronic device, a storage medium, and a computer program based on a large-scale model.

Means for Solving the Problems

[0004] According to an aspect of the present disclosure, it is to obtain a detection target image collected by relevant area lane attributes and vehicle-side sensors, where the detection target image represents a detection target road area, the relevant area lane attributes correspond to a relevant road area, and a predetermined similarity condition is satisfied between the relevant road area and the detection target road area; construct target presentation information based on the relevant area lane attributes; and use a large-scale model to process the target presentation information and the detection target image to obtain an area road map of the detection target road area. A map construction method based on a large-scale model is provided.

[0005] In another aspect of this disclosure, this embodiment provides a vehicle control method that includes controlling the movement of a vehicle based on a road map constructed by a map construction method based on a large-scale model provided by this embodiment.

[0006] In another aspect of this disclosure, a map construction device based on a large-scale model is provided, which includes: an acquisition module that acquires relevant area lane attributes and detection target images collected by vehicle-side sensors, wherein the detection target images represent a detection target road area, the relevant area lane attributes correspond to a relevant road area, and a predetermined similarity condition is met between the relevant road area and the detection target road area; a target presentation information construction module that constructs target presentation information based on the relevant area lane attributes; and a first map construction module that processes the target presentation information and detection target images using a large-scale model to acquire an area road map of the detection target road area.

[0007] In another aspect of this disclosure, this embodiment provides a vehicle control device that includes a vehicle control module that controls the movement of a vehicle based on a road map constructed by a map-building device based on a large-scale model provided by this embodiment.

[0008] An embodiment of the present disclosure provides an electronic device comprising at least one processor and memory communicated with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a map building method based on a large model provided by an embodiment of the present disclosure.

[0009] According to embodiments of the present disclosure, an electronic device is provided which includes at least one processor and a memory communicated with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a vehicle control method provided by embodiments of the present disclosure.

[0010] According to embodiments of the present disclosure, an autonomous vehicle is provided, which includes electronic equipment for performing a vehicle control method provided by embodiments of the present disclosure.

[0011] According to embodiments of the present disclosure, a non-temporary computer-readable storage medium is provided in which computer instructions are stored, the computer instructions being used to cause a computer to perform a method provided by embodiments of the present disclosure.

[0012] According to embodiments of the present disclosure, a computer program is provided that, when executed by a processor, implements the method provided by embodiments of the present disclosure.

[0013] It should be understood that the content described in this section is not intended to represent key points or important features of the embodiments of this disclosure, nor does it limit the scope of this disclosure. Other features of this disclosure will be readily apparent from the following description. [Brief explanation of the drawing]

[0014] The drawings are for the purpose of better understanding the present invention and do not limit the present disclosure.

[0015] [Figure 1] Figure 1 schematically shows an exemplary system architecture to which the map construction method and apparatus based on a large-scale model according to the embodiments of this disclosure can be applied.

[0016] [Figure 2] Figure 2 schematically shows a flowchart of a map construction method based on a large-scale model according to an embodiment of this disclosure.

[0017] [Figure 3] Figure 3 schematically shows the principle of a map construction method based on a large-scale model according to an embodiment of this disclosure.

[0018] [Figure 4]FIG. 4 schematically shows a schematic diagram of the principle of an encoder according to an embodiment of the present disclosure.

[0019] [Figure 5] FIG. 5 schematically shows a schematic diagram of the principle of a map construction method based on a large-scale model according to another embodiment of the present disclosure.

[0020] [Figure 6] FIG. 6 schematically shows a flowchart of a vehicle control method according to an embodiment of the present disclosure.

[0021] [Figure 7] FIG. 7 schematically shows a block diagram of a map construction apparatus based on a large-scale model according to an embodiment of the present disclosure.

[0022] [Figure 8] FIG. 8 schematically shows a block diagram of a vehicle control apparatus according to an embodiment of the present disclosure.

[0023] [Figure 9] FIG. 9 schematically shows a block diagram of an electronic device suitable for implementing a map construction method and a vehicle control method based on a large-scale model according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. Here, various details of the embodiments of the present disclosure are included for easier understanding, and they should be considered exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clear and concise description, descriptions of well-known functions and configurations are omitted in the following description.

[0025] In the proposed technology disclosed herein, the acquisition, storage, and application of user personal information comply with the provisions of relevant laws and regulations, necessary security measures are taken, and it does not violate public order and morals.

[0026] Road maps with lane-level navigation functionality can improve the safety and efficiency of autonomous vehicle driving. However, the inventors have discovered that some road map construction methods have low lane accuracy, resulting in low road map update efficiency and difficulty in accurately representing lane attributes such as actual lane boundary type and lane boundary direction.

[0027] Embodiments of this disclosure provide a map construction method based on a large-scale model, a vehicle control method, an apparatus, an electronic device, a storage medium, and a computer program. The map construction method based on the large-scale model includes acquiring relevant area lane attributes and detection target images collected by vehicle-side sensors, wherein the detection target images represent a detection target road area, the relevant area lane attributes correspond to a relevant road area, and predetermined similarity conditions are met between the relevant road area and the detection target road area; constructing target presentation information based on the relevant area lane attributes; and processing the target presentation information and detection target images using the large-scale model to obtain an area road map of the detection target road area.

[0028] According to the embodiments of this disclosure, by acquiring detection target images collected by vehicle-side sensors, acquiring related area lane attributes of related road areas that satisfy predetermined similarity conditions with the detection target road area, and constructing target presentation information based on the related area lane attributes, the target presentation information can include lane attribute information that is highly relevant to the lanes of the detection target road area. Furthermore, by processing the target presentation information and detection target images using a large-scale model, it is possible to make the lane attribute information represented by the target presentation information an auxiliary understanding of the detection target image. By controlling the large-scale model based on the target presentation information, it is possible to control the large-scale model under conditions that allow for relatively sufficient understanding of lane attributes that are highly relevant to the detection target road area, thereby enabling relatively accurate detection of the lanes of the detection target road area. This also improves the detection accuracy and rendering accuracy of the area road map. Simultaneously, detection target images can also be acquired by vehicle-side sensors of vehicles traveling near the detection target road area. Furthermore, a large number of detection target images for constructing the area road map can be acquired relatively conveniently, improving the convenience and timeliness of area road map updates.

[0029] Figure 1 schematically shows an exemplary system architecture to which the map construction method and apparatus based on a large-scale model according to the embodiments of this disclosure can be applied.

[0030] Figure 1 is merely an example of a system architecture to which the embodiments of this disclosure can be applied, intended to help those skilled in the art understand the technical content of this disclosure. It does not mean that the embodiments of this disclosure cannot be applied to other devices, systems, environments, or scenes. For example, in another embodiment, an exemplary system architecture to which a large-scale model-based map building method and apparatus can be applied may include a terminal device, which can implement the large-scale model-based map building method and apparatus provided by the embodiments of this disclosure without interacting with a server.

[0031] As shown in Figure 1, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a vehicle 103, a network 104, and a server 105. The network 104 provides a medium for communication links between the first terminal device 101, the second terminal device 102, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links.

[0032] Users can use the first terminal device 101 and the second terminal device 102 to interact with the server 105 via the network 104 and send and receive messages, etc. Various communication client applications such as knowledge browsing applications, web browser applications, search applications, instant messaging tools, mailbox clients and / or social platform software may be installed on the first terminal device 101 and the second terminal device 102 (examples only).

[0033] The first terminal device 101 and the second terminal device 102 may be various electronic devices having a display and supporting web page browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers.

[0034] Vehicle 103 may be equipped with vehicle-side sensors, which may include, but are not limited to, image sensors, and may also be other types of sensors such as laser radar or millimeter-wave radar. Vehicle 103 may be any type of vehicle, such as a passenger car or a truck. Vehicle 103 may be equipped with communication equipment, and vehicle 103 may transmit information to and from server 105 via network 104 based on the communication equipment.

[0035] Server 105 may be a server that provides various services, for example, a background management server (example only) that supports content viewed by a user using the first terminal device 101 and the second terminal device 102. The background management server can perform processing such as analysis on received data such as user requests and feed back the processing results (for example, web pages, information, or data acquired or generated in response to user requests) to the terminal devices.

[0036] The server, also called a cloud computing server or cloud host, may be a cloud server, which is one of the host products in a cloud computing service system, and solves the problems of high management difficulty and low service scalability that exist in conventional physical hosts and VPS services ("Virtual Private Server," or "VPS"). The server may be a server in a distributed system, or a server that incorporates blockchain.

[0037] The map construction method based on the large-scale model provided in the embodiments of this disclosure may generally be executed by server 105. Accordingly, the map construction device based on the large-scale model provided in the embodiments of this disclosure may generally be located on server 105. The map construction method based on the large-scale model provided in the embodiments of this disclosure may be executed by a server or server cluster that is not server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the vehicle 103 and / or server 105. Accordingly, the map construction device based on the large-scale model provided in the embodiments of this disclosure may be not server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the vehicle 103 and / or server 105 and is located on a server or server cluster.

[0038] It should be understood that the number of terminal devices, vehicles, networks, and servers in Figure 1 are merely illustrative. Any number of terminal devices, vehicles, networks, and servers may be used as needed for implementation.

[0039] Figure 2 schematically shows a flowchart of a map construction method based on a large-scale model according to an embodiment of this disclosure.

[0040] As shown in Figure 2, the map construction method based on this large-scale model includes operations S210 to S240.

[0041] In operation S210, the relevant area lane attributes and the detected target image collected by the vehicle-side sensor are acquired.

[0042] In operation S220, target presentation information is constructed based on the relevant area lane attributes.

[0043] In operation S230, a large-scale model is used to process target presentation information and detection target images to obtain an area road map of the detection target road area.

[0044] According to embodiments of this disclosure, the image to be detected may represent a road area to be detected, and may be, for example, a single frame or multiple frames obtained by image collecting the road area to be detected. The vehicle-side sensor may include any type of image acquisition device mounted on the vehicle, such as a monocular camera, a surround-view camera, or a depth camera. The image to be detected may include any type of image, such as a color image or a grayscale image.

[0045] According to the embodiments of this disclosure, the relevant area lane attribute corresponds to the relevant road area, and a predetermined similarity condition is met between the relevant road area and the target road area. The relevant road area may include an area that at least partially overlaps with the target road area, or the relevant road area may further include road areas adjacent to the target road area, for example, road areas adjacent to the target road area, or road areas within a predetermined distance range from the target road area. Alternatively, the relevant road area may further include areas that have a predetermined traffic relationship with the target road area, for example, the target road area and the relevant road area belong to the same traffic link (e.g., the same highway, the same elevated traffic route). The embodiments of this disclosure do not limit the specific method of setting the predetermined similarity condition and can be designed according to actual needs, as long as they satisfy those actual needs.

[0046] According to embodiments of this disclosure, the relevant area lane attributes may include attributes of lane boundary lines in the relevant road area, such as lane shape, lane type, location, and lane boundary line topology relationships. The lane type may include the direction of travel indicated by the lane boundary line (e.g., left turn, forward, etc.), or the traffic rules indicated by the lane boundary line, such as permission for vehicles to cut in or indication of oncoming traffic.

[0047] According to embodiments of this disclosure, the target-providing information may be a serialized identifier or tensor that helps the large-scale model understand the detection task. Based on the included relevant area lane attributes, the target-providing information can help the large-scale model detect area lane attributes in the target image with relative accuracy, thereby avoiding obvious attribute conflicts (e.g., changes in lane boundary indication direction) between area lane attributes and related area lane attributes in the area lane detection result, and improving the detection accuracy for lane boundaries in the target road area.

[0048] According to the embodiments of this disclosure, the large-scale model may be a model with a large number of parameters, with the parameter order generally being on the order of tens of millions, hundreds of millions, or more, and potentially reaching the order of billions or tens of billions. The network structure of the large-scale model can employ, for example, a network structure such as UFO (Unified Feature Optimization). By processing the target image and target presentation information with the large-scale model, the strong feature understanding and representation capabilities of the large-scale model can be demonstrated, enabling efficient processing of target images with large data sizes and improving the detection accuracy and detection efficiency of area road maps.

[0049] According to embodiments of this disclosure, the related area lane attribute includes the related area lane location and the related area lane type. The related area lane location may include the coordinate location of the lane boundary line. The related area lane type may include any type, such as the color of the lane boundary line in the related area or the type of traffic rule instruction.

[0050] In one example, the relevant area lane position may be the coordinates of one or more points on the lane boundary line within the relevant area.

[0051] According to embodiments of this disclosure, constructing target presentation information based on relevant area lane attributes may include performing feature fusion on relevant area lane locations and relevant area lane types to obtain relevant lane location and type features, and constructing target presentation information based on relevant lane location and type features.

[0052] According to embodiments of the present disclosure, feature fusion for related area lane locations and related area lane types may include processing related area lane locations and related area lane types based on a fusion algorithm, the fusion algorithm may include, for example, matrix multiplication, concatenation, attention algorithms, etc., and embodiments of the present disclosure do not limit the specific algorithm type of the fusion algorithm.

[0053] Furthermore, by performing processes such as tokenization and embedding on the relevant area lane location and related area lane type, feature vectors representing the relevant area lane location and related area lane type can be obtained. Subsequently, feature fusion is performed on the feature vectors.

[0054] According to embodiments of the present disclosure, constructing target presentation information based on relevant lane location and type features may further include constructing target presentation information based on relevant lane location and type features, and features representing other relevant area lane attributes.

[0055] In one example, the target presentation information may include related lane spatial features obtained by extracting spatial features from related area lane attributes, as well as related lane location and type features. Related area lane attributes can be represented based on an SD map (Standard Definition Map), and related lane spatial features can be obtained by processing the SD map based on any type of neural network algorithm, such as a convolutional neural network.

[0056] In one example, by processing related area lane attributes based on FPN (Feature Pyramid Networks), multiple levels of feature extraction and fusion are performed on the spatial features of the related area lane attributes, and the resulting related lane spatial features can relatively accurately represent the spatial semantic information of lane boundaries in the related road area.

[0057] For example, constructing target presentation information based on the relevant lane location and type characteristics may include using the relevant lane location and type characteristics as target presentation information.

[0058] According to embodiments of this disclosure, there may be multiple related road areas, and the related area lane attributes of the multiple related road areas correspond to multiple related lane location and type features.

[0059] According to embodiments of this disclosure, predetermined similarity conditions can be met between multiple related road areas and the road area to be detected. For example, predetermined distance conditions, predetermined traffic relationship conditions, etc., can be met, and the embodiments of this disclosure are omitted from this description.

[0060] According to embodiments of the present disclosure, a plurality of related lane position and type features may include at least one reference position and type feature, the reference position and type feature may be a related lane position and type feature obtained by performing the method provided by embodiments of the present disclosure based on other target images. For example, in the process of performing the method provided by embodiments of the present disclosure multiple times, the current i-th related lane position and type feature can be obtained in the current i-th execution, and the related lane position and type features obtained from the 1st to the (i-1)th execution can be used as the reference position and type feature.

[0061] According to embodiments of this disclosure, constructing target presentation information based on relevant lane location and type features may include identifying the respective relevance weights of multiple relevant lane location and type features based on at least one relevant lane location and type feature, and fusing the multiple relevant lane location and type features based on the multiple relevance weights to obtain target presentation information.

[0062] According to embodiments of this disclosure, the relevance weight can represent the degree of relevance between a related road area and a road area to be detected. For example, the degree of relevance can represent the degree of area overlap between the related road area and the road area to be detected. However, it is not limited to this, and the degree of relevance can also represent the distance between separated related road areas and road areas to be detected, or it can represent other related relationships, such as road traffic relationships between the related road area and the road area to be detected. Road traffic relationships may, for example, represent relationships such as convergence, branching, and connection between lane boundaries.

[0063] The relevance weights can be represented based on any method, such as a matrix or numerical values, and the embodiments of this disclosure are not limited thereto.

[0064] According to embodiments of this disclosure, fusing multiple related lane location and type features based on multiple relevance weights may include performing a weighted average calculation based on the respective relevance weights of the multiple related lane location and type features to obtain a fused related area feature. Target presentation information is identified based on the fused related area feature.

[0065] According to embodiments of this disclosure, fusing multiple related lane location and type features based on multiple relevance weights may further include multiplying the relevance weights by the related lane location and type features to obtain multiple initial related area fusion features. Processing the multiple initial related area fusion features based on an attention network algorithm or other type of neural network algorithm to obtain a fused related area fusion feature. Identifying target presentation information based on the fused related area fusion feature.

[0066] According to embodiments of this disclosure, identifying target presentation information based on the relevant area fusion features after fusion may further include constructing target presentation information based on the relevant area fusion features and features representing other relevant area lane attributes.

[0067] In one example, target presentation information can be constructed based on related area fusion features and related lane space features. The target presentation information may include related area fusion features and related lane space features.

[0068] According to embodiments of this disclosure, large-scale models may include an encoder and a decoder. The encoder and decoder can be constructed based on an attention network algorithm. For example, the encoder and decoder can be constructed based on a self-attention network algorithm.

[0069] According to embodiments of this disclosure, obtaining an area road map of a target road area by processing target presentation information and detection target images using a large-scale model may include fusing detection target features and target presentation information using an encoder to obtain detection target area features in a bird's-eye view space, and processing target presentation information and detection target area features using a decoder based on an attention mechanism to obtain an area road map.

[0070] According to embodiments of this disclosure, the features to be detected may be identified based on the image to be detected. For example, the features to be detected can be obtained by processing the image to be detected based on any type of network algorithm, such as a convolutional neural network algorithm. The features to be detected can represent semantic information of data collected by vehicle sensors, thereby allowing lane attributes in the road area to be detected to be represented based on the features to be detected.

[0071] According to the embodiments of this disclosure, the detected feature represents the road area to be detected based on the data acquisition viewing angle of the vehicle-side sensor, and the detected feature and target presentation information can be fused using an encoder. Based on a predetermined bird's-eye view (referred to as Bird's Eye View, BEV, or bird's-eye viewing angle) space, a viewing angle transformation can be performed on the detected feature at the vehicle-side sensor's viewing angle, and the attribute semantic information of the detected feature and target presentation information can be fused. In this way, the detected area feature in the obtained bird's-eye view space can represent the area lane attribute of the lane boundary line in the road area to be detected from the bird's-eye viewing angle, avoiding area lane attribute representation errors due to factors such as occlusion and light irradiation in the detected image acquired by the vehicle-side sensor based on the vehicle-side acquisition viewing angle, and improving the detection accuracy of the subsequent area road map based on the detected area feature.

[0072] According to embodiments of this disclosure, the detection target area features may be generated based on multiple detection target images detected by each vehicle-side sensor of multiple vehicles. By identifying the detection target features corresponding to each of the multiple vehicle-side sensors, the multiple detection target features can represent the detection target road area from each data collection viewing angle of multiple vehicles. Furthermore, the detection target area features can represent the area lane attributes of the detection target road area based on the semantic attributes of the multiple collection viewing angles, thereby improving the accuracy of the representation of the detection target road area by the detection target area features in a bird's-eye view space, and further improving the detection accuracy of the area road map.

[0073] According to embodiments of this disclosure, based on an attention mechanism, a decoder is used to process target presentation information and detection target area features, and based on the target presentation information, the decoder of the large model is again helped to understand semantic information representing area lane attributes in the detection target area features. This reduces attribute collisions between area lane attributes represented by the area road map and related area lane attributes, thereby improving the detection accuracy and map generation efficiency of the area road map.

[0074] According to the embodiments of this disclosure, a decoder is used to process the features of the area to be detected and some or all of the target presentation information to obtain an area road map.

[0075] According to embodiments of this disclosure, the features to be detected are multimodal features to be detected, the multimodal features to be detected are obtained by performing feature extraction on multimodal information to be detected, the multimodal information to be detected is detected by a vehicle-side sensor located in the road area to be detected, and the multimodal information to be detected includes the image to be detected.

[0076] According to embodiments of this disclosure, the multimodal detection target information may include a detection target image and a detection target point cloud. The detection target point cloud may include modals such as laser radar point clouds and millimeter-wave radar point clouds. The detection target point cloud may be collected by vehicle-side sensors such as vehicle laser radar and millimeter-wave radar. There can be spatial relationships between detection target information of different modals, thereby enabling alignment of multimodal detection target information, and also enabling alignment of multiple multimodal detection target information acquired from multiple vehicles.

[0077] The embodiments of this disclosure allow for feature extraction from multimodal detection target information based on any type of deep learning algorithm, such as a convolutional neural network or a feature pyramid network, and the embodiments of this disclosure are not limited thereto.

[0078] According to embodiments of this disclosure, obtaining a target area feature in a bird's-eye view by fusing a target feature and target presentation information using an encoder may further include obtaining a target area feature by fusing a multimodal target feature and target presentation information using an encoder. In this way, the multimodal target feature is mapped to a bird's-eye view, and the target area feature in the bird's-eye view can be fusing with the relevant lane attributes of the relevant road area represented by the target presentation information, thereby improving the accuracy of the representation of area lanes in the target road area by the target area feature.

[0079] According to embodiments of this disclosure, the large-scale model may further include a detection head. The detection head may be constructed based on a neural network algorithm. For example, the detection head can be constructed based on an attention network algorithm, a convolutional neural network algorithm, or a fully connected layer.

[0080] Furthermore, the embodiments of this disclosure, in order to illustrate the data processing process of a large-scale model, may be divided into an encoder, decoder, and detection head, and are not intended to limit the boundaries of the specific network structure of the large-scale model. It can be understood that the detection head of the large-scale model may be included in the decoder.

[0081] According to embodiments of this disclosure, processing target presentation information and detection target area features using a decoder based on an attention mechanism may include, based on an attention mechanism, fusing lane attribute query features, target presentation information, and detection target area features using a decoder to obtain target fusion features, and processing the target fusion features using a detection head to obtain an area road map.

[0082] According to embodiments of this disclosure, lane attribute query features may be obtained by fine-tuning a large-scale model. For example, lane attribute query features can be obtained by adjusting the initialized lane attribute query feature tensor during the process of fine-tuning the model parameters of a large-scale model.

[0083] According to the embodiments of this disclosure, lane attribute query features can be used as query features, and key features and value features can be identified based on target presentation information and detection target area features. Furthermore, by processing the query features, key features, and value features using decoder pairs based on a cross-attention mechanism, semantic information between the target presentation information, detection target area features, and lane attribute query features can be sufficiently fused. Moreover, under conditions of strong lane attribute understanding and representation capabilities of large-scale models, the obtained target fused features can relatively accurately represent area lane attributes in the detection target road area, thereby improving the detection accuracy of area road maps.

[0084] Figure 3 schematically shows the principle of a map construction method based on a large-scale model according to an embodiment of this disclosure.

[0085] As shown in Figure 3, the road area Q300 to be detected may include multiple vehicles traveling in the area lane, and each of these vehicles may be equipped with a vehicle-side sensor. Each vehicle-side sensor of the multiple vehicles can collect multimodal detection target information, such as detection target images and detection target point clouds. By acquiring the information collected by each vehicle-side sensor of the multiple vehicles in the road area Q300 to be detected, multiple detection target information P301 for input into a large-scale model can be obtained. Based on the location of the road area Q300 to be detected, multiple related area lane attributes A301 that satisfy predetermined similarity conditions with respect to the road area Q300 to be detected can also be identified.

[0086] As shown in Figure 3, by inputting multiple related area lane attributes A301 and multiple detection target information P301 into the feature extraction network, target presentation information and multiple detection target features can be obtained. The multiple detection target features and target presentation information are input into the large-scale model encoder 310, which can output the detection target area features F301 in the bird's-eye view space.

[0087] As shown in Figure 3, the road area Q300 to be detected may be divided into multiple grid sub-areas arranged in an array. For the first detection target information P3011 and the second detection target information P3012, which represent the target grid sub-area among the multiple grid sub-areas, the encoder 310 can map the detection target information collected at the vehicle's side viewing angle to a bird's-eye view space and obtain the detection target area sub-features F3011 in the bird's-eye view space. After obtaining the detection target area sub-features in the bird's-eye view space corresponding to each of the multiple grids, it can be understood that the detection target area feature F301 in the bird's-eye view space can be identified.

[0088] Furthermore, the first detection target information P3011 and the second detection target information P3012 may be collected by vehicle-side sensors of different vehicles in the detection target road area Q300. This allows for the representation of area lanes in the detection target road area from different vehicle-side viewing angles. By fusing detection target features corresponding to multiple vehicle-side viewing angles and mapping them to a bird's-eye view space, detection target area features can be obtained, enabling a relatively accurate representation of the area lane attributes of the detection target road area.

[0089] As shown in Figure 3, the detection target area features F301 and target presentation information are input to the decoder 320 of the large-scale model, and target fusion features can be output. The target fusion features are input to the detection head 330 of the large-scale model, and an area road map G301 including arbitrary area lane attributes such as area lane boundary lines, area lane boundary line topology relationships, and area lane boundary line types can be output.

[0090] As shown in Figure 3, the detection target area feature F301 output by the encoder 310 may be used to update the area feature library K310. The area feature library K310 can store detection target area features representing the detection target road area Q300 generated in multiple historical time periods.

[0091] As shown in Figure 3, the current target area feature F301 can be obtained by fusing the target area features generated during multiple time periods stored in the area feature library K310. Furthermore, the target road area Q300 can be detected relatively accurately based on the fusing target area feature F301, thereby improving the accuracy of the area road map.

[0092] According to embodiments of this disclosure, the target presentation information may include related lane spatial features and related lane location and type features, wherein the related lane spatial features are obtained by performing spatial feature extraction on related area lane attributes, the related lane location and type features are identified based on related area lane location and related area lane type, and the related area lane attributes include related area lane location and related area lane type.

[0093] According to embodiments of this disclosure, obtaining a target area feature by fusing a target feature and target presentation information using an encoder may include fusing a target feature and associated lane spatial features using a first attention network of the encoder to obtain a spatial fusion feature in a bird's-eye view space, and fusing the spatial fusion feature and associated lane position and type features using a second attention network of the encoder to obtain a target area feature.

[0094] According to embodiments of this disclosure, the first and second attention networks may be constructed based on a cross-attention network algorithm. By fusing the target feature and related lane spatial features using the first network, the spatial fusion feature can be made to learn semantic information of area lane attributes within the target road area in a bird's-eye view space with relative accuracy based on the cross-attention mechanism. Subsequently, the spatial fusion feature and related lane position and type features are fusing based on the second attention network, which helps the spatial fusion feature in a bird's-eye view space to learn area lane attributes of area lanes in the target road area through the position and type of lanes in the related road area. This enables the target area feature to represent important lane semantic attributes such as type, position, and topological relationship of area lanes with relative accuracy, further improving the detection accuracy and precision of subsequent area lane maps.

[0095] Figure 4 schematically shows the principle diagram of an encoder according to an embodiment of this disclosure.

[0096] As shown in Figure 4, the first attention network of the encoder 410 may include a first self-attention network layer 411 and a first cross-attention network layer 412, and the second attention network of the encoder 410 may include a second cross-attention network layer 421. The related area lane attribute can be represented based on a standard-resolution map A401 of the related road area. The standard-resolution map A401 is input to a spatial feature extraction network 401, which outputs related lane spatial features F401. The spatial feature extraction network 401 may be constructed based on a convolutional neural network algorithm. The standard-resolution map A401 is input to a feature coding network 402, which outputs related lane position and type features F402. The feature coding network 402 may be constructed based on the position coding layer of a transformer model, which allows the acquired related lane position and type features F402 to be represented as a token sequence. It should be understood that the target presentation information may include related lane spatial features F401 and related lane position and type features F402.

[0097] As shown in Figure 4, the related lane spatial feature F401 and the encoder query feature F411 are fused and input to the first self-attention network layer 411 to achieve self-attention enhancement for the related lane spatial feature and obtain the enhanced related lane spatial feature. The enhanced related lane spatial feature is used as the query feature (key), and the detection target feature F403 is input as the value feature and key feature to the first cross-attention network layer 412 to output the spatial fusion feature. The spatial fusion feature is used as the query feature, and the related lane position and type feature F402 is input as the value feature and key feature to the second cross-attention network layer 421 to output the detection target area feature F410'.

[0098] Note that encoder query feature F411 may be obtained by fine-tuning a large-scale model.

[0099] In one example, the relevant lane spatial features and the detection target area features from the target presentation information can be input to the decoder, and target fusion features can be output.

[0100] According to embodiments of this disclosure, processing target presentation information and detection target area features using a decoder based on an attention mechanism may include processing lane attribute query features, key features, and value features using a decoder based on an attention mechanism.

[0101] According to embodiments of this disclosure, key features and value features are identified based on the target area features and associated lane space features. For example, the target area features and associated lane space features can be merged to obtain a target area fused feature, and this target area fused feature can be used as the key features and value features.

[0102] According to the embodiments of this disclosure, lane attribute query features are obtained by fine-tuning a large-scale model.

[0103] According to embodiments of this disclosure, the detection head may include a lane boundary detection head.

[0104] According to embodiments of this disclosure, obtaining an area road map by processing target fusion features using a detection head may also include obtaining an area lane boundary by processing target fusion features using a lane boundary detection head.

[0105] According to embodiments of this disclosure, the area road map is identified based on area lane detection results, and the area lane detection results include area lane boundaries.

[0106] According to embodiments of this disclosure, the area lane boundary line may include the trajectory of the lane boundary line within the area lane, and may further include traffic rules such as the direction of travel indicated by the lane boundary line and the vehicle cutting method. The area lane boundary line can be represented based on vectors.

[0107] According to embodiments of the present disclosure, the detection head may further include a lane topology relationship detection head.

[0108] According to embodiments of this disclosure, obtaining an area road map by processing target fusion features using a detection head may also include obtaining area lane topology relationships by processing area lane boundary lines and target presentation information using a lane topology relationship detection head.

[0109] According to embodiments of this disclosure, the area lane detection result further includes area lane topology relationships. By processing the lane boundary line detection result and target presentation information using a lane topology relationship detection head, the detection of area lane topology relationships between lane boundaries in the target road area can be supported using the related lane attributes of the related road area represented by the target presentation information, thereby improving the detection accuracy of area lane topology relationships.

[0110] According to embodiments of the present disclosure, the detection head may further include a lane group detection head.

[0111] According to embodiments of the present disclosure, processing target fusion features using a detection head to obtain an area road map may further include processing target fusion features using a lane group detection head to obtain an area lane group.

[0112] According to embodiments of the present disclosure, the area road map is identified based on area lane detection results, which include area lane groups representing the relationship between at least two lanes in the detected road area. For example, it may represent two adjacent area lanes having the same direction of travel.

[0113] According to embodiments of this disclosure, the detection head may further include a lane difference detection head.

[0114] According to embodiments of this disclosure, processing target fusion features using a detection head to obtain an area road map may further include processing target fusion features using a lane difference detection head to obtain target difference information.

[0115] According to embodiments of this disclosure, the target difference information indicates that the degree of difference between the relevant area lane attribute and the area lane attribute of the detected road area satisfies a predetermined difference condition, and the area lane detection result further includes the target difference information.

[0116] For example, target difference information can indicate that the difference between the area lane attributes in the currently detected road area and the related area lane attributes representing the related road area is too large. This suggests that the related area lane attributes may not accurately represent the current related road area due to reasons such as road construction or changes in road driving rules. In this way, the detection target information (including at least one of the detection target image and detection target point cloud) collected by the vehicle's sensors can be used to identify related area lane attributes that need updating with relatively high accuracy. Furthermore, a related area road map can be generated using at least one of the detection target image and detection target point cloud related to the related road area, enabling accurate updates of map information and improving the accuracy and safety of the vehicle's autonomous driving function in multiple related road areas.

[0117] According to embodiments of the present disclosure, a map construction method based on a large-scale model may further include updating at least one related area lane attribute among related area lane boundaries, related area lane topology relationships, and related area lane groups based on area lane detection results when target difference information is obtained.

[0118] For example, it is possible to update the attributes of lane boundaries, such as lane position and lane type, of related area lane boundaries based on area lane boundaries, and to further enable rapid calibration and updating of related area lane boundaries.

[0119] In one example, related area lane topology relationships can be updated based on area lane topology relationships.

[0120] For example, the relevant area lane group may be updated based on the area lane group.

[0121] It should be understood that the corresponding related area lane attributes can be updated based on at least one of the area lane boundary lines, area lane topology relationships, and area lane groups. In this way, it is possible to update the related area lane attributes of the related road area in real time using detection target information acquired by highly real-time vehicle-side sensors, thereby improving the efficiency of lane attribute updates for the entire road traffic area including the detection target road area and related road areas, and improving vehicle traffic safety and traffic efficiency within the entire road traffic area.

[0122] Figure 5 schematically shows a principle diagram of a map construction method based on a large-scale model according to another embodiment of this disclosure.

[0123] As shown in Figure 5, the decoder 520 in the large-scale model may include a decoder self-attention network 521 and a decoder attention network 522. The detection head may include a lane boundary detection head 531, a lane topology relationship detection head 532, a lane group detection head 533, a lane difference detection head 534, and a lane boundary division detection head 535. The decoder attention network 522 may be constructed based on a cross-attention network algorithm.

[0124] As shown in Figure 5, the lane attribute query feature F511 obtained after fine-tuning can be input to the decoder self-attention network 521 to obtain the enhanced query feature. After fusing the detection target area feature F501 and the target presentation information F502, key features and value features are generated. The enhanced query feature, key features, and value features are input to the decoder attention network 522 to obtain the target fusion feature F521.

[0125] As shown in Figure 5, the target fusion feature F521 is input to the lane boundary detection head 531 to obtain an area lane boundary (polyline). The area lane boundary and target presentation information F502 can be input to the lane topology relationship detection head 532 to obtain an area lane topology relationship. The target fusion feature F521 is input to the lane group detection head 533 to obtain an area lane group. The target fusion feature F521 is input to the lane difference detection head 534 to obtain target difference information. The target fusion feature F521 and the detected target area feature 501 are input to the lane boundary division detection head 535 to output a division area between the foreground and background in the detection target image, and furthermore, the area lane boundary can be represented based on the division area.

[0126] Note that the detection target area features 501 and target presentation information 502 enclosed by the dotted line frame in Figure 5 may be the same as the detection target area features 501 and target presentation information 502 enclosed by the solid line frame.

[0127] In one example, the lane attribute query feature may include three query feature vectors: a lane boundary query feature, a lane group query feature, and a lane boundary division query feature. By inputting different query feature vectors into different decoder self-attention networks, and using the three enhanced query feature vectors, attention fusion can be performed with the detection target area feature and target presentation information, respectively, to obtain three different initial fused features. By superimposing the different initial fused features, a target fused feature can be obtained.

[0128] According to embodiments of the present disclosure, the map construction method based on a large-scale model may further include constructing a target navigation map based on area road maps of multiple target road areas.

[0129] According to embodiments of this disclosure, multiple road areas to be detected may belong to the same traffic area, for example, to the same work area or the same city's traffic area. By aggregating the road maps of each of the multiple road areas to be detected, a target navigation map representing the overall traffic area can be constructed, and this target navigation map can be used to support the autonomous vehicle in performing its autonomous driving functions, thereby improving the vehicle's driving efficiency and safety.

[0130] Based on the map construction method based on the large-scale model described above, embodiments of this disclosure further provide a vehicle control method. This will be described below with reference to the drawings and embodiments.

[0131] Furthermore, the process of acquiring, collecting, or processing information related to the embodiments of this disclosure shall be carried out on the condition that permission is obtained from the relevant user or organization, and the relevant user or organization shall be clearly notified that the purpose of the methods provided in the embodiments of this disclosure is to improve the driving safety and driving efficiency of vehicles. The methods provided in the embodiments of this disclosure shall employ necessary encryption or desensitization measures to the acquired information in order to prevent information leakage, and the acquired information shall include, but is not limited to, detection target information and relevant area lane attributes.

[0132] Figure 6 schematically shows a flowchart of a vehicle control method according to an embodiment of the present disclosure.

[0133] As shown in Figure 6, the vehicle control method may include operation S610.

[0134] In operation S610, the vehicle's movement is controlled based on the road map.

[0135] According to embodiments of this disclosure, the road map may be constructed by a map construction method based on a large-scale model provided by embodiments of this disclosure. For example, the road map may be an area road map obtained by a map construction method based on a large-scale model provided by embodiments of this disclosure. Alternatively, for example, the road map may be a target navigation map obtained by a map construction method based on a large-scale model provided by embodiments of this disclosure.

[0136] According to the embodiments of this disclosure, by obtaining road maps identified by a map construction method based on a large-scale model provided in the embodiments of this disclosure, vehicles can complete lane-level autonomous driving functions with the support of relatively accurate and timely road maps, thereby improving the safety and efficiency of vehicle operation.

[0137] Figure 7 schematically shows a block diagram of a map construction device based on a large-scale model according to an embodiment of this disclosure.

[0138] As shown in Figure 7, the map construction device 700 based on a large-scale model may include an acquisition module 710, a target presentation information construction module 720, and a first map construction module 730.

[0139] The acquisition module 710 is used to acquire the relevant area lane attributes and the detected target image collected by the vehicle-side sensor, where the detected target image represents the detected target road area, the relevant area lane attributes correspond to the relevant road area, and a predetermined similarity condition is met between the relevant road area and the detected target road area.

[0140] The target presentation information construction module 720 is used to construct target presentation information based on the relevant area lane attributes.

[0141] The first map construction module 730 is used to process target presentation information and detection target images using a large-scale model to obtain an area road map of the target road area.

[0142] According to the embodiments of this disclosure, the large-scale model includes an encoder and a decoder.

[0143] According to embodiments of this disclosure, the first map construction module includes a detection target area feature acquisition submodule and an area road map acquisition submodule.

[0144] The detection target area feature acquisition submodule is used to acquire detection target area features in a bird's-eye view space by fusing detection target features and target presentation information using an encoder, where the detection target features are identified based on the detection target image.

[0145] The area road map acquisition submodule uses an attention mechanism to process target presentation information and detected area features using a decoder to acquire an area road map.

[0146] According to the embodiments of this disclosure, the large-scale model further includes a detection head.

[0147] According to the embodiments of this disclosure, the area road map acquisition submodule includes a target fusion feature acquisition means and an area road map acquisition means.

[0148] The target fusion feature acquisition means is used to acquire target fusion features by fusing lane attribute query features, target presentation information, and detection target area features using a decoder based on an attention mechanism. Here, the lane attribute query features are obtained by fine-tuning a large-scale model.

[0149] The area road map acquisition method uses a detection head to process target fusion features and acquire area road maps.

[0150] According to embodiments of this disclosure, the detection head includes a lane boundary detection head.

[0151] According to embodiments of this disclosure, the area road map acquisition means includes an area lane boundary acquisition sub-means.

[0152] The area lane boundary acquisition sub-means are used to acquire area lane boundaries by processing target fusion features using a lane boundary detection head, where the area road map is identified based on the area lane detection result, and the area lane detection result includes the area lane boundaries.

[0153] According to embodiments of the present disclosure, the detection head further includes a lane topology relationship detection head.

[0154] According to embodiments of the present disclosure, the area road map acquisition means further includes area lane topology relationship acquisition sub-means.

[0155] The area lane topology relationship acquisition sub-means are used to acquire area lane topology relationships by processing area lane boundary lines and target presentation information using a lane topology relationship detection head, where the area lane detection result further includes area lane topology relationships.

[0156] According to embodiments of this disclosure, the detection head includes a lane group detection head.

[0157] According to embodiments of this disclosure, the area road map acquisition means includes an area lane group acquisition sub-means.

[0158] The area lane group acquisition sub-means are used to acquire area lane groups by processing target fusion features using a lane group detection head, where the area road map is identified based on the area lane detection results, and the area lane detection results include area lane groups that represent the relationship between at least two lanes in the road area to be detected.

[0159] According to embodiments of this disclosure, the detection head includes a lane difference detection head.

[0160] According to the embodiments of this disclosure, the area road map acquisition means includes a target difference information acquisition sub-means.

[0161] The target difference information acquisition sub-means are used to acquire target difference information by processing target fusion features using a lane difference detection head, where the target difference information indicates that the degree of difference between the relevant area lane attribute and the area lane attribute of the detected road area satisfies a predetermined difference condition, and the area lane detection result further includes target difference information.

[0162] According to the embodiments of this disclosure, the map building device 700 based on a large-scale model further includes an update module.

[0163] The update module is used to update at least one of the related area lane attributes—related area lane boundary lines, related area lane topology relationships, and related area lane groups—based on the area lane detection results, once target difference information has been acquired.

[0164] According to embodiments of this disclosure, the target presentation information includes related lane spatial features and related lane location and type features, wherein the related lane spatial features are obtained by performing spatial feature extraction on related area lane attributes, the related lane location and type features are identified based on related area lane location and related area lane type, and the related area lane attributes include related area lane location and related area lane type.

[0165] According to the embodiments of this disclosure, the detection target area feature acquisition submodule includes spatial fusion feature acquisition means and detection target area feature acquisition means.

[0166] The spatial fusion feature acquisition means is used to acquire spatial fusion features in a bird's-eye view space by fusing the detected target feature with related lane spatial features using the encoder's first attention network.

[0167] The detection target area feature acquisition means is used to acquire the detection target area features by fusing spatial fusion features with related lane position and type features using the encoder's second attention network.

[0168] According to the embodiments of this disclosure, the area road map acquisition submodule includes processing means.

[0169] The processing means is used to process lane attribute query features, key features, and value features using a decoder based on an attention mechanism. Here, the key features and value features are identified based on the target area features and related lane spatial features, while the lane attribute query features are obtained by fine-tuning a large-scale model.

[0170] According to embodiments of this disclosure, the features to be detected are multimodal features to be detected, the multimodal features to be detected are obtained by performing feature extraction on multimodal information to be detected, the multimodal information to be detected is detected by a vehicle-side sensor located in the road area to be detected, and the multimodal information to be detected includes the image to be detected.

[0171] According to embodiments of this disclosure, the relevant area lane attributes include the relevant area lane location and the relevant area lane type.

[0172] According to the embodiments of this disclosure, the target presentation information construction module includes a feature fusion submodule and a target presentation information construction submodule.

[0173] The feature fusion submodule is used to perform feature fusion on the relevant area lane location and the relevant area lane type in order to obtain the relevant lane location and type features.

[0174] The target presentation information construction submodule is used to construct target presentation information based on the relevant lane location and type characteristics.

[0175] According to embodiments of this disclosure, a plurality of related road areas are included, and the related area lane attributes of the plurality of related road areas correspond to a plurality of related lane location and type features.

[0176] According to embodiments of this disclosure, the target presentation information construction submodule includes a relevance weight identification means and a target presentation information acquisition means.

[0177] The relevance weight identification means is used to identify the relevance weight of each of a plurality of related lane location and type features based on at least one related lane location and type feature, and the relevance weight represents the degree of relevance between the related road area and the road area to be detected.

[0178] The target presentation information acquisition means acquires target presentation information by fusing multiple related lane position and type features based on multiple relevance weights.

[0179] According to embodiments of this disclosure, the map-building apparatus 700 based on a large-scale model further includes a second map-building module.

[0180] The second map-building module is used to construct a target navigation map based on the area road maps of each of the multiple target road areas.

[0181] Figure 8 schematically shows a block diagram of a vehicle control device according to an embodiment of the present disclosure.

[0182] As shown in Figure 8, the vehicle control device 800 includes a vehicle control module 810.

[0183] The vehicle control module 810 is used to control the vehicle's movement based on a road map, which is constructed by a map-building device based on a large-scale model provided in the embodiments of this disclosure.

[0184] According to embodiments of the present disclosure, the present disclosure further provides electronic devices, readable storage media, and computer programs.

[0185] An embodiment of the present disclosure provides an electronic device comprising at least one processor and memory communicated with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a map building method based on a large model provided by an embodiment of the present disclosure.

[0186] According to embodiments of the present disclosure, an electronic device is provided which includes at least one processor and a memory communicated with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a vehicle control method provided by embodiments of the present disclosure.

[0187] According to embodiments of the present disclosure, an autonomous vehicle is provided, which includes electronic equipment for performing a vehicle control method provided by embodiments of the present disclosure.

[0188] According to embodiments of the present disclosure, a non-temporary computer-readable storage medium is provided in which computer instructions are stored, the computer instructions being used to cause a computer to perform a method provided by embodiments of the present disclosure.

[0189] According to embodiments of the present disclosure, a computer program is provided that, when executed by a processor, implements the method provided by embodiments of the present disclosure.

[0190] Figure 9 schematically shows a block diagram of electronic equipment suitable for implementing a map construction method and a vehicle control method based on a large-scale model according to an embodiment of the present disclosure. The electronic equipment is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, large computers, and other suitable computers. The electronic equipment may further represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are illustrative and do not limit the implementation of the present disclosure as described herein and / or requested.

[0191] As shown in Figure 9, the device 900 includes a computing unit 901, which may perform various appropriate operations and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. The RAM 903 may also store various programs and data necessary for the operation of the device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0192] Multiple components in the device 900 are connected to the I / O interface 905 and include, for example, an input unit 906 such as a keyboard or mouse; an output unit 907 such as various types of displays or speakers; a storage unit 908 such as a magnetic disk or optical disk; and a communication unit 909 such as a network card, modem, or wireless communication transceiver. The communication unit 909 enables the device 900 to exchange information and data with other devices via computer networks such as the Internet and / or various electrical networks.

[0193] The computing unit 901 may be various general-purpose and / or dedicated processing modules having processing and computational capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a GPU (Graphics Processing Unit), various dedicated artificial intelligence (AI) computing chips, computing units for running various machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 901 executes each of the methods and processes described in the preceding paragraph, such as a map construction method based on a large-scale model and a vehicle control method. For example, in some embodiments, the map construction method based on a large-scale model and the vehicle control method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed in the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the map construction method based on a large-scale model and the vehicle control method described in the preceding paragraph may be executed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a map building method and a vehicle control method based on a large-scale model by any other suitable method (e.g., via firmware).

[0194] Various embodiments of the systems and technologies described herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented in one or more computer programs which can be executed and / or interpreted on a programmable system which includes at least one programmable processor, which may be a dedicated or general-purpose programmable processor which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0195] Program code for carrying out the methods of this disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions and operations defined in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on a device, partially on a device, partially on a device as a standalone software package, partially on a remote device, or entirely on a remote device or server.

[0196] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores programs used in or in combination with an instruction execution system, device, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or electronic devices, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wired electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0197] To provide interaction with a user, a computer may be made to implement the systems and techniques described herein, the computer comprising a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor), a keyboard and a pointing device (e.g., a mouse or trackball), the user may provide input to the computer via the keyboard and the pointing device. Other types of devices may further provide interaction with the user, for example, feedback provided to the user may be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback), and input from the user may be received in any form (including voice input, speech input, or haptic input).

[0198] The systems and technologies described herein can be implemented in computing systems including background components (e.g., a data server), computing systems including middleware components (e.g., an application server), computing systems including front-end components (e.g., a user computer having a graphical user interface or a web browser, through which the user can interact with embodiments of the systems and technologies described herein), or in computing systems including any combination of such background components, middleware components, or front-end components. Components of the system can be connected to one another by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include, but are not limited to, local area networks (LANs), wide area networks (WANs), and the Internet.

[0199] A computer system may include clients and servers. Clients and servers are generally geographically separated and typically communicate via a communication network. The client-server relationship is generated by a computer program running on the relevant computer that has a client-server relationship. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0200] It should be understood that various forms of flows shown above may be used, and operations may be re-sorted, added, or deleted. For example, each operation described herein may be performed in parallel, sequentially, or in a different order, as long as the desired results of the proposed techniques disclosed herein can be achieved.

[0201] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. The process involves acquiring related area lane attributes and detection target images collected by vehicle-side sensors, wherein the detection target images represent the target road area, the related area lane attributes correspond to the related road area, and a predetermined similarity condition is met between the related road area and the detection target road area. Based on the aforementioned related area lane attributes, target presentation information is constructed, This includes processing the target presentation information and the detected target image using a large-scale model to obtain an area road map of the detected target road area, The aforementioned large-scale model includes an encoder and a decoder, Using a large-scale model to process the target presentation information and the detected target image to obtain an area road map of the detected target road area is: The method involves fusing the detection target features and the target presentation information using the encoder to obtain the detection target area features in the bird's-eye view space, wherein the detection target features are identified based on the detection target image. This includes processing the target presentation information and the detected target area features using the decoder based on the attention mechanism to obtain the area road map, The aforementioned related area lane attributes include the related area lane location and the related area lane type. Building target presentation information based on the aforementioned related area lane attributes is: The features of the related area lane position and the related area lane type are fused to obtain the related lane position and type features, This includes constructing the target presentation information based on the aforementioned related lane location and type characteristics, A map construction method based on large-scale models.

2. The aforementioned large-scale model further includes a detection head, Processing the target presentation information and the detected target area features using the decoder based on the attention mechanism is: Based on the attention mechanism, the lane attribute query features, the target presentation information, and the detected target area features are fused using the decoder to obtain target fused features, wherein the lane attribute query features are obtained by fine-tuning the large-scale model. This includes processing the target fusion features using the detection head to obtain the area road map, The method according to claim 1.

3. The detection head includes a lane boundary detection head, Using the detection head to process the target fusion features and obtain the area road map is: The process includes processing the target fusion features using the lane boundary detection head to obtain the area lane boundary, wherein the area road map is identified based on the area lane detection result, and the area lane detection result includes the area lane boundary. The method according to claim 2.

4. The detection head further includes a lane topology relationship detection head, Using the detection head to process the target fusion features and obtain the area road map is: The method further includes processing the area lane boundary line and the target presentation information using the lane topology relationship detection head to obtain the area lane topology relationship, wherein the area lane detection result further includes the area lane topology relationship. The method according to claim 3.

5. The detection head includes a lane group detection head, Using the detection head to process the target fusion features and obtain the area road map is: The process includes processing the target fusion features using the lane group detection head to obtain an area lane group, wherein the area road map is identified based on the area lane detection result, the area lane detection result includes the area lane group, and the area lane group represents the relationship between at least two lanes in the target road area. The method according to claim 2.

6. The detection head includes a lane difference detection head, Using the detection head to process the target fusion features and obtain the area road map is: The process includes processing the target fusion features using the lane difference detection head to obtain target difference information, wherein the target difference information indicates that the degree of difference between the related area lane attribute and the area lane attribute of the road area to be detected satisfies a predetermined difference condition, and the area lane detection result further includes the target difference information. The method according to claim 3.

7. When the aforementioned target difference information is obtained, the process further includes updating the relevant area lane boundary line, the relevant area lane topology relationship, and at least one relevant area lane attribute of the relevant area lane group based on the area lane detection result. The method according to claim 6.

8. The aforementioned target presentation information includes related lane spatial features and related lane location and type features, wherein the related lane spatial features are obtained by performing spatial feature extraction on the related area lane attributes, the related lane location and type features are identified based on the related area lane location and related area lane type, and the related area lane attributes include the related area lane location and related area lane type. By fusing the detected target features and the target presentation information using the encoder, the detected target area features can be obtained. The detection target feature and the related lane spatial feature are fused using the first attention network of the encoder to obtain a spatial fusion feature in the bird's-eye view space. This includes merging the spatial fusion features with the associated lane position and type features using the second attention network of the encoder to obtain the detection target area features, The method according to claim 1.

9. Processing the target presentation information and the detected target area features using the decoder based on the attention mechanism is: The process includes processing lane attribute query features, key features, and value features using the decoder based on an attention mechanism, wherein the key features and value features are identified based on the target area features and the associated lane spatial features, and the lane attribute query features are obtained by fine-tuning the large-scale model. The method according to claim 8.

10. The aforementioned detection target feature is a multimodal detection target feature, which is obtained by performing feature extraction on multimodal detection target information, which is detected by a vehicle-side sensor located in the detection target road area, and which includes the detection target image. The method according to claim 1.

11. The aforementioned related road areas include multiple such areas, and the related area lane attributes of the multiple such related road areas correspond to multiple such related lane location and type features. Constructing the target presentation information based on the aforementioned related lane location and type characteristics is: This includes identifying the relevance weight of each of the multiple related lane location and type features based on at least one of the related lane location and type features, wherein the relevance weight represents the degree of relevance between the related road area and the detected road area. This includes merging multiple related lane position and type features based on multiple related weights to obtain the target presentation information, The method according to claim 1.

12. The further includes constructing a target navigation map based on the area road maps of each of the multiple road areas to be detected. The method according to claim 1.

13. The method includes controlling the movement of a vehicle based on a road map constructed by any one of claims 1 to 12. Vehicle control method.

14. The system acquires related area lane attributes and detection target images collected by vehicle-side sensors, wherein the detection target image represents the detection target road area, the related area lane attributes correspond to the related road area, and the acquisition module satisfies predetermined similarity conditions between the related road area and the detection target road area. A target presentation information construction module that constructs target presentation information based on the aforementioned related area lane attributes, The system includes a first map construction module that processes the target presentation information and the detected target image using a large-scale model to obtain an area road map of the detected target road area, The aforementioned large-scale model includes an encoder and a decoder, The first map construction module described above is: A submodule for acquiring detection target area features in a bird's-eye view space, which uses the encoder to fuse the detection target features and the target presentation information, wherein the detection target features are identified based on the detection target image, Includes an area road map acquisition submodule that processes the target presentation information and the detected target area features using the decoder based on an attention mechanism to acquire the area road map, The aforementioned related area lane attributes include the related area lane location and the related area lane type. The aforementioned target presentation information construction module is: A feature fusion submodule that performs feature fusion on the aforementioned related area lane position and the aforementioned related area lane type to obtain related lane position and type features, A target presentation information construction submodule that constructs the target presentation information based on the aforementioned related lane position and type characteristics, A map construction device based on large-scale models.

15. The aforementioned large-scale model further includes a detection head, The aforementioned area road map acquisition submodule is: Based on an attention mechanism, the system uses the decoder to fuse lane attribute query features, target presentation information, and detection target area features to obtain target fusion features, wherein the lane attribute query features are obtained by fine-tuning the large-scale model. The means includes an area road map acquisition means that processes the target fusion features using the detection head to acquire the area road map, The apparatus according to claim 14.

16. The detection head includes a lane boundary detection head, The means for acquiring the area road map is, The area lane boundary acquisition sub-means include processing the target fusion features using the lane boundary detection head to acquire the area lane boundary, the area road map being identified based on the area lane detection result, and the area lane detection result including the area lane boundary. The apparatus according to claim 15.

17. The detection head further includes a lane topology relationship detection head, The means for acquiring the area road map is, The system further includes an area lane topology relationship acquisition sub-means that processes the area lane boundary line and the target presentation information using the lane topology relationship detection head to acquire the area lane topology relationship, and the area lane detection result further includes the area lane topology relationship. The apparatus according to claim 16.

18. The detection head includes a lane group detection head, The means for acquiring the area road map is, The area lane group acquisition sub-means include processing the target fusion features using the lane group detection head to acquire an area lane group, the area road map is identified based on the area lane detection result, the area lane detection result includes the area lane group, and the area lane group represents the relationship between at least two lanes in the detected road area. The apparatus according to claim 15.

19. The detection head includes a lane difference detection head, The means for acquiring the area road map is, The system includes a target difference information acquisition sub-means that processes the target fusion features using the lane difference detection head to acquire target difference information, wherein the target difference information indicates that the degree of difference between the related area lane attribute and the area lane attribute of the detected road area satisfies a predetermined difference condition, and the area lane detection result further includes the target difference information. The apparatus according to claim 16.

20. When the aforementioned target difference information is obtained, the update module further includes updating the relevant area lane boundary line, the relevant area lane topology relationship, and at least one relevant area lane attribute of the relevant area lane group based on the area lane detection result, The apparatus according to claim 19.

21. The aforementioned target presentation information includes related lane spatial features and related lane location and type features, wherein the related lane spatial features are obtained by performing spatial feature extraction on the related area lane attributes, the related lane location and type features are identified based on the related area lane location and related area lane type, and the related area lane attributes include the related area lane location and related area lane type. The aforementioned submodule for acquiring features of the target area for detection is: A spatial fusion feature acquisition means that uses the first attention network of the encoder to fuse the detected target feature and the related lane spatial feature to acquire a spatial fusion feature in a bird's-eye view space, The system includes a means for acquiring detection target area features, which uses a second attention network of the encoder to fuse the spatial fusion features with the related lane position and type features to acquire the detection target area features. The apparatus according to claim 14.

22. The aforementioned area road map acquisition submodule is: The system includes processing means for processing lane attribute query features, key features, and value features using the decoder based on an attention mechanism, wherein the key features and value features are identified based on the detected target area features and the associated lane space features, and the lane attribute query features are obtained by fine-tuning the large-scale model. The apparatus according to claim 21.

23. The aforementioned detection target feature is a multimodal detection target feature, which is obtained by performing feature extraction on multimodal detection target information, which is detected by a vehicle-side sensor located in the detection target road area, and which includes the detection target image. The apparatus according to claim 14.

24. The aforementioned related road areas include multiple such areas, and the related area lane attributes of the multiple such related road areas correspond to multiple such related lane location and type features. The aforementioned target presentation information construction submodule is: A means for identifying the relevance weight of each of a plurality of related lane location and type features based on at least one of the related lane location and type features, wherein the relevance weight represents the degree of relevance between the related road area and the detected road area, A target presentation information acquisition means that acquires target presentation information by fusing a plurality of the aforementioned related lane position and type features based on a plurality of aforementioned related weights, The apparatus according to claim 14.

25. The system further includes a second map building module that constructs a target navigation map based on the area road maps of each of the multiple road areas to be detected. The apparatus according to claim 14.

26. A vehicle control module that controls the movement of a vehicle based on a road map constructed by the device described in any one of claims 14 to 25, Vehicle control system.

27. At least one processor, The memory includes at least one processor and is connected to it in communication, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 12. electronic equipment.

28. At least one processor, The memory includes at least one processor and is connected to it in communication, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to claim 13. electronic equipment.

29. The electronic device described in claim 28, Self-driving vehicles.

30. A non-temporary, computer-readable storage medium in which computer instructions are stored, The computer instruction is used to cause a computer to perform the method described in any one of claims 1 to 12. A non-temporary, computer-readable storage medium.

31. A computer program, when executed by a processor, that implements the method according to any one of claims 1 to 12.

32. A non-temporary, computer-readable storage medium in which computer instructions are stored, The computer instruction is used to cause a computer to perform the method described in claim 13. A non-temporary, computer-readable storage medium.

33. A computer program that, when executed by a processor, implements the method described in claim 13.

Images