Map construction method, map construction system, map construction device, vehicle, equipment and storage medium

By combining vehicle-mounted sensor data and cloud map features, high-precision maps are built using neural network models, the problem of low accuracy in real-time construction of intelligent vehicles is solved, and the precise description of road topology structure and display of detailed information is achieved, which improves vehicle driving Security.

CN119085692APending Publication Date: 2024-12-06杭州吉利汽车有限公司 +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411462322.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-18
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The maps built in real-time during driving of existing smart vehicles are difficult to achieve high accuracy, especially when the road topology description is not accurate enough, the lane connection relationship within the intersection, and the lane center line description is insufficient, and are greatly affected by factors such as visual obstruction and weather.

Method used

By obtaining the surrounding environment images and location information collected by on-board sensors on the vehicle, sending map acquisition requests to the cloud server, receiving cloud map features, and combining the vehicle-side neural network model for feature encoding and fusion, building a high-precision target map, displaying it in On the vehicle equipment.

Benefits of technology

It realizes an accurate description of the road topology structure, improves the accuracy of map construction, reduces the impact of weather and field of view occlusion, and allows the map to clearly display detailed information such as roadside lines and center lines, improving the safety of vehicle driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085692A_ABST
    Figure CN119085692A_ABST
Patent Text Reader

Abstract

The invention relates to a map construction method, a map construction system, a map construction device, a vehicle, equipment and a storage medium. The method comprises the steps of obtaining a surrounding environment image collected by a vehicle-mounted sensor on a vehicle and position information of the vehicle, sending a map obtaining request to a cloud server, receiving cloud map features sent by the cloud server, constructing a target map according to the surrounding environment image and the cloud map features, and sending the target map to the cloud server. The target map is displayed on a display interface on the vehicle-mounted equipment, and the map obtaining request carries the position information and the surrounding environment image. According to the method, when the vehicle-mounted equipment constructs the map online, the cloud map features can be obtained from the cloud server, and the target map is constructed in combination with the cloud map features and the surrounding environment images collected by the vehicle-mounted sensor, so that the quality of the target map is not affected by the environment where the vehicle is located, such as weather or view shielding, and the quality of the target map is improved. And the precision of constructing the target map can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a map construction method, a map construction system, an apparatus, a vehicle, a device and a storage medium. Background Art

[0002] With the continuous development of intelligent vehicle technology, intelligent vehicles can build vehicle environmental maps online in real time, so that vehicle drivers can refer to the environmental maps to make wise decisions while driving, thereby bypassing obstacles and avoiding accidents, which can improve the safety of intelligent vehicle driving.

[0003] In the related art, when a smart vehicle is driving, a laser radar or an environmental information sensor on the smart vehicle collects the surrounding environmental information in real time, and constructs an environmental map based on the environmental information in real time. However, the above-mentioned method of constructing a map in real time is difficult to construct a map with high accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a map construction method, map construction system, device, vehicle, equipment and storage medium that can construct a high-precision map in response to the above technical problems.

[0005] In a first aspect, the present application provides a map construction method. The method comprises:

[0006] Obtaining surrounding environment images collected by on-board sensors on the vehicle and location information of the vehicle;

[0007] Sending a map acquisition request to a cloud server, and receiving a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image;

[0008] Constructing a target map according to the surrounding environment image and the cloud map features;

[0009] The target map is displayed on a display interface on the vehicle-mounted device.

[0010] In one embodiment, constructing a target map according to the surrounding environment image and the cloud map features includes:

[0011] Inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features;

[0012] Fusing the environment map features with the cloud map features to obtain target fusion features;

[0013] The target fusion features are input into a map decoder to construct the target map.

[0014] In one embodiment, the vehicle-side neural network model includes a feature extraction network, a deep branch network, a feature conversion network, and a content semantic branch network; the surrounding environment image is input into the vehicle-side neural network model for feature encoding to obtain the environment map features, including:

[0015] Inputting the surrounding environment image into the feature extraction network to extract global features, so as to obtain global features of the surrounding environment image;

[0016] Inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, so as to obtain the environmental deep features of the surrounding environment image;

[0017] Inputting the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image;

[0018] The global features and the surrounding environment image are input into the content semantic branch network to extract semantic features, so as to obtain the environmental content features of the surrounding environment image.

[0019] In one embodiment, the target fusion feature includes a fusion depth feature, a fusion bird's-eye view feature and a fusion content feature, and the cloud map feature includes a cloud depth feature, a cloud bird's-eye view feature and a cloud content feature; and fusing the environment map feature with the cloud map feature to obtain the target fusion feature includes:

[0020] Inputting the environment depth feature and the cloud depth feature into a first feature fusion network for feature fusion to obtain the fused depth feature;

[0021] Inputting the environment bird's-eye view feature and the cloud bird's-eye view feature into a second feature fusion network for feature fusion to obtain the fused bird's-eye view feature;

[0022] The environmental content features and the cloud content features are input into a third feature fusion network for feature fusion to obtain the fused content features.

[0023] In one embodiment, the method further comprises:

[0024] Sending the navigation path of the vehicle to the cloud server;

[0025] Receiving intersection map features returned by the cloud server based on the navigation path and the location information, and constructing an intersection map according to the intersection map features;

[0026] The intersection map is displayed on the display interface.

[0027] In one embodiment, the method further comprises:

[0028] When the vehicle is started, determining whether the vehicle-mounted device meets a preset map building condition;

[0029] When the vehicle-mounted device meets the preset map building condition, the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor and the location information of the vehicle is performed;

[0030] The preset map construction condition includes any of the following:

[0031] The in-vehicle device responds to a triggering operation of a map generation control or a navigation control on a display interface of the in-vehicle device by a user;

[0032] The vehicle-mounted device is powered on and started;

[0033] The vehicle-mounted device detects that the vehicle travels to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0034] In a second aspect, a map construction method is provided, the method being applied to a cloud server, the method comprising:

[0035] Receiving a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle;

[0036] Determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0037] The cloud map feature is sent to the vehicle-mounted device; the cloud map feature is used to instruct the vehicle-mounted device to construct a target map based on the cloud map feature and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0038] In one embodiment, the method further comprises:

[0039] Receiving the navigation path sent by the vehicle-mounted device;

[0040] Determining the next intersection position of the vehicle according to the navigation path and the position information;

[0041] Extracting an intersection map feature corresponding to the next intersection position from the map feature library;

[0042] The intersection map feature is sent to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

[0043] In one embodiment, the method further comprises:

[0044] Receiving other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by on-board sensors on the other vehicles;

[0045] Inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features;

[0046] The other cloud map features are associated with the map location corresponding to the location information, and the other cloud map features are stored in the map feature library, and the association relationship between the other cloud map features and the map location is recorded in the map feature library.

[0047] In a third aspect, a map construction system is provided, the system comprising a vehicle-mounted sensor, a vehicle-mounted device, and a cloud server;

[0048] The vehicle-mounted device is used to execute the method according to the first aspect;

[0049] The cloud server is used to execute the method described in the second aspect.

[0050] In a fourth aspect, a map construction device is provided, the device comprising:

[0051] An acquisition module is used to acquire the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, as well as the location information of the vehicle;

[0052] A sending and receiving module, used to send a map acquisition request to a cloud server and receive a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image;

[0053] A construction module, used to construct a target map according to the surrounding environment image and the cloud map features;

[0054] A display module is used to display the target map on a display interface on the vehicle-mounted device.

[0055] In a fifth aspect, a map construction device is provided, the device comprising:

[0056] A receiving module, used to receive a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle;

[0057] A determination module, configured to determine a map location based on the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0058] A sending module is used to send the cloud map features to the vehicle-mounted device; the cloud map features are used to instruct the vehicle-mounted device to construct a target map based on the cloud map features and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0059] In a sixth aspect, a vehicle is provided, the vehicle comprising an on-board device, the on-board device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method described in the first aspect when executing the computer program.

[0060] In a seventh aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect or the second aspect when executing the computer program.

[0061] In an eighth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0062] In a ninth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect or the second aspect.

[0063] In the above-mentioned map construction method, map construction system, device, vehicle, equipment and storage medium, the vehicle-mounted device obtains the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, and the location information of the vehicle, sends a map acquisition request to the cloud server, and receives the cloud map features sent by the cloud server, constructs the target map according to the surrounding environment image and the cloud map features, and displays the target map on the display interface of the vehicle-mounted device, wherein the map acquisition request carries the location information and the surrounding environment image. When the vehicle-mounted device in the above-mentioned method constructs the map online, the cloud map features can be obtained from the cloud server, and the target map can be constructed by combining the cloud map features and the surrounding environment image collected by the vehicle-mounted sensor, so that the quality of the target map is not affected by the environment in which the vehicle is located, such as weather or field of view obstruction, etc., thereby improving the accuracy of constructing the target map. In addition, the fusion of cloud map features and surrounding environment images can be realized to improve the refinement of map features, so that the constructed map can clearly reflect the road sidelines, road centerlines and other detailed information in the road, thereby improving the accuracy of map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A diagram of an application environment of a map construction method in an embodiment;

[0065] Figure 2 One of the flowcharts of a map construction method in one embodiment;

[0066] Figure 2A One of the schematic diagrams of a map building interface in one embodiment;

[0067] Figure 3 The second flowchart of the map construction method in one embodiment;

[0068] Figure 4 A schematic diagram of the structure of a vehicle-side neural network model in one embodiment;

[0069] Figure 5 The third flowchart of a map construction method in one embodiment;

[0070] Figure 6 A schematic diagram of a network architecture in one embodiment;

[0071] Figure 7 The fourth flowchart of the map construction method in one embodiment;

[0072] Figure 8 FIG5 is a flowchart of a map construction method in one embodiment;

[0073] Fig. 8A The second schematic diagram of a map construction interface in one embodiment;

[0074] Fig. 9 The sixth flowchart of a map construction method in one embodiment;

[0075] Fig.10 FIG7 is a flowchart of a map construction method in one embodiment;

[0076] Fig.11 FIG8 is a flowchart of a map construction method in one embodiment;

[0077] Fig.12 FIG9 is a flowchart of a map construction method according to an embodiment of the present invention;

[0078] Fig.13 A schematic diagram of the structure of a map construction system in one embodiment;

[0079] Fig.14 FIG10 is a flowchart of a map construction method in an embodiment;

[0080] Fig.15 One of the structural block diagrams of a map building device in one embodiment;

[0081] Fig.16 This is a second structural block diagram of a map building device in an embodiment;

[0082] Fig.17 is a structural block diagram of a vehicle in one embodiment;

[0083] Fig.18 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] The map construction method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. It includes a vehicle 102, a cloud server 104 and a data storage system 106, and the vehicle 102 communicates with the cloud server 104 through the network. The vehicle 102 is equipped with vehicle-mounted equipment, vehicle-mounted sensors, wireless communication modules, positioning modules and processing modules, etc. The vehicle-mounted sensors are used to collect perceived information, that is, the surrounding environment image of the vehicle 102; the positioning module is used to locate the location of the vehicle, that is, the location information of the vehicle; the processing module can be connected to the vehicle-mounted sensor, the wireless communication module, and the positioning module respectively, and the processing module is used to obtain the surrounding environment image collected from the vehicle-mounted sensor, and the processing module is also used to obtain the location information of the vehicle from the positioning module. The processing module can also communicate with the cloud server 104 through the wireless communication module and obtain cloud map features, thereby combining the vehicle's location information, the surrounding environment image and the cloud map features to construct a target map, and display the target map in real time on the display screen of the vehicle-mounted equipment of the vehicle 102 for user viewing. The data storage system 106 can store data that the cloud server 104 needs to process, such as cloud map features, prior maps, etc. The data storage system can be integrated on the cloud server 104, or placed on the cloud or other network servers. The cloud server 104 can be implemented as an independent server or a server cluster composed of multiple servers.

[0086] At present, in the field of intelligent vehicle driving, vehicles can build environmental maps of the road sections where the vehicles are located online, that is, intelligent vehicles generate environmental maps of road sections in real time. Online real-time map construction ensures the freshness of map data, even in dynamic environments where road conditions change frequently. It enables vehicles to make wise decisions, bypass obstacles and avoid accidents. In related technologies, when intelligent vehicles are driving, the laser radar or environmental information sensor device on the intelligent vehicles will perceive the surrounding environmental information in real time, and build an environmental map based on this environmental information in real time. However, the above-mentioned method of real-time map construction has the following problems: the description of road topology is not accurate and detailed enough; there is a lack of description of the lane connection relationship and lane centerline in the intersection; the map construction distance is limited due to line of sight obstruction and weather.

[0087] In view of this, an embodiment of the present application provides a map construction method, which can realize real-time construction of a high-precision map by combining map features on a cloud server. The map can accurately describe the topological structure of the road and is not easily affected by the limited mapping distance caused by reasons such as line of sight obstruction and weather.

[0088] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0089] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] In one embodiment, Figure 2 As shown, a map construction method is provided, which is applied to Figure 1 The vehicle-mounted device on the vehicle in the example is used to illustrate, including the following steps:

[0091] S201, obtaining the surrounding environment image collected by the vehicle-mounted sensor and the vehicle's position information.

[0092] Among them, the vehicle-mounted sensor is used to collect road information of the vehicle's surrounding environment. The vehicle can be equipped with multiple vehicle-mounted sensors. Different vehicle-mounted sensors can collect environmental images from different perspectives, so all vehicle-mounted sensors on the vehicle can collect environmental images from all perspectives, such as 360° environmental images. The location information includes the current geographical location information of the vehicle (e.g., geographical name, geographical location description) or the current latitude and longitude information of the vehicle (e.g., latitude and longitude coordinates).

[0093] In the embodiment of the present application, the vehicle-mounted device is provided with a wireless communication module, a processing module and a positioning module, and the processing module is respectively connected to the vehicle-mounted sensor, the wireless communication module and the positioning module. When the vehicle is started, the vehicle-mounted device can immediately start the vehicle-mounted sensor on the vehicle to collect the perception information around the vehicle, obtain the surrounding environment image, and start the positioning module to locate the position of the vehicle, and obtain the vehicle's position information; then, the processing module can obtain the surrounding environment image and the vehicle's position information. One scenario is that the surrounding environment image and the vehicle's position information are reported to the cloud server for storage, so that the cloud server can build a map of the current road section where the vehicle is located based on the surrounding environment image and the vehicle's position information; in another scenario, when the processing module obtains the surrounding environment image and the vehicle's position information, it can immediately perform steps S202-S204 based on this information to generate and display the target map. It should be noted that different on-board sensors can collect environmental images from different perspectives, so multiple on-board sensors on the vehicle collect environmental images from their respective perspectives, so the method for obtaining the environmental map around the vehicle specifically includes: the processing module can use a corresponding image stitching algorithm or image processing algorithm to stitch the environmental images collected from their respective perspectives by all on-board sensors to generate a stitched 360° surrounding environment image. In addition, the positioning module can be a GPS positioning device or other types of positioning devices.

[0094] S202, sending a map acquisition request to a cloud server, and receiving cloud map features sent by the cloud server.

[0095] Among them, the map acquisition request carries the vehicle's location information and surrounding environment image; the cloud map feature is determined by the cloud server based on the location information and the surrounding environment image; the cloud map feature is associated with the environmental map feature of the surrounding environment image, or the cloud map feature is an enhanced feature of the map in the surrounding environment image, and the enhanced feature may include the features of lane lines, lane connecting lines in intersections, and lane center lines in the map. The cloud map feature is a map feature obtained by the cloud server after pre-extracting features from the prior environment image. The cloud map feature includes cloud depth features, cloud bird's-eye view features, and cloud content features, which is equivalent to the cloud server pre-extracting depth features, bird's-eye view features, and content features from the prior environment image to obtain the above cloud depth features, cloud bird's-eye view features, and cloud content features. The prior environment image can be the surrounding environment image reported by other vehicles when driving to the road section where the vehicle is currently located, or it can be a map image of the road section where the vehicle is currently located provided by the road traffic management platform.

[0096] In the embodiment of the present application, when the vehicle-mounted device locates the vehicle based on the above steps, obtains the vehicle's location information, and collects the surrounding environment image, it can immediately generate a map acquisition request based on the location information and the surrounding environment image, and send the map acquisition request to the cloud server. When the cloud server receives the map acquisition request, the first scenario is that the cloud server obtains the cloud map feature associated with the location information from the map feature library and returns the cloud map feature to the vehicle-mounted device; the second scenario is that the cloud server obtains the cloud map feature associated with the surrounding environment image from the map feature library and returns the cloud map feature to the vehicle-mounted device; the third scenario is that since the location information is not necessarily accurate, the cloud server can first identify the surrounding environment image, realize the combination of the surrounding environment image and the location information, obtain new location information, and then obtain the cloud map feature associated with the new location information from the map feature library. It should be noted that the map feature library pre-stores multiple map features, all of which are obtained by the cloud server extracting features from the map of each road section based on the corresponding feature extraction algorithm. The fourth scenario is that the cloud server can first obtain the prior map associated with the location information or the surrounding environment image from the map library, and use the feature extraction algorithm in real time to extract features from the prior map to obtain cloud map features, and return the cloud map features to the vehicle-mounted device. It should be noted that the map feature library pre-stores the cloud map features of each road section, and the stored cloud map features are updated in real time or regularly, so that the cloud server can return accurate cloud map features to the vehicle-mounted device. The cloud line in the map library stores multiple prior maps of each road section, and the stored prior maps are updated in real time or regularly, so that the cloud server can perform feature extraction based on accurate prior maps, obtain accurate cloud map features and return them to the vehicle-mounted device.

[0097] S203, constructing a target map based on the surrounding environment image and cloud map features.

[0098] In an embodiment of the present application, when the vehicle-mounted device receives the cloud map features sent by the cloud server, the processing module thereon can call the trained vehicle-side neural network model to extract features from the surrounding environment image, and obtain the environmental map features of the surrounding environment image; optionally, when the vehicle-mounted device receives the cloud map features sent by the cloud server, it can also receive the map processing neural network model associated with the cloud map features sent by the cloud server, and then use the neural network model as the vehicle-side neural network model to extract features from the surrounding environment image, and obtain the environmental map features. It should be noted that when the cloud server receives the map acquisition request sent by the vehicle-mounted device, when it obtains the cloud map features associated with the map acquisition request, it can also return the neural network model associated with the cloud map features to the vehicle-mounted terminal at the same time. Here, it is explained that the neural network model associated with the cloud map features can include a vehicle-side neural network model and a map decoder. Please refer to the following Figure 3 The method described in the embodiment; Optionally, the neural network model associated with the cloud map feature may also include a feature extraction network, a deep branch network, a feature conversion network, and a content semantic branch network, as shown below Figure 4 The method described in the embodiment; Optionally, the neural network model associated with the cloud map feature may also include a first feature fusion network, a second feature fusion network and a third feature fusion network, as described below Figure 5 The method described in the embodiment. When the vehicle-mounted device obtains the environmental map features of the surrounding environment image, the environmental map features and the cloud map features can be feature-fused, and a map is constructed based on the fused features to obtain a target map; optionally, the vehicle-mounted device converts the environmental map features and the cloud map features into feature vectors, and a map is constructed based on the feature vectors to obtain a target map.

[0099] In the embodiment of the present application, when the vehicle-mounted device receives the cloud map features sent by the cloud server, the processing module thereon can also call the trained image enhancement model, and use the image enhancement model to perform map feature enhancement processing on the surrounding environment image according to the cloud map features to obtain the enhanced surrounding environment image, and then the processing module calls the trained vehicle-side neural network model to extract features from the enhanced surrounding environment image to obtain the environmental map features, and finally the vehicle-mounted device can construct the target map based on the environmental map features. Optionally, when the vehicle-mounted device obtains the surrounding environment image in real time, it can send the surrounding environment image to the cloud server in real time so that the cloud server can store the surrounding environment image as a reference map image in the map library, or use the surrounding environment image to perform feature extraction to obtain the cloud map features of the road section where the surrounding environment image corresponds to the geographical location, and store them in the map feature library for use by the vehicle-mounted devices of other vehicles.

[0100] S204, displaying the target map on the display interface of the vehicle-mounted device.

[0101] In the embodiment of the present application, when the processing module on the vehicle-mounted device completes the construction of the target map, if the display screen on the vehicle-mounted device is not turned on or is in a screen-off state, the processing module can automatically turn on the display interface of the display screen or activate the display interface of the display screen, and then display the target map on the display interface (see Figure 2A Optionally, when the processing module completes the construction of the target map, it may also display the target map on the display interface in response to the user's triggering operation of the corresponding map display control or navigation control on the display interface (see Figure 2A The display interface shown, in which a map of the vehicle's current location is constructed online, wherein any M in the display interface can be a map display control or a navigation control).

[0102] In the above-mentioned map construction method, the vehicle-mounted device obtains the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, as well as the location information of the vehicle, sends a map acquisition request to the cloud server, and receives the cloud map features sent by the cloud server, constructs a target map according to the surrounding environment image and the cloud map features, and displays the target map on the display interface of the vehicle-mounted device, wherein the map acquisition request carries the location information and the surrounding environment image. When the vehicle-mounted device in the above-mentioned method constructs the map online, it can obtain the cloud map features from the cloud server, and construct the target map by combining the cloud map features and the surrounding environment image collected by the vehicle-mounted sensor, so that the quality of the target map is not affected by the environment in which the vehicle is located, such as weather or field of view obstruction, etc., thereby improving the accuracy of constructing the target map. In addition, the fusion of cloud map features and surrounding environment images can be realized to improve the refinement of map features, so that the constructed map can clearly reflect the road sidelines, road centerlines and other detailed information in the road, thereby improving the accuracy of map construction.

[0103] In one embodiment, Figure 3 As shown, a method for constructing a target map is provided, that is, the above S203 "constructing a target map according to the surrounding environment image and the cloud map features", including:

[0104] S301, input the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environmental map features.

[0105] Among them, the environmental map features include environmental depth features, environmental bird's-eye view features, and environmental content features. The vehicle-side neural network model is a neural network model that has been pre-trained based on images, and the vehicle-side neural network model is used to perform multi-dimensional feature extraction on the input image to obtain the image's features in depth dimension, bird's-eye view type dimension, and content dimension. Optionally, the vehicle-side neural network model can be a neural network model pre-stored on the processing module, or it can be a neural network model sent to the vehicle-mounted device by a cloud server.

[0106] In an embodiment of the present application, when the processing module on the vehicle-mounted device obtains the surrounding environment image, it can call a pre-trained vehicle-side neural network model, or obtain the vehicle-side neural network model from the cloud server, or receive the vehicle-side neural network model sent by the cloud server, and then input the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environmental map features including environmental depth features, environmental bird's-eye view features, and environmental content features.

[0107] S302: Fusing the environment map features and the cloud map features to obtain target fusion features.

[0108] In the embodiment of the present application, when the processing module on the vehicle-mounted device obtains the environmental map features based on the above steps, the processing module can use the corresponding image fusion algorithm to fuse the environmental map features and the cloud map features to obtain the target fusion features; optionally, the processing module can first perform deduplication processing, enhancement processing or complementary processing on the environmental map features and the cloud map features to obtain the processed map features, and finally use the corresponding image fusion algorithm to fuse the processed map features to obtain the target fusion features. Among them, on the one hand, since the environmental map features and the cloud map features are both features extracted from the relevant map of the surrounding environment of the current position of the vehicle, there may be the same features. Based on this, in order to improve the processing efficiency, the processing module can perform deduplication processing on the environmental map features and the cloud map features, which can improve the efficiency of the subsequent fusion based on the processed map features, that is, improve the efficiency of image processing; on the other hand, since the environmental map features may be inaccurate due to the limitations of the vehicle's environment (for example, rainy days or blocked vision), based on this, in order to improve the accuracy of map construction, the processing module can use the cloud map features to enhance the environmental map features. Processing can improve the accuracy of vehicle map construction in harsh environments in the later stage; on the other hand, since both the environmental map features and the cloud map features are features extracted from the relevant maps of the surrounding environment of the vehicle's current location, and the map areas involved in the relevant maps may not be exactly the same, based on this, in order to improve the accuracy of map construction or refine the detailed information in the map, such as road boundary lines, road lines, road center lines, etc., and also because the images collected by the on-board sensors may be limited by the environment in which the vehicle is located, some necessary features or detailed features in the surrounding environment map are missing, such as the lack of road line features due to rainy days, and the lack of intersection road features due to obstruction of vision. In this scenario, the processing module can first determine the complementary features in the cloud map features and the environmental map features, and then use the complementary features and environmental map features for supplementary processing, which can improve the accuracy of vehicle map construction in harsh environments in the later stage, or highlight the detailed information of the constructed map, so that the target map can display more information.

[0109] Optionally, the environmental map features include environmental depth features, environmental bird's-eye view features, and environmental content features; the cloud map features include cloud depth features, cloud bird's-eye view features, and cloud content features; based on this, the vehicle-mounted device can first fuse the same type of features once, and then perform a second fusion based on the fused features of each type to finally obtain the target fusion features. For example, the vehicle-mounted device can fuse the environmental depth features with the cloud depth features to obtain fused depth features; fuse the environmental bird's-eye view features with the cloud bird's-eye view features to obtain fused bird's-eye view features; fuse the environmental content features with the cloud content features to obtain fused content features; finally, the target fusion features including the fused depth features, the fused bird's-eye view features, and the fused content features can be directly obtained, or the fused depth features, the fused bird's-eye view features, and the fused content features can be fused to obtain the target fusion features. Optionally, the on-board device may first fuse the environmental depth features, environmental bird's-eye view features, and environmental content features to obtain environmental fusion features; and fuse the cloud depth features, cloud bird's-eye view features, and cloud content features to obtain cloud fusion features; finally, the on-board device may directly obtain target fusion features including environmental fusion features and cloud fusion features; or, the on-board device may fuse the environmental fusion features and cloud fusion features to obtain target fusion features.

[0110] S303, inputting the target fusion features into a map decoder to construct a target map.

[0111] The map decoder is used to output vectorized map elements, and then construct a map based on the vectorized map elements. The map decoder is a pre-trained image decoder, which can be trained together with the image encoder. The map decoder can be pre-stored in the vehicle-mounted device or sent to the vehicle-mounted device by the cloud.

[0112] In the embodiment of the present application, when the vehicle-mounted device obtains the target fusion feature based on the above steps, the target fusion feature can be directly input into the pre-trained map decoder for map construction to obtain the target map. Based on the above description of S303, the target fusion feature can include fused depth features, fused bird's-eye view features, and fused content features. If based on this, the vehicle-mounted device can input the fused depth features, fused bird's-eye view features, and fused content features into the map decoder to construct the target map; optionally, the target fusion feature can include environmental fusion features and cloud fusion features. If based on this, the vehicle-mounted device can input the environmental fusion features and cloud fusion features into the map decoder to construct the target map.

[0113] The method described in the embodiment of the present application obtains multi-dimensional image features through multi-dimensional feature extraction, and constructs a target map based on the fused multi-dimensional features, which can improve the accuracy of map construction and enable detailed information to be fully displayed in the target map.

[0114] In one embodiment, Figure 4 As shown, a vehicle-side neural network model is provided, which includes a feature extraction network 401, a deep branch network 402, a feature conversion network 403, and a content semantic branch network 404; the output of the feature extraction network 401 is respectively connected to the input of the deep branch network 402, the feature conversion network 403, and the content semantic branch network 404; based on this, a method for obtaining environmental map features is provided, that is, the above-mentioned S301 "inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environmental map features", such as Figure 5 As shown, including:

[0115] S501, input the surrounding environment image into the feature extraction network to extract global features, and obtain the global features of the surrounding environment image.

[0116] The feature extraction network is used for global feature extraction, and specifically can be a 2D feature extraction network backbone, and the feature extraction network is a pre-trained neural network.

[0117] In an embodiment of the present application, when the vehicle-mounted device inputs the surrounding environment image into the vehicle-side neural network model for feature extraction, the surrounding environment image can be first input into the feature extraction network for global feature extraction to obtain the global features of the surrounding environment map.

[0118] S502, inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, and obtaining the environmental depth features of the surrounding environment image.

[0119] Among them, the deep branch network is used for deep feature extraction and is a pre-trained neural network.

[0120] In an embodiment of the present application, when the vehicle-mounted device obtains the global features based on the aforementioned steps, the global features and the surrounding environment image can be input into the deep branch network for deep feature extraction to obtain the environmental depth features of the surrounding environment image.

[0121] S503, inputting the global features and the surrounding environment image into a feature conversion network for feature conversion to obtain the environment bird's-eye view features of the surrounding environment image.

[0122] The feature conversion network is used for feature conversion, specifically for converting features into bird's eye view features, and is a pre-trained neural network. For example, the feature conversion network may be a 2D to BEV network, which can convert 2D features into bird's eye view features (Bird's eye view, BEV).

[0123] In an embodiment of the present application, when the vehicle-mounted device obtains the global features based on the aforementioned steps, the global features and the surrounding environment image can be input into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image.

[0124] S504, inputting the global features and the surrounding environment image into the content semantic branch network to extract semantic features, and obtaining the environmental content features of the surrounding environment image.

[0125] Among them, the content semantic branch network is used for semantic feature extraction and is a pre-trained neural network.

[0126] In an embodiment of the present application, when the vehicle-mounted device obtains global features based on the aforementioned steps, the global features and the surrounding environment image can be input into a deep branch network for semantic feature extraction to obtain environmental content features of the surrounding environment image.

[0127] The method described in the above embodiment obtains multi-dimensional image features through multi-dimensional feature extraction such as global, depth, image type (bird's eye view), semantics, etc., and constructs a target map based on the fused multi-dimensional features, which can improve the accuracy of map construction and enable the target map to fully display detailed information.

[0128] In one embodiment, the target fusion feature may include fusion depth feature, fusion bird's-eye view feature and fusion content feature, and the cloud map feature may include cloud depth feature, cloud bird's-eye view feature and cloud content feature. Based on this, Figure 3 The network architecture corresponding to the method described in the embodiment (composed of the vehicle-side neural network model 601 and the map decoder 602) is as follows: Figure 6 As shown, it also includes a first feature fusion network 603, a second feature fusion network 604 and a third feature fusion network 605, correspondingly, Figure 3 The method S302 described in the embodiment "fusing the environment map features and the cloud map features to obtain the target fusion features", such as Figure 7 As shown, including:

[0129] S701, input the environmental depth features and the cloud depth features into the first feature fusion network for feature fusion to obtain fused depth features.

[0130] Among them, the first feature fusion network is used for deep feature fusion and is a pre-trained neural network.

[0131] In an embodiment of the present application, when the vehicle-mounted device obtains the environmental depth features and the cloud depth features based on the aforementioned steps, the environmental depth features and the cloud depth features can be input into the first feature fusion network for feature fusion to obtain fused depth features.

[0132] S702: Input the environment bird's-eye view features and the cloud bird's-eye view features into a second feature fusion network for feature fusion to obtain fused bird's-eye view features.

[0133] Among them, the second feature fusion network is used for bird's-eye view feature fusion, which is a pre-trained neural network.

[0134] In an embodiment of the present application, when the vehicle-mounted device obtains the environmental bird's-eye view features and the cloud bird's-eye view features based on the above steps, the environmental bird's-eye view features and the cloud bird's-eye view features can be input into the second feature fusion network for feature fusion to obtain fused bird's-eye view features.

[0135] S703: Input the environment content features and the cloud content features into a third feature fusion network for feature fusion to obtain fused content features.

[0136] Among them, the third feature fusion network is used for content feature fusion and is a pre-trained neural network.

[0137] In an embodiment of the present application, when the vehicle-mounted device obtains environmental content features and cloud content features based on the aforementioned steps, the environmental content features and cloud content features can be input into a third feature fusion network for feature fusion to obtain fused content features.

[0138] The method described in the above embodiment can improve the accuracy of feature fusion by performing targeted feature fusion through different types of fusion networks, thereby improving the accuracy of map construction based on the fusion features in the later stage.

[0139] In one embodiment, the vehicle-mounted device can also trigger the display of the intersection map. Based on this, Figure 2 The method described in the embodiment, such as Figure 8 As shown, it also includes:

[0140] S801, sending the navigation path of the vehicle to the cloud server.

[0141] The navigation path may refer to a path between a destination address of the vehicle and a starting address of the vehicle, and may be determined by a user triggering a navigation control on a display screen of the vehicle-mounted device or inputting navigation information.

[0142] In the embodiment of the present application, the user can perform navigation operations on the vehicle-mounted device, specifically, the corresponding navigation control can be triggered on the display screen of the vehicle-mounted device or navigation information can be input. For example, the user can input the destination address on the display screen, and the vehicle-mounted device can locate the starting address of the vehicle by itself, or the user inputs the starting address of the vehicle on the display screen, and the vehicle-mounted device can obtain the path between the destination address of the vehicle and the starting address of the vehicle. When the vehicle-mounted device obtains the navigation path, the navigation path can be sent to the cloud server so that the cloud server can generate a corresponding navigation map according to the navigation path and return the navigation map to the vehicle-mounted device, or so that the cloud server can generate a corresponding intersection map feature according to the navigation path and return the intersection map feature to the vehicle-mounted device.

[0143] S802, receiving intersection map features returned by the cloud server based on the navigation path and location information, and constructing an intersection map according to the intersection map features.

[0144] Among them, the intersection map is the intersection map of the next intersection that the vehicle is expected to travel; the intersection map feature is the map feature of the next intersection that the vehicle is expected to travel. The navigation path includes the location information and other relevant description information of multiple intersections where the vehicle travels, that is, it includes multiple intersections between the destination address and the start address of the travel. The intersection map feature is the map feature obtained after the cloud server extracts the features of the prior intersection image in advance. The intersection map feature includes intersection depth features, intersection bird's-eye view features, and cloud intersection content features, which is equivalent to the cloud server pre-extracting the depth feature extraction, bird's-eye view feature extraction, and content feature extraction of the prior intersection image, respectively, to obtain the intersection depth feature, intersection bird's-eye view feature, and intersection content feature. The prior intersection image can be an intersection image reported by other vehicles when driving to the intersection, or it can be a map image of the intersection provided by the road traffic management platform.

[0145] In the embodiment of the present application, when the cloud server obtains the location information and navigation path of the vehicle based on the above steps, the next intersection that the vehicle is expected to travel can be determined based on this information, including the location information and related description information of the next intersection, wherein the related description information is the geographical location description or geographical location name of the intersection, and the location information can be the latitude and longitude coordinates of the next intersection. Then, the cloud server can obtain the intersection map feature associated with the next intersection from the map feature library based on the location information and related description information of the next intersection, and return the intersection map feature to the vehicle-mounted device; it should be noted that the map feature library pre-stores multiple map features, all of which are obtained by the cloud server extracting features from the map of each intersection based on the corresponding feature extraction algorithm. It should be noted that the cloud server can first obtain the prior intersection map associated with the location information of the next intersection from the map library, and use the feature extraction algorithm in real time to extract features from the prior intersection map to obtain the intersection map feature, and return the intersection map feature to the vehicle-mounted device. When the vehicle-mounted device obtains the intersection map features returned by the cloud server, the intersection map can be constructed based on the intersection map features and combined with the surrounding environment image.

[0146] S803, displaying the intersection map on the display interface.

[0147] In the embodiment of the present application, when the processing module on the vehicle-mounted device completes the construction of the intersection map, if the display screen on the vehicle-mounted device displays the target map at this time, the processing module can automatically display the intersection map on the target map, for example, displaying the intersection map in a small window so that the user can view the situation of the next intersection that the vehicle is expected to travel through the intersection map, so as to predict the situation of the next intersection in advance, especially the intersection that is more likely to be congested, and can be explored in advance, for example, see Fig. 8A The interface diagram shown is a diagram in which the intersection map is displayed in the form of a small window on the map display interface of the vehicle-mounted device, and the window is provided with a close control and a shrink and hide display control; it can also be displayed simultaneously with the target map based on other methods. Optionally, when the processing module on the vehicle-mounted device completes the construction of the intersection map, if the target map is displayed on the display screen of the vehicle-mounted device at this time, the current display interface can also switch to the intersection map display, so that the user can fully and clearly view the situation of the next intersection on the display screen. Optionally, if the target map is not displayed on the display screen of the vehicle-mounted device at this time, the processing module can automatically trigger the display screen to display the intersection map, so that the user can preview the situation of the next intersection on the display screen.

[0148] According to the method described in the above embodiment, when the on-board equipment constructs a map of the vehicle's surrounding environment online, it can also construct an intersection map of the next intersection where the vehicle is expected to travel, so that the user can predict the situation at the next intersection. The on-board equipment constructs the intersection map based on the intersection map features sent by the cloud server. The quality of the intersection map is not affected by the environment in which the vehicle is located, such as weather or field of view obstruction, thereby improving the accuracy of constructing the intersection map and thus improving the safety of vehicle driving.

[0149] In one embodiment, a method for triggering target map display is provided, such as Fig. 9 As shown, the method includes:

[0150] S901, when the vehicle is started, determining whether the vehicle-mounted equipment meets the preset map building conditions.

[0151] Among them, the preset map construction conditions include any of the following: the vehicle-mounted device responds to the user's triggering operation of the map generation control or the navigation control on the display interface of the vehicle-mounted device; the vehicle-mounted device is powered on and started; the vehicle-mounted device detects that the vehicle is driving to the target road; the target road includes any of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0152] S902, when the vehicle-mounted device meets the preset map building conditions, execute the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor on the vehicle and the vehicle's position information.

[0153] The embodiment of the present application relates to a method for triggering the display of a target map on a display screen of a vehicle-mounted device. Specifically, when the vehicle is started, the first scenario is to determine whether the vehicle-mounted device is powered on. If the vehicle-mounted device is powered on, the vehicle-mounted device can execute the aforementioned Figure 2-Figure 8 The method described in the embodiment constructs a target map and / or intersection map, and displays the target map and / or intersection map on the display screen of the vehicle-mounted device. The second scenario is that the vehicle-mounted device has been started, that is, the display screen on the vehicle-mounted device has been turned on, and the user can trigger the map generation control or navigation control on the display interface of the display screen. At this time, the processing module can determine whether the user has performed a triggering operation on the map generation control or navigation control on the display interface of the vehicle-mounted device, that is, whether it responds to the user's triggering operation on the map generation control or navigation control on the display interface of the vehicle-mounted device. If the user triggers the map generation control or navigation control, the vehicle-mounted device can immediately execute the aforementioned Figure 2-Figure 8The method described in the embodiment constructs a target map and / or a road intersection map, and displays the target map and / or the road intersection map on the display screen of the vehicle-mounted device. The third scenario is that the corresponding sensor or detection device on the vehicle can detect the road state of the vehicle and send the road state of the vehicle to the processing module of the vehicle-mounted device. When the vehicle-mounted device detects that the vehicle has traveled to the target road according to the road state of the vehicle, the vehicle-mounted device can immediately execute the above-mentioned Figure 2-Figure 8 The method described in the embodiment constructs a target map and / or an intersection map.

[0154] Above Figure 2-Figure 9 The method described in the embodiment is a map construction method on the vehicle-mounted device side. Figure 10-Figure 14 The method described in the embodiment is a map construction method on the cloud server side.

[0155] In one embodiment, Fig.10 As shown, a map construction method is provided, which is applied to Figure 1 Taking the cloud server on the vehicle in FIG. 1 as an example, the following steps are included:

[0156] S1001, receiving a map acquisition request sent by an on-board device of a vehicle.

[0157] The map acquisition request carries the vehicle's location information and surrounding environment images collected by the vehicle's onboard sensors.

[0158] The method described in the embodiment of the present application is basically consistent with the step described in the aforementioned S202. Please refer to the aforementioned description for details, which will not be repeated here.

[0159] S1002, determining a map location based on the location information and the surrounding environment image, and extracting a cloud map feature corresponding to the map location from a map feature library.

[0160] In the embodiment of the present application, when the cloud server receives the map acquisition request, since the vehicle's location information may not be accurate, the cloud server may first identify the surrounding environment image, combine the surrounding environment image and location information at the same time, obtain the map location, and then obtain the cloud map features associated with the map location from the map feature library. Optionally, when the cloud server obtains the map location by combining the surrounding environment image and location information, it may obtain the prior map associated with the map location from the map library, perform feature extraction on the prior map based on the corresponding feature extraction algorithm online, obtain the cloud map features, and return the cloud map features to the vehicle-mounted device. It should be noted that the map feature library pre-stores multiple map features, all of which are obtained by the cloud server extracting features from the map of each road section based on the corresponding feature extraction algorithm. The map feature library can be stored on the cloud server or on other servers.

[0161] S1003, sending the cloud map features to the vehicle-mounted device.

[0162] Among them, the cloud map features are used to instruct the vehicle-mounted device to construct a target map based on the cloud map features and the surrounding environment image, and display the target map on the display interface of the vehicle-mounted device.

[0163] The method described in the embodiment of the present application is basically consistent with the steps S1002-S1003 above. Please refer to the above description for details, which will not be repeated here. It should be noted that when the cloud server sends the cloud map feature to the vehicle-mounted device, or after sending it to the vehicle-mounted device, it can also obtain the neural network model associated with the cloud map feature. Here, the neural network model associated with the cloud map feature is the map construction model required for the vehicle-mounted device to build a map, which may include the vehicle-side neural network model and the map decoder. Please refer to the above Figure 3 The method described in the embodiment; Optionally, the neural network model associated with the cloud map feature may also include a feature extraction network, a deep branch network, a feature conversion network, and a content semantic branch network, as described above. Figure 4 The method described in the embodiment; Optionally, the neural network model associated with the cloud map feature may also include a first feature fusion network, a second feature fusion network and a third feature fusion network, see the aforementioned Figure 5 The method described in the embodiment.

[0164] In the above map construction method, the cloud server receives a map acquisition request sent by the vehicle-mounted device on the vehicle, determines the map location according to the location information and the surrounding environment image, extracts the cloud map feature corresponding to the map location from the map feature library, and sends the cloud map feature to the vehicle-mounted device; the map acquisition request carries the vehicle's location information and the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, and the cloud map feature is used to instruct the vehicle-mounted device to construct a target map according to the cloud map feature and the surrounding environment image, and display the target map on the display interface of the vehicle-mounted device. When the vehicle-mounted device in the above method constructs the map online, the cloud server can provide the vehicle-mounted device with the cloud map feature, so that the cloud server can construct the target map by combining the cloud map feature and the surrounding environment image collected by the vehicle-mounted sensor, and because the cloud map feature is extracted from the map feature library, the map features included in the map feature library are all extracted based on a map with good quality, so the extracted cloud map feature is very accurate, so the quality of the target map constructed by the vehicle-mounted device based on the surrounding environment image and the cloud map feature is not affected by the environment where the vehicle is located, such as weather or field of view obstruction, thereby improving the accuracy of constructing the target map. In addition, the fusion of cloud map features and surrounding environment images can be realized to improve the refinement of map features, so that the constructed map can clearly reflect detailed information such as road edges and road center lines in the road, thereby improving the accuracy of map construction.

[0165] In one embodiment, Fig.10 The method described in the embodiment, such as Fig.11 As shown, it also includes:

[0166] S1101, receiving a navigation path sent by an in-vehicle device.

[0167] S1102, determining the next intersection location of the vehicle based on the navigation path and location information.

[0168] S1103, extracting the intersection map feature corresponding to the next intersection position from the map feature library.

[0169] S1104, sending the intersection map features to the vehicle-mounted device.

[0170] Among them, the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on the display interface of the vehicle-mounted device. The intersection map is the intersection map of the next intersection that the vehicle is expected to travel; the intersection map feature is the map feature of the next intersection that the vehicle is expected to travel. The navigation path includes the location information and other related description information of multiple intersections where the vehicle is traveling, that is, including multiple intersections between the destination address and the start address of the travel. The intersection map feature is a map feature obtained after the cloud server pre-extracts the feature of the prior intersection image. The intersection map feature includes intersection depth feature, intersection bird's-eye view feature, and cloud intersection content feature, which is equivalent to the cloud server pre-extracting the depth feature, bird's-eye view feature, and content feature of the prior intersection image to obtain the intersection depth feature, intersection bird's-eye view feature, and intersection content feature. The prior intersection image can be an intersection image reported by other vehicles when driving to the intersection, or it can be a map image of the intersection provided by the road traffic management platform.

[0171] In the embodiment of the present application, when the cloud server obtains the vehicle's location information and navigation path based on the above steps, the next intersection location that the vehicle is expected to travel can be determined based on this information. Then, the cloud server can obtain the intersection map feature associated with the next intersection location from the map feature library based on the next intersection location, and return the intersection map feature to the vehicle-mounted device; it should be noted that the map feature library pre-stores multiple map features, all of which are obtained by the cloud server performing feature extraction on the map of each intersection based on the corresponding feature extraction algorithm. It should be noted that the cloud server can first obtain the prior intersection map associated with the next intersection location from the map library, and use the feature extraction algorithm in real time to extract the feature of the prior intersection map to obtain the intersection map feature, and return the intersection map feature to the vehicle-mounted device. When the vehicle-mounted device obtains the intersection map feature returned by the cloud server, it can construct an intersection map based on the intersection map feature and in combination with the surrounding environment image.

[0172] In one embodiment, Fig.10 The method described in the embodiment, such as Fig.12 As shown, it also includes:

[0173] S1201, receiving other environment images and location information sent by other vehicles within a preset time period.

[0174] The other environment images are environment images collected by vehicle-mounted sensors on other vehicles. The preset time period can be determined in advance according to actual needs. For example, the preset time period can be different time periods such as 30 days, 15 days, or 10 days.

[0175] In an embodiment of the present application, the cloud server can collect other environmental images and location information of each vehicle sent by each vehicle at preset time periods, and associate the other environmental images and location information with each other, and also record the association relationship between other environmental images and location information.

[0176] S1202: Input other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features.

[0177] In the embodiment of the present application, when the cloud server obtains other environmental images of each vehicle, it can input the other environmental images into the pre-trained vehicle-side neural network model for feature encoding, obtain other cloud map features corresponding to the other environmental images, and store them in the map feature library. The other cloud map features may include other cloud depth features, other cloud bird's-eye view features, and other cloud content features.

[0178] S1203, associating other cloud map features with the map location corresponding to the location information, storing the other cloud map features in a map feature library, and recording the association relationship between the other cloud map features and the map location in the map feature library.

[0179] In an embodiment of the present application, when the cloud server obtains other cloud map features and stores them in a map feature library, the cloud server can convert the location information into a map location, and associate the other cloud map features with the map location corresponding to the location information, and then record the association between the other cloud map features and the map location in the map feature library, so that when a vehicle needs to call the map feature library, the required cloud map features can be obtained according to the association.

[0180] According to the method described in the above embodiment, the cloud server can pre-build a map feature library, and when the vehicle needs to build a target map, the map feature library can be used to provide cloud map features for the vehicle's on-board equipment, so that the on-board equipment can combine the cloud map features and the surrounding environment images collected by itself to build a high-precision target map.

[0181] Based on the methods described in all the above embodiments, the present application also provides a map construction system, such as Fig.13As shown, the system includes: an on-board sensor 1301, an on-board device 1302, and a cloud server 1303; the on-board sensor 1301 and the on-board device 1302 are arranged on the vehicle; the on-board device 1302 is also provided with a wireless communication module, a processing module and a positioning module, the processing module is respectively connected to the on-board sensor 1301, the processing module is also connected to the positioning module, and the processing module is also connected to the cloud server 1303 through the wireless communication module. The functions of the on-board sensor 1301, the on-board device 1302, and the cloud server 1303 are all explained in the methods described in the aforementioned embodiments. Please refer to the aforementioned description for details and will not be repeated here.

[0182] Based on the methods described in all the above embodiments, the present application embodiment also provides a map construction method, such as Fig.14 As shown, the method includes:

[0183] S1401, the vehicle-mounted device obtains the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, and the vehicle's location information.

[0184] S1402, the vehicle-mounted device sends a map acquisition request to the cloud server; the map acquisition request carries the vehicle's location information and surrounding environment images.

[0185] S1403, the cloud server receives the map acquisition request, determines the map location according to the location information and the surrounding environment image, and extracts the cloud map features corresponding to the map location from the map feature library.

[0186] S1404, the cloud server sends the cloud map features to the vehicle-mounted device.

[0187] S1405, the vehicle-mounted device inputs the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environmental map features.

[0188] Among them, the environmental map features include environmental depth features, environmental bird's-eye view features, and environmental content features.

[0189] S1406, the vehicle-mounted device fuses the environmental map features with the cloud map features to obtain target fusion features.

[0190] S1407, the vehicle-mounted device inputs the target fusion feature into the map decoder to construct a target map. S1408, the vehicle-mounted device displays the target map on a display interface.

[0191] S1409, the vehicle-mounted device sends the vehicle's navigation path to the cloud server.

[0192] S1410, the cloud server receives the navigation path sent by the vehicle-mounted device, and determines the next intersection position of the vehicle based on the navigation path and location information.

[0193] S1411, the cloud server extracts the intersection map features corresponding to the next intersection location from the map feature library, and sends the intersection map features to the vehicle-mounted device.

[0194] The intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

[0195] S1412, the vehicle-mounted device displays the intersection map on the display interface.

[0196] Each step in the above embodiment has been described above. Please refer to the above description for details and will not be repeated here.

[0197] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0198] Based on the same inventive concept, the embodiment of the present application also provides a map construction device for implementing the map construction method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more map construction device embodiments provided below can refer to the limitations on the map construction method above, and will not be repeated here.

[0199] In one embodiment, Fig.15 As shown, a map construction device is provided, comprising:

[0200] The acquisition module 1501 is used to acquire the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, and the position information of the vehicle.

[0201] The sending and receiving module 1502 is used to send a map acquisition request to the cloud server and receive the cloud map features sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image.

[0202] The construction module 1503 is used to construct a target map according to the surrounding environment image and the cloud map features.

[0203] The display module 1504 is used to display the target map on the display interface of the vehicle-mounted device.

[0204] In one embodiment, the above-mentioned building blocks include:

[0205] An encoding unit, used for inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features;

[0206] A fusion unit, used for fusing the environment map feature with the cloud map feature to obtain a target fusion feature;

[0207] A construction unit is used to input the target fusion feature into a map decoder to construct the target map.

[0208] In one embodiment, the above-mentioned encoding unit is specifically used to input the surrounding environment image into the feature extraction network for global feature extraction to obtain global features of the surrounding environment image; input the global features and the surrounding environment image into the depth branch network for depth feature extraction to obtain environmental depth features of the surrounding environment image; input the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain environmental bird's-eye view features of the surrounding environment image; input the global features and the surrounding environment image into the content semantic branch network for semantic feature extraction to obtain environmental content features of the surrounding environment image.

[0209] In one embodiment, the above-mentioned fusion unit is specifically used to input the environmental depth features and the cloud depth features into a first feature fusion network for feature fusion to obtain the fused depth features; input the environmental bird's-eye view features and the cloud bird's-eye view features into a second feature fusion network for feature fusion to obtain the fused bird's-eye view features; input the environmental content features and the cloud content features into a third feature fusion network for feature fusion to obtain the fused content features.

[0210] In one embodiment, the map construction device further includes:

[0211] A sending module, used for sending the navigation path of the vehicle to the cloud server;

[0212] A junction map construction module is used to receive the junction map features returned by the cloud server based on the navigation path and the location information, and construct an intersection map according to the junction map features;

[0213] The intersection map display module is used to display the intersection map on the display interface.

[0214] In one embodiment, the map construction device further includes:

[0215] A determination module is used to determine whether the vehicle-mounted device meets a preset map building condition when the vehicle is started; wherein the preset map building condition includes any one of the following: the vehicle-mounted device responds to a user triggering operation of a map generation control or a navigation control on a display interface of the vehicle-mounted device; the vehicle-mounted device is powered on and started; the vehicle-mounted device detects that the vehicle has traveled to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0216] An execution module is used to execute the step of acquiring the surrounding environment image collected by the on-board sensor of the vehicle and the position information of the vehicle when the on-board device meets the preset map building condition.

[0217] In one embodiment, Fig.16 As shown, a map construction device is provided, comprising:

[0218] The receiving module 1601 is used to receive a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and the surrounding environment image collected by the on-board sensor of the vehicle;

[0219] A determination module 1602, configured to determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0220] The sending module 1603 is used to send the cloud map features to the vehicle-mounted device; the cloud map features are used to instruct the vehicle-mounted device to construct a target map based on the cloud map features and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0221] In one embodiment, the map construction device further includes:

[0222] A navigation path receiving module, used for receiving the navigation path sent by the vehicle-mounted device;

[0223] An intersection determination module, used to determine the next intersection position of the vehicle according to the navigation path and the position information;

[0224] A feature extraction module, used for extracting the intersection map feature corresponding to the next intersection position from the map feature library;

[0225] The intersection feature sending module is used to send the intersection map feature to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature and display the intersection map on the display interface of the vehicle-mounted device.

[0226] In one embodiment, the map construction device further includes:

[0227] The module for receiving other information is used to receive other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by the onboard sensors of the other vehicles;

[0228] A feature encoding module, used for inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features;

[0229] The association module is used to associate the other cloud map features with the map location corresponding to the location information, store the other cloud map features in the map feature library, and record the association relationship between the other cloud map features and the map location in the map feature library.

[0230] Each module in the above-mentioned map construction device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0231] In one embodiment, a vehicle is provided. Fig.17 As shown, the vehicle includes an on-board device, and the on-board device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the on-board device is used to provide computing and control capabilities. The memory of the on-board device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the on-board device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a map construction method is implemented. The display screen of the on-board device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the on-board device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the on-board device, or an external keyboard, touch pad or mouse, etc.

[0232] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.18 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a map construction method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0233] Those skilled in the art will understand that Fig.18 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0234] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0235] Obtaining surrounding environment images collected by on-board sensors on the vehicle and location information of the vehicle;

[0236] Sending a map acquisition request to a cloud server, and receiving a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image;

[0237] Constructing a target map according to the surrounding environment image and the cloud map features;

[0238] The target map is displayed on a display interface on the vehicle-mounted device.

[0239] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0240] Inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features;

[0241] Fusing the environment map features with the cloud map features to obtain target fusion features;

[0242] The target fusion features are input into a map decoder to construct the target map.

[0243] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0244] Inputting the surrounding environment image into the feature extraction network to extract global features, so as to obtain global features of the surrounding environment image;

[0245] Inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, so as to obtain the environmental deep features of the surrounding environment image;

[0246] Inputting the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image;

[0247] The global features and the surrounding environment image are input into the content semantic branch network to extract semantic features, so as to obtain the environmental content features of the surrounding environment image.

[0248] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0249] Inputting the environment depth feature and the cloud depth feature into a first feature fusion network for feature fusion to obtain the fused depth feature;

[0250] Inputting the environment bird's-eye view feature and the cloud bird's-eye view feature into a second feature fusion network for feature fusion to obtain the fused bird's-eye view feature;

[0251] The environmental content features and the cloud content features are input into a third feature fusion network for feature fusion to obtain the fused content features.

[0252] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0253] Sending the navigation path of the vehicle to the cloud server;

[0254] Receiving intersection map features returned by the cloud server based on the navigation path and the location information, and constructing an intersection map according to the intersection map features;

[0255] The intersection map is displayed on the display interface.

[0256] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0257] When the vehicle is started, determining whether the vehicle-mounted device meets a preset map building condition;

[0258] When the vehicle-mounted device meets the preset map building condition, the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor and the location information of the vehicle is performed;

[0259] The preset map construction condition includes any of the following:

[0260] The in-vehicle device responds to a triggering operation of a map generation control or a navigation control on a display interface of the in-vehicle device by a user;

[0261] The vehicle-mounted device is powered on and started;

[0262] The vehicle-mounted device detects that the vehicle travels to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0263] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0264] Receiving a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle;

[0265] Determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0266] The cloud map feature is sent to the vehicle-mounted device; the cloud map feature is used to instruct the vehicle-mounted device to construct a target map based on the cloud map feature and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0267] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0268] Receiving the navigation path sent by the vehicle-mounted device;

[0269] Determining the next intersection position of the vehicle according to the navigation path and the position information;

[0270] Extracting an intersection map feature corresponding to the next intersection position from the map feature library;

[0271] The intersection map feature is sent to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

[0272] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0273] Receiving other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by on-board sensors on the other vehicles;

[0274] Inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features;

[0275] The other cloud map features are associated with the map location corresponding to the location information, and the other cloud map features are stored in the map feature library, and the association relationship between the other cloud map features and the map location is recorded in the map feature library.

[0276] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0277] Obtaining surrounding environment images collected by on-board sensors on the vehicle and location information of the vehicle;

[0278] Sending a map acquisition request to a cloud server, and receiving a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image;

[0279] Constructing a target map according to the surrounding environment image and the cloud map features;

[0280] The target map is displayed on a display interface on the vehicle-mounted device.

[0281] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0282] Inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features;

[0283] Fusing the environment map features with the cloud map features to obtain target fusion features;

[0284] The target fusion features are input into a map decoder to construct the target map.

[0285] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0286] Inputting the surrounding environment image into the feature extraction network to extract global features, so as to obtain global features of the surrounding environment image;

[0287] Inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, so as to obtain the environmental deep features of the surrounding environment image;

[0288] Inputting the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image;

[0289] The global features and the surrounding environment image are input into the content semantic branch network to extract semantic features, so as to obtain the environmental content features of the surrounding environment image.

[0290] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0291] Inputting the environment depth feature and the cloud depth feature into a first feature fusion network for feature fusion to obtain the fused depth feature;

[0292] Inputting the environment bird's-eye view feature and the cloud bird's-eye view feature into a second feature fusion network for feature fusion to obtain the fused bird's-eye view feature;

[0293] The environmental content features and the cloud content features are input into a third feature fusion network for feature fusion to obtain the fused content features.

[0294] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0295] Sending the navigation path of the vehicle to the cloud server;

[0296] Receiving intersection map features returned by the cloud server based on the navigation path and the location information, and constructing an intersection map according to the intersection map features;

[0297] The intersection map is displayed on the display interface.

[0298] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0299] When the vehicle is started, determining whether the vehicle-mounted device meets a preset map building condition;

[0300] When the vehicle-mounted device meets the preset map building condition, the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor and the location information of the vehicle is performed;

[0301] The preset map construction condition includes any of the following:

[0302] The in-vehicle device responds to a triggering operation of a map generation control or a navigation control on a display interface of the in-vehicle device by a user;

[0303] The vehicle-mounted device is powered on and started;

[0304] The vehicle-mounted device detects that the vehicle travels to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0305] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0306] Receiving a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle;

[0307] Determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0308] The cloud map feature is sent to the vehicle-mounted device; the cloud map feature is used to instruct the vehicle-mounted device to construct a target map based on the cloud map feature and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0309] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0310] Receiving the navigation path sent by the vehicle-mounted device;

[0311] Determining the next intersection position of the vehicle according to the navigation path and the position information;

[0312] Extracting an intersection map feature corresponding to the next intersection position from the map feature library;

[0313] The intersection map feature is sent to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

[0314] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0315] Receiving other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by on-board sensors on the other vehicles;

[0316] Inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features;

[0317] The other cloud map features are associated with the map location corresponding to the location information, and the other cloud map features are stored in the map feature library, and the association relationship between the other cloud map features and the map location is recorded in the map feature library.

[0318] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0319] Obtaining surrounding environment images collected by on-board sensors on the vehicle and location information of the vehicle;

[0320] Sending a map acquisition request to a cloud server, and receiving a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image;

[0321] Constructing a target map according to the surrounding environment image and the cloud map features;

[0322] The target map is displayed on a display interface on the vehicle-mounted device.

[0323] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0324] Inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features;

[0325] Fusing the environment map features with the cloud map features to obtain target fusion features;

[0326] The target fusion features are input into a map decoder to construct the target map.

[0327] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0328] Inputting the surrounding environment image into the feature extraction network to extract global features, so as to obtain global features of the surrounding environment image;

[0329] Inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, so as to obtain the environmental deep features of the surrounding environment image;

[0330] Inputting the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image;

[0331] The global features and the surrounding environment image are input into the content semantic branch network to extract semantic features, so as to obtain the environmental content features of the surrounding environment image.

[0332] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0333] Inputting the environment depth feature and the cloud depth feature into a first feature fusion network for feature fusion to obtain the fused depth feature;

[0334] Inputting the environment bird's-eye view feature and the cloud bird's-eye view feature into a second feature fusion network for feature fusion to obtain the fused bird's-eye view feature;

[0335] The environmental content features and the cloud content features are input into a third feature fusion network for feature fusion to obtain the fused content features.

[0336] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0337] Sending the navigation path of the vehicle to the cloud server;

[0338] Receiving intersection map features returned by the cloud server based on the navigation path and the location information, and constructing an intersection map according to the intersection map features;

[0339] The intersection map is displayed on the display interface.

[0340] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0341] When the vehicle is started, determining whether the vehicle-mounted device meets a preset map building condition;

[0342] When the vehicle-mounted device meets the preset map building condition, the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor and the location information of the vehicle is performed;

[0343] The preset map construction condition includes any of the following:

[0344] The in-vehicle device responds to a triggering operation of a map generation control or a navigation control on a display interface of the in-vehicle device by a user;

[0345] The vehicle-mounted device is powered on and started;

[0346] The vehicle-mounted device detects that the vehicle travels to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

[0347] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0348] Receiving a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle;

[0349] Determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library;

[0350] The cloud map feature is sent to the vehicle-mounted device; the cloud map feature is used to instruct the vehicle-mounted device to construct a target map based on the cloud map feature and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

[0351] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0352] Receiving the navigation path sent by the vehicle-mounted device;

[0353] Determining the next intersection position of the vehicle according to the navigation path and the position information;

[0354] Extracting an intersection map feature corresponding to the next intersection position from the map feature library;

[0355] The intersection map feature is sent to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

[0356] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0357] Receiving other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by on-board sensors on the other vehicles;

[0358] Inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features;

[0359] The other cloud map features are associated with the map location corresponding to the location information, and the other cloud map features are stored in the map feature library, and the association relationship between the other cloud map features and the map location is recorded in the map feature library.

[0360] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0361] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0362] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A map construction method, characterized in that: The method is applied to a vehicle-mounted device, and the method comprises: Obtaining surrounding environment images collected by on-board sensors on the vehicle and location information of the vehicle; Sending a map acquisition request to a cloud server, and receiving a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image; the cloud map feature is determined by the cloud server according to the location information and the surrounding environment image; Constructing a target map according to the surrounding environment image and the cloud map features; The target map is displayed on a display interface on the vehicle-mounted device.

2. The method according to claim 1, characterized in that The step of constructing a target map according to the surrounding environment image and the cloud map features includes: Inputting the surrounding environment image into the vehicle-side neural network model for feature encoding to obtain environment map features; the environment map features include environment depth features, environment bird's-eye view features, and environment content features; Fusing the environment map features with the cloud map features to obtain target fusion features; The target fusion features are input into a map decoder to construct the target map.

3. The method according to claim 2, characterized in that The vehicle-side neural network model includes a feature extraction network, a deep branch network, a feature conversion network, and a content semantic branch network; the surrounding environment image is input into the vehicle-side neural network model for feature encoding to obtain the environment map features, including: Inputting the surrounding environment image into the feature extraction network to extract global features, so as to obtain global features of the surrounding environment image; Inputting the global features and the surrounding environment image into the deep branch network to extract the deep features, so as to obtain the environmental deep features of the surrounding environment image; Inputting the global features and the surrounding environment image into the feature conversion network for feature conversion to obtain the environmental bird's-eye view features of the surrounding environment image; The global features and the surrounding environment image are input into the content semantic branch network to extract semantic features, so as to obtain the environmental content features of the surrounding environment image.

4. The method according to claim 2, characterized in that: The target fusion features include fusion depth features, fusion bird's-eye view features and fusion content features, and the cloud map features include cloud depth features, cloud bird's-eye view features and cloud content features; The step of fusing the environment map features with the cloud map features to obtain target fusion features includes: Inputting the environment depth feature and the cloud depth feature into a first feature fusion network for feature fusion to obtain the fused depth feature; Inputting the environment bird's-eye view feature and the cloud bird's-eye view feature into a second feature fusion network for feature fusion to obtain the fused bird's-eye view feature; The environmental content features and the cloud content features are input into a third feature fusion network for feature fusion to obtain the fused content features.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Sending the navigation path of the vehicle to the cloud server; Receiving intersection map features returned by the cloud server based on the navigation path and the location information, and constructing an intersection map according to the intersection map features; The intersection map is displayed on the display interface.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the vehicle is started, determining whether the vehicle-mounted device meets a preset map building condition; When the vehicle-mounted device meets the preset map building condition, the step of acquiring the surrounding environment image collected by the vehicle-mounted sensor and the location information of the vehicle is performed; The preset map construction condition includes any of the following: The in-vehicle device responds to a triggering operation of a map generation control or a navigation control on a display interface of the in-vehicle device by a user; The vehicle-mounted device is powered on and started; The vehicle-mounted device detects that the vehicle travels to a target road; the target road includes any one of a curve, a ramp, an intersection, a narrow road, and an accident road.

7. A map construction method, characterized in that: The method is applied to a cloud server, and the method includes: Receiving a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle; Determine a map location according to the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library; The cloud map feature is sent to the vehicle-mounted device; the cloud map feature is used to instruct the vehicle-mounted device to construct a target map based on the cloud map feature and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

8. The method according to claim 7, characterized in that The method further comprises: Receiving the navigation path sent by the vehicle-mounted device; Determining the next intersection position of the vehicle according to the navigation path and the position information; Extracting an intersection map feature corresponding to the next intersection position from the map feature library; The intersection map feature is sent to the vehicle-mounted device; the intersection map feature is used to instruct the vehicle-mounted device to construct an intersection map according to the intersection map feature, and display the intersection map on a display interface on the vehicle-mounted device.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: Receiving other environmental images and location information sent by other vehicles within a preset time period; the other environmental images are environmental images collected by on-board sensors on the other vehicles; Inputting the other environment images into the vehicle-side neural network model for feature encoding to obtain other cloud map features; The other cloud map features are associated with the map location corresponding to the location information, and the other cloud map features are stored in the map feature library, and the association relationship between the other cloud map features and the map location is recorded in the map feature library.

10. A map construction system, characterized in that: The system includes vehicle-mounted sensors, vehicle-mounted equipment, and a cloud server; The vehicle-mounted device is used to execute the method according to any one of claims 1 to 6; The cloud server is used to execute the method according to any one of claims 7 to 9.

11. A map construction device, characterized in that: The device comprises: An acquisition module is used to acquire the surrounding environment image collected by the vehicle-mounted sensor on the vehicle, as well as the location information of the vehicle; A sending and receiving module, used to send a map acquisition request to a cloud server and receive a cloud map feature sent by the cloud server; the map acquisition request carries the location information and the surrounding environment image; A construction module, used to construct a target map according to the surrounding environment image and the cloud map features; The display module is used to display the target map on a display interface on the vehicle-mounted device.

12. A map construction device, characterized in that: The device comprises: A receiving module, used to receive a map acquisition request sent by an on-board device on a vehicle; the map acquisition request carries the location information of the vehicle and an image of the surrounding environment collected by an on-board sensor on the vehicle; A determination module, configured to determine a map location based on the location information and the surrounding environment image, and extract a cloud map feature corresponding to the map location from a map feature library; A sending module is used to send the cloud map features to the vehicle-mounted device; the cloud map features are used to instruct the vehicle-mounted device to construct a target map based on the cloud map features and the surrounding environment image, and display the target map on the display interface on the vehicle-mounted device.

13. A vehicle, characterized in that: The vehicle includes an on-board device, the on-board device includes a memory and a processor, the memory stores a computer program, and is characterized in that the processor implements the steps of any one of claims 1 to 6 when executing the computer program.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • SLAM map construction method and related equipment

    CN114089362A

  • Map generation method and device, electronic equipment and storage medium

    CN114494618A

  • Map generation method and apparatus, electronic device, and storage medium

    WO2023123837A1