A vehicle driving and parking map construction method and device, electronic equipment and medium

Abstract

This application provides a method, apparatus, electronic device, and medium for constructing a vehicle driving and parking map. The method includes: acquiring mapping images of the surrounding environment during vehicle teaching mapping and obtaining mapping data corresponding to each frame of the mapping image; performing teaching mapping based on the mapping data to obtain an initial map; determining whether the vehicle is located at the driving and parking boundary based on the mapping data; if so, determining the driving area and parking area in the initial map based on the vehicle's speed before and after reaching the driving and parking boundary; and optimizing the map elements included in the driving area and parking area respectively based on the vehicle's driving characteristics during driving and parking to obtain the vehicle's driving and parking map. Since the electronic device can first construct the initial map for both the driving and parking areas in the same way, and then optimize the map elements in different areas, it achieves integrated construction of driving and parking maps and improves map accuracy.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, and medium for constructing a vehicle parking map. Background Technology

[0002] Currently, the main application scenarios of autonomous driving technology include memory parking in parking lots and autonomous driving on open roads. Both of these require pre-built maps to achieve autonomous driving.

[0003] However, for memory parking and autonomous driving, the complexity of the scenarios, the frequency of scenario updates, and the vehicle speeds are all different. For example, the requirements for mapping elements are different, the driving speed is much higher than the parking speed, the requirements for positioning security are higher, and the probability of scenario changes is higher when driving compared to parking. The map update requirements in the driving scenario are also higher. Therefore, the current methods for building maps for the two are also different. Maps need to be built separately for driving and parking scenarios, which is a rather cumbersome process. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, electronic device, and medium for constructing a vehicle driving and parking map, so as to achieve the integrated construction of driving and parking maps. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a method for constructing a vehicle parking map, the method comprising:

[0006] During the vehicle teaching mapping process, mapping images of the surrounding environment are collected, and mapping data corresponding to each frame of the mapping image is obtained. The mapping data includes visual feature elements and vehicle pose.

[0007] Based on the mapping data, a teaching mapping is performed to obtain an initial map;

[0008] For each frame of the mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the intersection of driving and parking, wherein the target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images;

[0009] If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after it is located at the boundary between driving and parking.

[0010] Based on the driving characteristics of the vehicle during driving and the driving characteristics of the vehicle during parking, the map elements included in the driving area and the parking area are optimized respectively to obtain the driving and parking map of the vehicle.

[0011] Optionally, the mapping data may also include satellite solution data;

[0012] Determining whether the vehicle is located at the intersection of parking and driving zones based on the mapping data corresponding to the target mapping image includes:

[0013] Based on the visual feature elements corresponding to each frame of the target mapping image, the existence probability of a preset category element in each frame of the target mapping image is determined, wherein the preset category element is the feature element of the parking area entrance;

[0014] Based on the vehicle pose corresponding to the target mapping image, determine the vehicle slope information corresponding to the mapping image frame.

[0015] Based on the number of satellite solutions corresponding to the target mapping image, the satellite solution result corresponding to the mapping image frame is determined;

[0016] Based on the existence probability, the vehicle slope information, and the satellite calculation results, it is determined whether the vehicle is located at the intersection of parking and driving.

[0017] Optionally, determining whether the vehicle is located at the intersection of parking and driving zones based on the existence probability, the vehicle slope information, and the satellite calculation results includes:

[0018] Based on the existence probability, the vehicle slope information, and the satellite calculation results, the probability that the vehicle is located at the boundary between parking and berthing is determined as the boundary probability between parking and berthing.

[0019] Determine whether the probability of the berth boundary is greater than a first preset threshold;

[0020] If the probability of the parking boundary is greater than the first preset threshold, the vehicle is determined to be located at the parking boundary.

[0021] Optionally, determining the probability that the vehicle is located at the intersection of parking and driving zones based on the existence probability, the vehicle slope information, and the satellite calculation results includes:

[0022] Based on the existence probability, the vehicle slope information, and the satellite calculation results, the probability p1 of the vehicle being located at the intersection of parking and berthing is calculated according to the following formula:

[0023]

[0024] in, This represents the probability of the existence of a preset category element j in the i-th frame of the target mapping image. and σ θ Let represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image, respectively; Δn represents the difference between the number of satellite solutions corresponding to that frame and the number of satellite solutions corresponding to the first frame of the target mapping image; α v α r and α g These are the visual probability coefficient, the slope probability coefficient, and the satellite probability coefficient, respectively.

[0025] Optionally, after obtaining the parking map of the vehicle, the method further includes:

[0026] During vehicle operation, positioning images of the surrounding environment are collected, and positioning feature point clouds corresponding to each frame of positioning image are obtained. The positioning feature point clouds include rod-shaped feature point clouds and area-shaped feature point clouds.

[0027] The location feature point cloud is matched with map elements in the berthing map to obtain the matching result;

[0028] Based on the matching results, the positioning pose of the vehicle is determined.

[0029] Optionally, after determining the vehicle's positioning pose based on the matching result, the method further includes:

[0030] For each target positioning feature point cloud included in each frame of positioning image, a local coordinate system corresponding to each target positioning feature point cloud is constructed, and the transformation matrix between the local coordinate system and the coordinate system of the berthing map is determined, wherein the target positioning feature point cloud is the positioning feature point cloud that successfully matches the map element in the berthing map;

[0031] For each target localization feature point cloud, based on the preset correspondence between local positioning confidence and feature point cloud type, the target local positioning confidence corresponding to the target localization feature point cloud is determined; wherein, the local positioning confidence represents the constraint of the corresponding feature point cloud on positioning in the local coordinate system;

[0032] Based on the transformation matrix and the target local positioning confidence, the global positioning confidence of the target positioning feature point cloud in the coordinate system of the mooring map is determined;

[0033] The global positioning confidence corresponding to each target localization feature point cloud is fused to obtain the target positioning confidence corresponding to the localization image of that frame.

[0034] Optionally, determining the transformation matrix between the local coordinate system and the coordinate system of the mooring map includes:

[0035] Based on the coordinate axis direction vectors of the local coordinate system, the rotation transformation matrix between the local coordinate system and the coordinate system of the mooring map is calculated according to the following rotation relationship matrix calculation formula.

[0036]

[0037] Where, n x n represents the direction vector of the x-axis of the local coordinate system. y n represents the direction vector of the y-axis of the local coordinate system. z This represents the direction vector of the z-axis of the local coordinate system;

[0038] Based on the rotation transformation matrix Using the coordinates of feature points in the target location feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the mooring map according to the following transformation matrix calculation formula.

[0039]

[0040] Where t represents the coordinates of the feature point in the target localization feature point cloud.

[0041] Optionally, determining the global positioning confidence of the target's localization feature point cloud in the coordinate system of the mooring map based on the transformation matrix and the target's local positioning confidence includes:

[0042] Based on the transformation matrix, calculate the first-order approximate Jacobian matrix used for error propagation according to the following Jacobian matrix calculation formula.

[0043]

[0044] Among them, J r Denote the left and right Jacobian functions on Lie algebras. Transformation matrix The Lie algebra representation of Ad, where Ad denotes the computation of the adjoint matrix function;

[0045] Based on the first-order approximate Jacobian matrix Based on the target's local location confidence, the global location confidence Ω of the target's location feature point cloud in the coordinate system of the mooring map is calculated according to the following global location confidence calculation formula. M :

[0046]

[0047] Among them, Ω L The matrix representation of the local location confidence of the target.

[0048] Optionally, after obtaining the target location confidence corresponding to the frame location image, the method further includes:

[0049] Based on the target positioning confidence corresponding to each frame of positioning image, determine whether the positioning pose corresponding to each frame of positioning image is accurate;

[0050] For a positioning image with accurate pose, extract the visual feature elements of that positioning image frame as the target element;

[0051] Determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map;

[0052] If discrepancies exist, the mooring map is updated based on the target location confidence and the target elements.

[0053] Optionally, updating the mooring map based on the target location confidence and the target elements includes:

[0054] Based on the feature point cloud corresponding to the target element, determine the centroid coordinates of the target element;

[0055] The target location confidence is transformed by Lie algebra vector transformation to obtain the confidence rotation and translation matrix;

[0056] The observation error corresponding to the target element is calculated based on the centroid coordinates and the confidence level rotation and translation matrix.

[0057] Based on the observation error and the detection probability corresponding to the target element, the fusion weight corresponding to the target element is calculated.

[0058] For each target element, based on the fusion weight corresponding to the target element in each frame of the target localization image, the target element in each frame of the target localization image is fused to obtain the fused target element. The target localization image includes the current frame localization image and a preset number of previous frames of localization images.

[0059] The berthing map is updated based on the fused target elements.

[0060] Optionally, the step of calculating the observation error corresponding to the target element based on the centroid coordinates and the confidence rotation / translation matrix includes:

[0061] Based on the centroid coordinates and the confidence level rotation / translation matrix, the observation error e corresponding to the target element is calculated according to the following observation error calculation formula. m :

[0062] e m =||T err P||

[0063] Among them, T err Let P be the confidence level rotation and translation matrix, and let P be the centroid coordinates.

[0064] The calculation of the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element includes:

[0065] Based on the observation error and the detection probability corresponding to the target element, the fusion weight ω corresponding to the target element is calculated according to the following fusion weight calculation formula. m :

[0066] ω m =a m / e m

[0067] Among them, a m The detection probability is the probability corresponding to the target element.

[0068] Optionally, updating the mooring map based on the fused target elements includes:

[0069] For each fused target element, calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element.

[0070] Search for map elements of the same type as the fused target element within a preset range in the berthing map, and calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element;

[0071] Based on the second distance and the fusion error, the update probability corresponding to the fused target element is determined, wherein the fusion error is the mean of the first distance corresponding to each target element;

[0072] If the update probability is greater than the second preset threshold, the fused target elements are used to update the parking map.

[0073] Optionally, determining the update probability corresponding to the fused target element based on the distance and the fusion error includes:

[0074] Based on the distance and fusion error, the update probability p2 is determined according to the following update probability calculation formula:

[0075]

[0076] Where α is a preset coefficient, n represents the total number of frames included in the target positioning image, d represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element, and e represents the fusion error.

[0077] Secondly, embodiments of this application also provide a vehicle parking map construction apparatus, the apparatus comprising:

[0078] The mapping data acquisition module is used to collect mapping images of the surrounding environment during the vehicle teaching mapping process and acquire mapping data corresponding to each frame of mapping image, wherein the mapping data includes visual feature elements and vehicle pose.

[0079] An initial map construction module is used to perform teaching map construction based on the map construction data to obtain an initial map;

[0080] The boundary location determination module is used to determine whether the vehicle is located at the parking boundary location for each frame of the mapping image based on the mapping data corresponding to the target mapping image. The target mapping image includes the current frame of the mapping image and a preset number of previous frames of the mapping image.

[0081] The driving and parking area determination module is used to determine the driving area and parking area in the initial map based on the speed of the vehicle before it is at the driving and parking boundary position and the speed of the vehicle after it is at the driving and parking boundary position when the judgment result of the boundary position determination module is yes.

[0082] The map element optimization module is used to optimize the map elements included in the driving area and the parking area based on the driving characteristics of the vehicle when driving and the driving characteristics of the vehicle when parking, respectively, to obtain the driving and parking map of the vehicle.

[0083] Optionally, the mapping data may also include satellite solution data;

[0084] The boundary location determination module includes:

[0085] The existence probability determination unit is used to determine the existence probability of a preset category element in each frame of the target mapping image based on the visual feature elements corresponding to each frame of the target mapping image, wherein the preset category element is the feature element of the parking area entrance;

[0086] The vehicle slope information determination unit is used to determine the vehicle slope information corresponding to the frame of the mapping image based on the vehicle pose corresponding to the target mapping image.

[0087] The satellite solution result determination unit is used to determine the satellite solution result corresponding to the frame of the map image based on the number of satellite solutions corresponding to the target map image;

[0088] The boundary location determination unit is used to determine whether the vehicle is located at the boundary of the parking area based on the existence probability, the vehicle slope information, and the satellite calculation results.

[0089] Optionally, the boundary location determination unit includes:

[0090] The boundary probability determination subunit is used to determine the probability that the vehicle is located at the boundary between parking and berthing based on the existence probability, the vehicle slope information and the satellite calculation results, as the parking boundary probability;

[0091] The boundary probability judgment subunit is used to determine whether the boundary probability of the berthing is greater than a first preset threshold.

[0092] The boundary location determination subunit is used to determine that the vehicle is located at the boundary location when the probability of the parking boundary is greater than the first preset threshold.

[0093] Optionally, the boundary probability determination subunit is specifically used to calculate the probability p1 that the vehicle is located at the boundary between parking and berthing, based on the existence probability, the vehicle slope information, and the satellite calculation results, according to the following parking boundary probability calculation formula:

[0094]

[0095] in, This represents the probability of the existence of a preset category element j in the i-th frame of the target mapping image. and σ θ Let represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image, respectively; Δn represents the difference between the number of satellite solutions corresponding to that frame and the number of satellite solutions corresponding to the first frame of the target mapping image; α v α r and α g These are the visual probability coefficient, the slope probability coefficient, and the satellite probability coefficient, respectively.

[0096] Optionally, the device further includes:

[0097] The positioning feature point cloud acquisition module is used to collect positioning images of the surrounding environment during vehicle operation and acquire the positioning feature point cloud corresponding to each frame of positioning image. The positioning feature point cloud includes rod-shaped object feature point cloud and area-shaped object feature point cloud.

[0098] The matching module is used to match the positioning feature point cloud with map elements in the mooring map to obtain the matching result;

[0099] The positioning module is used to determine the positioning pose of the vehicle based on the matching results.

[0100] Optionally, the device further includes:

[0101] The transformation matrix determination module is used to construct a local coordinate system corresponding to each target positioning feature point cloud for each target positioning feature point cloud included in each frame positioning image, and determine the transformation matrix between the local coordinate system and the coordinate system of the berthing map, wherein the target positioning feature point cloud is a positioning feature point cloud that successfully matches the map elements in the berthing map;

[0102] The target local positioning confidence determination module is used to determine the target local positioning confidence corresponding to each target positioning feature point cloud based on a preset correspondence between local positioning confidence and feature point cloud type; wherein, the local positioning confidence represents the constraint of the corresponding feature point cloud on positioning in the local coordinate system.

[0103] The global positioning confidence determination module is used to determine the global positioning confidence of the target positioning feature point cloud in the coordinate system of the mooring map based on the transformation matrix and the target local positioning confidence;

[0104] The target location confidence determination module is used to fuse the global location confidence corresponding to each target location feature point cloud to obtain the target location confidence corresponding to the frame location image.

[0105] Optionally, the transformation matrix determination module is specifically used to calculate the rotation transformation matrix between the local coordinate system and the coordinate system of the mooring map based on the coordinate axis direction vectors of the local coordinate system, according to the following rotation relationship matrix calculation formula.

[0106]

[0107] Where, n x n represents the direction vector of the x-axis of the local coordinate system. y n represents the direction vector of the y-axis of the local coordinate system. z This represents the direction vector of the z-axis of the local coordinate system;

[0108] Based on the rotation transformation matrix Using the coordinates of feature points in the target location feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the mooring map according to the following transformation matrix calculation formula.

[0109]

[0110] Where t represents the coordinates of the feature point in the target localization feature point cloud.

[0111] Optionally, the global location confidence determination module includes:

[0112] The first-order approximate Jacobian matrix calculation unit is used to calculate, based on the transformation matrix and according to the following Jacobian matrix calculation formula, the first-order approximate Jacobian matrix used for error propagation.

[0113]

[0114] Among them, J r Denote the left and right Jacobian functions on Lie algebras. Transformation matrix The Lie algebra representation of Ad, where Ad denotes the computation of the adjoint matrix function;

[0115] The global location confidence calculation unit is used to calculate the confidence level based on the first-order approximate Jacobian matrix. Based on the target's local location confidence, the global location confidence Ω of the target's location feature point cloud in the coordinate system of the mooring map is calculated according to the following global location confidence calculation formula. M :

[0116]

[0117] Among them, Ω L The matrix representation of the local location confidence of the target.

[0118] Optionally, the device further includes:

[0119] The localization pose determination module is used to determine whether the localization pose corresponding to each frame of localization image is accurate based on the target localization confidence level corresponding to each frame of localization image.

[0120] The target element determination module is used to extract the visual feature elements of the positioning image frame with accurate positioning pose, and use them as target elements.

[0121] The difference judgment module is used to determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map;

[0122] The map element update module is used to update the mooring map based on the target location confidence and the target element when the difference judgment module determines that the result is yes.

[0123] Optionally, the map element update module includes:

[0124] The centroid coordinate determination unit is used to determine the centroid coordinates of the target element based on the feature point cloud corresponding to the target element.

[0125] The confidence level rotation and translation matrix determination unit is used to perform Lie algebra vector transformation on the target location confidence to obtain the confidence level rotation and translation matrix;

[0126] The observation error calculation unit is used to calculate the observation error corresponding to the target element based on the centroid coordinates and the confidence rotation and translation matrix.

[0127] The fusion weight calculation unit is used to calculate the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element.

[0128] The target element fusion unit is used to fuse the target element in each frame of the target positioning image based on the fusion weight corresponding to the target element in each frame of the target positioning image, so as to obtain the fused target element. The target positioning image includes the positioning image of the frame and a preset number of positioning images before the positioning image of the frame.

[0129] The map element update unit is used to update the mooring map based on the fused target elements.

[0130] Optionally, the observation error calculation unit is specifically used to calculate the observation error e corresponding to the target element based on the centroid coordinates and the confidence rotation / translation matrix, according to the following observation error calculation formula. m :

[0131] e m =||T err P||

[0132] Among them, T err Let P be the confidence level rotation and translation matrix, and let P be the centroid coordinates.

[0133] The fusion weight calculation unit is specifically used to calculate the fusion weight ω corresponding to the target element based on the observation error and the detection probability corresponding to the target element, according to the following fusion weight calculation formula. m :

[0134] ω m =a m / e m

[0135] Among them, a m The detection probability is the probability corresponding to the target element.

[0136] Optionally, the map element update unit includes:

[0137] The centroid coordinate difference calculation subunit is used to calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element for each fused target element.

[0138] The centroid coordinate distance calculation subunit is used to search for map elements of the same type as the fused target element within a preset range in the berthing map, and to calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element.

[0139] The update probability determination subunit is used to determine the update probability corresponding to the fused target element based on the second distance and the fusion error, wherein the fusion error is the mean of the first distance corresponding to each target element;

[0140] The map element update subunit is used to update the mooring map using the fused target elements when the update probability is greater than a second preset threshold.

[0141] Optionally, the update probability determination subunit is specifically used to determine the update probability p2 based on the distance and the fusion error, according to the following update probability calculation formula:

[0142]

[0143] Where α is a preset coefficient, n represents the total number of frames included in the target positioning image, d represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element, and e represents the fusion error.

[0144] Thirdly, embodiments of this application also provide an electronic device, including:

[0145] Memory, used to store computer programs;

[0146] When a processor executes a program stored in memory, it implements the method described in the first aspect above.

[0147] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method described in the first aspect above.

[0148] Beneficial effects of the embodiments in this application:

[0149] In the solution provided in this application embodiment, the electronic device can acquire mapping images of the surrounding environment during vehicle teaching mapping and obtain mapping data corresponding to each frame of mapping image. The mapping data includes visual feature elements and vehicle pose. Based on the mapping data, teaching mapping is performed to obtain an initial map. For each frame of mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the boundary between driving and parking. The target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images. If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after the boundary between driving and parking. Based on the vehicle's driving characteristics and parking characteristics, the map elements included in the driving area and parking area are optimized respectively to obtain the vehicle's driving and parking map. Because electronic devices can first construct an initial map for both driving and parking areas using the same method, and then determine the boundary between driving and parking areas based on this initial map, thus distinguishing between them, and then optimize the map elements included in the driving and parking areas separately according to the different driving characteristics of vehicles during driving and parking. Therefore, the constructed map can not only include both driving and parking areas simultaneously, achieving integrated construction of driving and parking maps, but also optimize the map elements according to the different driving characteristics during driving and parking, further improving the accuracy of the map.

[0150] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0151] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0152] Figure 1A flowchart illustrating a method for constructing a vehicle parking map as provided in this application embodiment;

[0153] Figure 2 Based on Figure 1 A flowchart illustrating a method for determining the boundary position of berths in the illustrated embodiment;

[0154] Figure 3 Based on Figure 2 A specific flowchart illustrating a method for determining the boundary position of berths in the illustrated embodiment;

[0155] Figure 4 Based on Figure 1 A flowchart of a vehicle positioning method according to the embodiment shown;

[0156] Figure 5 Based on Figure 4 A flowchart illustrating a target location confidence determination method in the embodiment shown;

[0157] Figure 6 Based on Figure 5 The illustrated embodiment is a schematic diagram of constructing a local coordinate system and determining the transformation matrix;

[0158] Figure 7 Based on Figure 5 A flowchart of a map update method according to the embodiment shown;

[0159] Figure 8 Based on Figure 7 A specific flowchart of the map update method in the illustrated embodiment;

[0160] Figure 9 Based on Figure 8 A specific flowchart of the map update method in the illustrated embodiment;

[0161] Figure 10 Based on Figure 1 A schematic diagram of a sensor equipped in a vehicle according to the embodiment shown;

[0162] Figure 11 This is a specific flowchart of a method for constructing a vehicle parking map based on the embodiment shown in Figure 10;

[0163] Figure 12 This is a flowchart of a map construction method based on the embodiment shown in 11;

[0164] Figure 13 This is a flowchart of the positioning method based on the embodiment shown in Figure 11;

[0165] Figure 14 This is a flowchart of a map update method based on the embodiment shown in 11;

[0166] Figure 15 A schematic diagram of a vehicle parking map construction device provided in an embodiment of this application;

[0167] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0168] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0169] In the solutions provided in this application, the collection, storage, use, processing, transmission, provision, and disclosure of vehicle information are all performed with the user's knowledge and authorization, and comply with relevant laws and regulations, and do not violate public order and good morals.

[0170] To achieve the integrated construction of driving maps and parking maps, this application provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for constructing vehicle driving and parking maps. The following is a description of a method for constructing vehicle driving and parking maps provided by this application.

[0171] The method for constructing a vehicle parking map provided in this application can be applied to any electronic device that needs to construct a vehicle parking map, such as a processor or terminal device connected to various vehicle sensors on a vehicle, or a server remotely connected to vehicle sensors, etc. There are no specific limitations here. For clarity, it will be referred to as an electronic device below.

[0172] like Figure 1 As shown, a method for constructing a vehicle parking map includes:

[0173] S101, during the vehicle teaching mapping process, collect mapping images of the surrounding environment and obtain mapping data corresponding to each frame of the mapping image;

[0174] The mapping data includes visual feature elements and vehicle pose.

[0175] S102, Based on the mapping data, perform teaching mapping to obtain an initial map;

[0176] S103, for each frame of the mapping image, based on the mapping data corresponding to the target mapping image, determine whether the vehicle is located at the intersection of parking and driving;

[0177] The target mapping image includes the current mapping image and a preset number of previous mapping images.

[0178] S104, If the vehicle is located at the intersection of driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before it is at the intersection and the vehicle's speed after it is at the intersection.

[0179] S105, based on the driving characteristics of the vehicle when driving and the driving characteristics of the vehicle when parking, the map elements included in the driving area and the parking area are optimized respectively to obtain the driving and parking map of the vehicle.

[0180] In the solution provided in this application embodiment, the electronic device can acquire mapping images of the surrounding environment during vehicle teaching mapping and obtain mapping data corresponding to each frame of mapping image. The mapping data includes visual feature elements and vehicle pose. Based on the mapping data, teaching mapping is performed to obtain an initial map. For each frame of mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the boundary between driving and parking. The target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images. If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after the boundary between driving and parking. Based on the vehicle's driving characteristics and parking characteristics, the map elements included in the driving area and parking area are optimized respectively to obtain the vehicle's driving and parking map. Because electronic devices can first construct an initial map for both driving and parking areas using the same method, and then determine the boundary between driving and parking areas based on this initial map, thus distinguishing between them, and then optimize the map elements included in the driving and parking areas separately according to the different driving characteristics of vehicles during driving and parking. Therefore, the constructed map can not only include both driving and parking areas simultaneously, achieving integrated construction of driving and parking maps, but also optimize the map elements according to the different driving characteristics during driving and parking, further improving the accuracy of the map.

[0181] When a map needs to be built for a certain area to guide vehicle parking, the user can drive the vehicle in that area. The vehicle's movement trajectory is recorded by sensors such as cameras, radar, and inertial navigation equipment on the vehicle, and the surrounding environment is observed and scanned to obtain various elements in the area and their positions, which is to perform the vehicle teaching mapping process.

[0182] After the vehicle teaching mapping is completed, that is, after the vehicle has collected elements in the entire area, or during the vehicle teaching mapping process, that is, during the real-time collection of elements in the area, the electronic equipment can acquire various mapping images of the vehicle's surrounding environment collected by the vehicle's cameras, radar and other sensors, as well as the inertial navigation information recorded by the vehicle's inertial navigation equipment.

[0183] In one implementation, after acquiring the mapping image and inertial navigation information, the electronic device can perform invalid data filtering on the mapping image and inertial navigation information to reduce the amount of subsequent data processing. It can also perform time synchronization on the mapping image and inertial navigation information to establish a correspondence between the mapping image and the inertial navigation information.

[0184] After acquiring the aforementioned mapping images and inertial navigation information, the electronic device can extract the necessary elements in the mapping images, namely the elements used for vehicle positioning, and obtain visual feature elements such as lane lines, traffic lights, signs, and streetlights included in each mapping image. Based on the inertial navigation information recorded by the aforementioned inertial navigation device, the relative vehicle pose between the acquired mapping images is obtained through matching or recursion, and then the aforementioned visual feature elements and vehicle pose are used as mapping data.

[0185] In one implementation, in order to improve the extraction accuracy of the above-mentioned visual feature elements, after the electronic device acquires the visual feature elements in each mapping image, it can take a preset number of adjacent mapping images as a mapping image group, and then perform weighted optimization on the same visual feature elements in multiple mapping images according to the distance of observation to obtain optimized visual feature elements.

[0186] After acquiring the aforementioned visual feature elements and vehicle pose, the electronic device can determine which elements are present in the region based on the extracted visual feature elements, and further determine the location of each element based on the vehicle pose. Therefore, the electronic device can perform teaching mapping based on the aforementioned visual feature elements and vehicle pose to obtain an initial map. For example, it can generate vector information of necessary elements in the region based on the aforementioned visual feature elements and vehicle pose, and then generate an initial map based on the vector information of each necessary element.

[0187] Because the generated initial map includes both driving and parking areas, and the complexity, update frequency, and vehicle speed of these areas differ, the electronic device can further determine the boundaries between driving and parking areas after generating the initial map. This allows for the identification of the driving and parking areas within the initial map, and different types of optimization can be applied to the map elements within these areas. Specifically:

[0188] For each frame of the mapping image, the electronic device can determine whether the vehicle is located at the boundary between parking and driving zones when the frame is acquired, based on the mapping data of that frame and a preset number of previous frames. In other words, it can determine all parking and driving zone locations included in the initial map by using the mapping data corresponding to the target image. The preset number can be a pre-determined number, or, since mapping images with the same visual features may have been acquired at the same location, the preset number can be determined in real-time based on the visual features included in each mapping image, identifying the number of other mapping images with the same visual features preceding that frame. No specific limitation is made here.

[0189] For example, electronic devices can determine whether a vehicle is located at the boundary between parking and driving zones based on whether the visual feature elements in the mapping data are the same as those at the entrance of the parking area; they can also determine whether the vehicle's pitch angle or height information has changed significantly based on the vehicle's pose in the mapping data; they can also determine whether the vehicle has moved from indoors to outdoors or from outdoors to indoors based on changes in the connection quality of the external signals connected to the vehicle or changes in the brightness of the mapping image; and they can also combine the above indicators to determine whether the vehicle is located at the boundary between parking and driving zones, and so on.

[0190] Because the speed of a vehicle in a driving area is relatively faster than that in a parking area, the electronic equipment can determine the driving area and parking area in the initial map based on the vehicle's speed before and after entering the driving and parking boundary positions, after traversing all driving and parking boundary positions included in the initial map.

[0191] For example, electronic devices can determine the vehicle's average speed over a preset time period before it reaches the boundary between driving and parking, and the average speed over a preset time period after it reaches the boundary. The area corresponding to the higher average speed is then designated as the driving area, and the other area as the parking area. Alternatively, the device can determine the vehicle's maximum speed before and after reaching the boundary, and the area corresponding to the higher maximum speed is designated as the driving area, and the other area as the parking area, and so on.

[0192] After determining the driving area and parking area, the electronic device can optimize the map elements included in the driving area and parking area based on the driving characteristics of the vehicle when driving and the driving characteristics of the vehicle when parking, respectively, to obtain the vehicle driving and parking map.

[0193] In one implementation, the electronic device can determine the element type of each map element included in the driving area and the parking area, and then remove map elements that do not belong to the driving area and the parking area based on a pre-stored correspondence between map element types and driving and parking areas. For example, parking areas generally do not contain traffic lights, so if it is determined that there is a traffic light element among the map elements included in the parking area, it can be considered a misidentification, and the traffic light element can be removed.

[0194] In another implementation, during the map building process, including backend optimization, the requirements for the map are higher for driving areas compared to parking areas. Therefore, electronic devices can control the vehicle to collect more elements through multiple sensors in the driving area and use corresponding element optimization algorithms to determine the accurate elements of the driving area, thus performing extensive map optimization. In the parking area, only a small number of elements can be collected for minor map optimization, thereby reducing the workload of map optimization while ensuring the quality of the optimized map.

[0195] In the solution provided in this application, the electronic device can first construct an initial map for both driving and parking areas in the same way. Then, based on this initial map, the boundary between driving and parking areas is determined to distinguish between them. Furthermore, the map elements included in the driving and parking areas are optimized according to the different driving characteristics of vehicles during driving and parking. Therefore, the constructed map can not only simultaneously include both driving and parking areas, achieving integrated construction of driving and parking maps, but also optimize the map elements according to the different driving characteristics during driving and parking, further improving the map's accuracy.

[0196] As one implementation method of this application, such as Figure 2 As shown, the mapping data mentioned above may also include satellite solution data; determining whether the vehicle is located at the intersection of parking and driving areas based on the mapping data corresponding to the target mapping image may include:

[0197] S201, determine the existence probability of preset category elements in each frame of target mapping image based on the visual feature elements corresponding to each frame of target mapping image;

[0198] Since the boundary between parking and parking areas is usually the entrance area of ​​a parking zone, and this entrance area typically has preset category elements such as parking lot entrance signs and park entrance gates, after acquiring the visual feature elements in the target mapping image, the electronic device can perform target detection on the aforementioned visual feature elements to determine the probability that the aforementioned visual feature elements are preset category elements present at the parking zone entrance. Thus, the maximum probability corresponding to each preset category element is determined as the probability of the existence of the preset category element in each frame of the target mapping image.

[0199] For example, after acquiring visual feature elements a1 and a2 included in the first frame of the mapped image, the electronic device can perform target detection on each visual feature element. If the detection probability of visual feature element a1 being a category A feature element is 50% and that of it being a category B feature element is 30%, and the detection probability of visual feature element a2 being a category A feature element is 30% and that of it being a category B feature element is 60%, then the electronic device can determine that the detection probability of category A feature elements in the first frame of the target mapped image is 50%, and the detection probability of category B feature elements is 60%.

[0200] S202, Based on the vehicle pose corresponding to the target mapping image, determine the vehicle slope information corresponding to the mapping image of that frame;

[0201] Because the vehicle pose corresponding to the target mapping image reflects the vehicle's displacement and height changes when two adjacent target mapping images are acquired, and vehicles usually pass through a ramp when entering an indoor parking lot, the electronic device can determine the slope information of the scene where the vehicle is located when two adjacent target mapping images are acquired based on the vehicle's displacement and height changes included in the vehicle pose corresponding to the target mapping image. Then, based on the slope corresponding to each target mapping image, the electronic device can determine the vehicle slope information corresponding to the last frame of the target mapping image, i.e., the aforementioned frame of the mapping image.

[0202] In one implementation, after determining the slope corresponding to each target mapping image, the electronic device can calculate the average value and variance of each slope, and then use the ratio of the average value to the variance, which reflects the degree of slope change when the above-mentioned target mapping images are collected, as the vehicle slope information corresponding to that frame mapping image.

[0203] S203, Based on the number of satellite solutions corresponding to the target mapping image, determine the satellite solution result corresponding to the mapping image frame;

[0204] Because the vehicle cannot detect satellite signals after entering an indoor space due to the obstruction of walls, the satellite resolution output changes drastically as the vehicle enters and exits the space. Therefore, after acquiring the satellite resolution corresponding to each target mapping image, the electronic equipment can determine the difference in satellite resolution between the first and last frames of each target mapping image, and thus determine whether the vehicle entered the outdoor space or the outdoor space when acquiring the target mapping images.

[0205] S204, based on the existence probability, the vehicle slope information, and the satellite calculation results, determine whether the vehicle is located at the intersection of parking and driving.

[0206] Because the existence probability, vehicle slope information, and satellite calculation results can reflect whether a vehicle has entered the parking area from different dimensions, after obtaining the existence probability, vehicle slope information, and satellite calculation results, the electronic device can determine whether the vehicle is located at the boundary between parking and driving areas based on one of the indicators, or it can comprehensively consider the above multiple indicators to determine whether the vehicle is located at the boundary between parking and driving areas.

[0207] In one implementation, the electronic device can be set with a preset probability threshold corresponding to the probability of existence, a slope threshold corresponding to the vehicle slope information, and a calculation number threshold corresponding to the satellite calculation result. When any of the above indicators is greater than the preset threshold, it is determined that the vehicle is located at the intersection of the parking and driving areas.

[0208] In another implementation, the electronic device can determine the comprehensive coefficient corresponding to each of the above indicators based on the existence probability, vehicle slope information and satellite calculation results, according to a preset algorithm, and determine whether the vehicle is located at the intersection of parking and driving.

[0209] In the solution provided in this application embodiment, the electronic device can determine the existence probability of preset category elements in each frame of the mapping image, the vehicle slope information corresponding to the frame of the mapping image, and the satellite calculation result corresponding to the frame of the mapping image based on the visual feature elements, vehicle pose, and satellite calculation results corresponding to each frame of the mapping image. Then, based on the above existence probability, vehicle skin information, and satellite calculation results, it can accurately determine whether the vehicle is located at the intersection of parking and driving.

[0210] As one implementation method of this application, such as Figure 3 As shown, determining whether a vehicle is located at the intersection of parking and driving zones based on the existence probability, the vehicle slope information, and the satellite calculation results may include:

[0211] S301, Based on the existence probability, the vehicle slope information and the satellite calculation results, determine the probability that the vehicle is located at the boundary between parking and berthing, and use it as the boundary probability between parking and berthing;

[0212] After acquiring the probability of presence, vehicle slope information, and satellite calculation results, the electronic device can determine the comprehensive coefficient corresponding to each of the above indicators according to a preset algorithm, and determine whether the vehicle is located at the intersection of parking and driving based on the comprehensive coefficient.

[0213] In one implementation, the electronic device can calculate the probability p1 of a vehicle being located at the intersection of parking and berthing, based on the existence probability, vehicle slope information, and satellite calculation results, according to the following formula for calculating the probability of parking and berthing intersection:

[0214]

[0215] in, This represents the probability of the existence of element j of the preset category in the i-th frame of the target mapping image. and σ θ Let represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image, respectively; Δn represents the difference between the number of satellite solutions corresponding to that frame and the number of satellite solutions corresponding to the first frame of the target mapping image; α v α r and α g These are the visual probability coefficient, the slope probability coefficient, and the satellite probability coefficient, respectively.

[0216] S302, determine whether the probability of the intersection of the row and berth is greater than a first preset threshold;

[0217] S303, if the probability of the parking boundary is greater than the first preset threshold, determine that the vehicle is located at the parking boundary.

[0218] Because of the above formula for calculating the probability of berthing boundary... and α g |Δn| are used to characterize the probability of the vehicle having a preset category element at the entrance of the parking area in the current scene, the change of the vehicle's slope when driving in the current scene, and the change of the satellite calculation number when driving in the current scene, respectively. The higher the above probability or change, the greater the probability that the vehicle is at the boundary between parking and driving. Therefore, when the probability of parking and driving boundary calculated by the above parking and driving boundary probability calculation formula is greater than the first preset threshold, it can be determined that the vehicle is at the boundary between parking and driving.

[0219] In the solution provided in this application embodiment, the electronic device can accurately calculate the probability that the vehicle is located at the intersection of parking and berth by comprehensively considering the existence probability, vehicle slope information and satellite calculation results through a preset parking boundary probability calculation formula. Then, based on the relationship between the probability that the vehicle is located at the intersection of parking and berth and a first preset threshold, it can accurately determine whether the vehicle is located at the intersection of parking and berth.

[0220] As one implementation method of this application, such as Figure 4 As shown, after obtaining the parking map of the vehicle, the method may further include:

[0221] S401 acquires positioning images of the surrounding environment while the vehicle is in motion, and obtains the positioning feature point cloud corresponding to each frame of positioning image.

[0222] After obtaining the vehicle's parking map, the electronic device can locate the vehicle based on the parking map. The electronic device that builds the vehicle's parking map and the electronic device that locates the vehicle based on the parking map can be the same device or different devices, without specific limitations.

[0223] For example, after constructing a parking map for a vehicle, the same electronic device can locate the vehicle when it re-enters the area corresponding to that map. The accuracy of the location result can then be used to determine the accuracy of the parking map, allowing for map updates and optimizations. Alternatively, after constructing the parking map, the electronic device can send it to other electronic devices. These devices can then use the pre-constructed parking map to automatically drive and remember parking locations when they reach the corresponding areas.

[0224] When electronic devices locate a vehicle based on the parking map, they can first acquire positioning images of the surrounding environment collected by sensors such as cameras and radar on the vehicle during the vehicle's journey. Then, they can extract point clouds of three-dimensional pole-shaped objects such as streetlights or tree trunks from the positioning images, as well as point clouds of three-dimensional planar objects such as signs or walls from the positioning images.

[0225] S402, Match the positioning feature point cloud with the map elements in the mooring map to obtain the matching result;

[0226] S403, Based on the matching result, determine the positioning pose of the vehicle.

[0227] After acquiring the location feature point cloud of the vehicle's current surrounding environment, the electronic device can match this location feature point cloud with map elements in the parking map. When a map element corresponding to the location feature point cloud is matched, the vehicle's positioning pose can be determined based on the location information of that map element. Specifically:

[0228] Because the parking map includes map elements for the entire area, directly matching the localization feature point cloud with map elements in the parking map requires numerous matching steps, resulting in low matching efficiency and slow positioning speed. Therefore, before matching the localization feature point cloud with map elements in the parking map to determine the vehicle's positioning pose, the electronic device can first determine the vehicle's predicted pose. Then, based on this predicted pose, it can select a subset of map elements corresponding to that predicted pose from the map elements included in the parking map. The localization feature point cloud is then matched only with this subset of map elements, thereby improving matching efficiency and positioning speed.

[0229] In one implementation, the electronic device can directly determine the vehicle's predicted pose using initial positioning methods such as RTK (Real-Time Kinematic) or visual feature point initial positioning. For example, the electronic device can acquire the vehicle's predicted pose through satellite positioning measurements based on the RTK initial positioning method.

[0230] In another implementation, during the initial positioning, the electronic device can first determine the vehicle's initial pose using methods such as RTK initial positioning or visual feature point initial positioning. Then, by collecting the positioning feature point cloud in steps S401-S403 and matching it with map elements, the initial vehicle pose is corrected to determine the predicted vehicle pose. In subsequent positioning, the predicted vehicle pose for the current positioning can be derived based on the vehicle pose determined in the previous positioning and the relative vehicle pose between the previous and current positioning determined by the inertial navigation information recorded by the inertial navigation equipment.

[0231] Based on the predicted pose, the electronic device selects a portion of the map elements corresponding to the predicted pose from the map elements included in the parking map. It can then convert these map elements into a map feature point cloud. Subsequently, it uses ICP point cloud registration algorithms, such as point-to-pole and point-to-surface, to match the pose of the localization feature point cloud with the map feature point cloud. Based on the pose matching result, the current position and attitude of the vehicle can be determined, that is, the localization pose of the vehicle can be determined.

[0232] In one implementation, to improve the matching accuracy of matching the location feature point cloud with map elements in the parking map, after acquiring the location feature point cloud corresponding to each frame of the location image, the electronic device can group a preset number of adjacent location images as a location image group. Then, the location feature point clouds in multiple location images are fused according to the relative vehicle pose between each location image to obtain a fused location feature point cloud. Then, a clustering algorithm is used to cluster the fused location feature point cloud and delete the location feature point clouds with too few points to obtain more accurate and richer location feature point cloud observation results. Finally, the optimized location feature point cloud is matched with the map feature point cloud corresponding to the map element for pose matching.

[0233] In the solution provided in this application embodiment, the electronic device can collect positioning images of the surrounding environment during vehicle operation. By matching the positioning feature point cloud in each frame of positioning image with map elements in the constructed driving and parking map, the vehicle is positioned, thereby providing a reference for vehicle positioning pose for automatic driving and memory parking, and realizing autonomous driving of the vehicle.

[0234] As one implementation method of this application, such as Figure 5 As shown, after determining the vehicle's positioning pose based on the matching result, the method further includes:

[0235] S501, for each target positioning feature point cloud included in each frame of positioning image, construct a local coordinate system corresponding to each target positioning feature point cloud, and determine the transformation matrix between the local coordinate system and the coordinate system of the berthing map;

[0236] In addition to outputting high-precision positioning pose, electronic devices can also provide corresponding positioning confidence scores when performing real-time positioning to evaluate the accuracy of positioning. Specifically, electronic devices can start from the positioning constraints provided by the feature points themselves and realize the transfer of positioning constraints by constructing a local coordinate system on the feature points.

[0237] Furthermore, since electronic devices are meant to provide confidence in positioning, it is not necessary to construct a local coordinate system for all feature points in the target positioning feature cloud included in the positioning image. It is only necessary to construct a local coordinate system for the feature points in the positioning feature cloud that successfully match the map elements in the navigation map in each frame of the positioning image, i.e., the feature points in the successfully positioned point cloud.

[0238] Therefore, electronic devices can construct a local coordinate system corresponding to each target positioning feature point cloud for each frame of positioning image, that is, for each frame of positioning image containing positioning feature point clouds that successfully match map elements in the navigation map.

[0239] Specifically, for point clouds of rod-shaped objects and point clouds of area-shaped objects, electronic devices can construct local coordinate systems corresponding to each feature point in the point cloud using different local coordinate system construction methods. However, the origin of the local coordinate system for both is the location of the corresponding feature point; only the orientation of the coordinate system differs. Specifically:

[0240] For each rod feature point in the feature point cloud of rod-shaped objects, such as Figure 6 As shown, the electronic device can calculate the direction vector n of the rod's extension direction based on the distribution of feature points around the feature point of the rod. pz =[n x n y n z ] T This direction is the z-axis direction of the local coordinate system of the feature point of the rod, and the two direction vectors corresponding to the x-axis and y-axis can be determined by the following formula:

[0241] n px =[0 -n z n y ] T n py =n px ×n pz

[0242] For feature points on each surface in a feature point cloud of a planar object, such as Figure 6 As shown, the electronic device can calculate the normal vector n perpendicular to the feature point cloud of the surface based on the distribution of feature points around the feature points on the surface. Fz =[n x n y n z ] T This direction is the z-axis direction of the local coordinate system of the feature point on the surface, and the two direction vectors corresponding to the x-axis and y-axis can be determined by the following formula:

[0243] n px =[0 -n z n y ] T n py =n px ×n pz

[0244] like Figure 6As shown, after determining the local coordinate system corresponding to each feature point in the target positioning feature point cloud, the electronic device can further determine the transformation matrix between the local coordinate system and the coordinate system of the berthing map. That is, it can determine the transformation matrix between the local coordinate system and the map coordinate system corresponding to each pole feature point, and the transformation matrix between the local coordinate system and the map coordinate system corresponding to each surface feature point. Specifically:

[0245] For each feature point in the feature point cloud of rod-shaped objects and the feature point cloud of area-shaped objects, the electronic device can first normalize the direction vectors of the three coordinate axes of the local coordinate system of the feature point according to the following rotation relationship matrix calculation formula, thereby calculating the rotation relationship matrix between the local coordinate system and the coordinate system of the berthing map.

[0246]

[0247] Where, n x n represents the direction vector of the x-axis in the local coordinate system. y n represents the direction vector of the y-axis in the local coordinate system. z This represents the direction vector of the z-axis in the local coordinate system.

[0248] Furthermore, electronic devices can be based on rotation transformation matrices. Using the coordinates of feature points in the target localization feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the map, according to the following transformation matrix calculation formula.

[0249]

[0250] Where t represents the coordinates of the feature point in the target localization feature point cloud.

[0251] Figure 6 The transformation matrix between the local coordinate system and the map coordinate system corresponding to the rod feature points shown is... And the transformation matrix between the local coordinate system and the map coordinate system corresponding to the feature points. The calculation methods are the same, and both can be based on the aforementioned general rotation relation matrix. and transformation matrix The calculation is performed using the formula.

[0252] S502, for each target localization feature point cloud, based on the preset correspondence between local localization confidence and feature point cloud type, determine the target local localization confidence corresponding to the target localization feature point cloud;

[0253] The constraints on localization of feature points in different types of feature point clouds are different, and these constraints are difficult to describe intuitively in the map coordinate system, but are relatively simple to describe in the local coordinate system. Therefore, for each type of feature point cloud, the local positioning confidence level used to characterize the constraints on localization of feature points in the local coordinate system can be determined in advance, and then the correspondence between the local positioning confidence level and the feature point cloud type can be stored in the electronic device.

[0254] In this way, the electronic device can determine the target local positioning confidence level corresponding to the feature point cloud type of each target positioning feature point cloud included in each frame of positioning image, based on the above correspondence.

[0255] For feature points in the feature point cloud of a rod-shaped object, from the perspective of the local coordinate system, it can provide constraints in four degrees of freedom: x, y, roll angle, and pitch angle. Therefore, the positional confidence of the feature points in the feature point cloud of the rod-shaped object in its local coordinate system can be set as follows:

[0256]

[0257] Where, r p s p u p and v p All are constant values ​​that are set, and satisfy r p <<s p and u p <<v p .

[0258] For feature points in a planar object feature point cloud, from the perspective of the local coordinate system, it can provide constraints of three degrees of freedom: z, roll angle, and pitch angle. Therefore, the positional confidence of feature points in the planar object feature point cloud in its local coordinate system can be set as follows:

[0259]

[0260] Where, r F s F u F and v F All are constant values ​​that are set, and satisfy r F >>s F and u F <<v F .

[0261] S503, based on the transformation matrix and the target local positioning confidence, determine the global positioning confidence of the target positioning feature point cloud in the coordinate system of the berthing map;

[0262] Since the purpose of localization is to determine the vehicle's precise position and attitude on the map, the representation of localization reliability also needs to be relative to the map. Therefore, it is necessary to transform the local localization reliability expressed in the local coordinate system to the coordinate system of the parking map to obtain the global localization reliability of the target localization feature point cloud in the coordinate system of the parking map. Specifically:

[0263] When determining the global positioning confidence of feature points in the target positioning feature point cloud within the coordinate system of the navigation map based on the transformation matrix and the target's local positioning confidence, electronic devices can first calculate the first-order approximate Jacobian matrix used for error propagation based on the aforementioned transformation matrix, according to the following Jacobian matrix calculation formula.

[0264]

[0265] Among them, J r Denote the left and right Jacobian functions on Lie algebras. Transformation matrix The Lie algebra representation of Ad, where Ad denotes the computation of the adjoint matrix function;

[0266] The electronic device obtains a first-order approximate Jacobian matrix for error propagation. Then, based on this first-order approximate Jacobian matrix, Based on the target's local location reliability, the global location reliability Ω of the feature points in the target's localization feature point cloud in the coordinate system of the navigation map is calculated according to the following global location reliability calculation formula. M :

[0267]

[0268] Among them, Ω L The matrix representation of the local location confidence of the target.

[0269] S504, the global positioning confidence corresponding to each target positioning feature point cloud is fused to obtain the target positioning confidence corresponding to the positioning image of that frame.

[0270] Since the transformation matrix is ​​applied to each feature point, the global positioning confidence of the target positioning feature point cloud obtained by the electronic device actually includes the global positioning confidence of each feature point in the target positioning feature point cloud. Therefore, when the electronic device fuses the global positioning confidence of each target positioning feature point cloud, it can fuse the global positioning confidence of each feature point.

[0271] In one implementation, the electronic device can calculate the target positioning confidence Ω corresponding to the localization image of the frame based on the global positioning confidence corresponding to each feature point, using the following fusion formula. all :

[0272]

[0273] in, This represents the global localization confidence level corresponding to the i-th feature point in the localization image of this frame.

[0274] In the solution provided in this application embodiment, the electronic device prevents the situation where positioning errors occur without the user's knowledge by evaluating the current positioning confidence. Furthermore, based on the constraints of the features themselves on positioning, a more accurate positioning confidence can be obtained through transformation and fusion, thereby improving the safety of navigation and berthing.

[0275] As one implementation method of this application, such as Figure 7 As shown, after obtaining the target location confidence score corresponding to the frame location image, the method further includes:

[0276] S701, based on the target positioning confidence corresponding to each frame of positioning image, determine whether the positioning pose corresponding to each frame of positioning image is accurate;

[0277] S702, For a positioning image with accurate positioning pose, extract the visual feature elements of the positioning image frame as target elements;

[0278] During vehicle operation, in addition to real-time vehicle positioning, electronic devices can also update maps to ensure map timeliness, prevent errors in real-time positioning or decision-making due to scene changes, and solve mapping occlusion problems. They can also supplement the situation where some elements are not observed or are not observed sufficiently due to dynamic obstacles during the teaching mapping process.

[0279] However, the prerequisite for updating the map is that the current positioning pose is accurate. Only in this way can the map be updated based on the current positioning result to obtain a more accurate map. Otherwise, the map will have greater errors after the update.

[0280] Therefore, after obtaining the target positioning confidence level corresponding to each frame of positioning image, the electronic device can first determine whether the positioning pose corresponding to each frame of positioning image is accurate based on the target positioning confidence level. If it is accurate, it indicates that the visual feature elements in the positioning image of that frame can be used as elements for updating the mooring map. Therefore, after determining the positioning image with accurate positioning pose based on the target positioning confidence level corresponding to each frame of positioning image, the electronic device can extract the visual feature elements of the positioning image of that frame as target elements.

[0281] In one implementation, when the electronic device determines whether the positioning pose corresponding to each positioning image is accurate based on the target positioning confidence corresponding to each positioning image, it can determine whether each diagonal element in the matrix expression corresponding to the target positioning confidence is less than a certain threshold. If all elements are less than a certain threshold, the positioning pose is determined to be accurate.

[0282] S703, determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map;

[0283] S704, if there is a discrepancy, update the mooring map based on the target location confidence and the target element.

[0284] After the electronic device extracts the target element used to update the mooring map, it can further determine whether there is a difference between the target element and the map element in the mooring map that is at the same location as the target element. That is, it can determine whether the map element at that location in the mooring map needs to be updated. If there is a difference, it means that the acquisition result of the map element at that location in the mooring map during the teaching mapping process is different from the acquisition result during the real-time positioning process. At this time, the mooring map can be updated based on the target positioning confidence and the target element.

[0285] In the solution provided in this application embodiment, when the electronic device performs real-time vehicle positioning, it can also perform map update operations. Based on the visual feature elements in the positioning image with accurate positioning pose, the map elements at the corresponding locations are updated, thereby ensuring the timeliness of the map, preventing errors in real-time positioning or decision planning due to scene changes, and solving the problem of map occlusion. It also supplements the situation where some elements are not observed or are not observed sufficiently due to dynamic obstacles during the teaching and mapping process.

[0286] As one implementation method of this application, such as Figure 8 As shown, updating the mooring map based on the target location confidence and the target elements can include:

[0287] S801, Based on the feature point cloud corresponding to the target element, determine the centroid coordinates corresponding to the target element;

[0288] After identifying target elements with accurate positioning poses and differences from corresponding map elements in the navigation map, in order to further improve the accuracy of the elements used for updating, the electronic device can fuse the same target elements in each frame of positioning images based on the target positioning confidence, and then use the fused target elements to update the map elements.

[0289] During the fusion process of target elements, the electronic device can first determine the centroid coordinates corresponding to the target element. In one implementation, the electronic device can consider the target element as an object with uniform density, and then discretize the target element into a feature point cloud, determine the average coordinate of each feature point in the feature point cloud, and use this average coordinate to determine the centroid coordinates corresponding to the target element.

[0290] S802, perform Lie algebra vector transformation on the target location confidence to obtain the confidence rotation and translation matrix;

[0291] The electronic device can extract the diagonal elements of the matrix representation corresponding to the target positioning confidence and take the square root of each diagonal element to obtain the corresponding positioning error Lie algebra. Then, the electronic device can convert this Lie algebra vector into a Lie group matrix to obtain the confidence rotation and translation matrix.

[0292] S803, the observation error corresponding to the target element is calculated based on the centroid coordinates and the confidence level rotation and translation matrix;

[0293] After obtaining the centroid coordinates and the confidence level rotation / translation matrix, the electronic device can calculate the observation error e corresponding to the target element based on the centroid coordinates and the confidence level rotation / translation matrix, according to the following observation error calculation formula. m :

[0294] e m =||T err P||

[0295] Among them, T err Let P be the confidence level rotation and translation matrix, and P be the centroid coordinates.

[0296] S804, Based on the observation error and the detection probability corresponding to the target element, the fusion weight corresponding to the target element is calculated;

[0297] Electronic devices can determine the degree of similarity between a target element and various preset element types when extracting a target element from a positioning image, and then use the highest degree of similarity as the detection probability of the target element.

[0298] Furthermore, the electronic device can calculate the fusion weight ω corresponding to the target element based on the observation error and detection probability of the target element, according to the following fusion weight calculation formula. m :

[0299] ω m =a m / e m

[0300] Among them, a mThis represents the detection probability of the target element.

[0301] S805, For each target element, based on the fusion weight corresponding to the target element in each frame of the target localization image, the target element in each frame of the target localization image is fused to obtain the fused target element;

[0302] After obtaining the fusion weights corresponding to each target element, the electronic device can fuse the same target elements in each frame of the target localization image to obtain the fused target elements. The target localization image includes the current frame of localization image and a preset number of previous frames of localization images. The preset number can be a pre-determined number, or it can be determined in real-time based on the elements included in each localization image, identifying the number of localization images containing the target elements preceding the current frame. No specific limitation is made here.

[0303] In one implementation, the electronic device can construct a multi-frame element observation error objective function and use the aforementioned fusion weights to weight each observation error, ultimately obtaining the accurate fused target element through an optimization algorithm.

[0304] In another implementation, the electronic device can directly perform weighted fusion of each target element based on the fusion weight corresponding to the same target element in each frame of the target localization image, thereby obtaining the fused target element.

[0305] S806, Update the mooring map based on the fused target elements.

[0306] After obtaining the fused target element, the electronic device can add the target element to the berthing map. If there is already a map element at the location where the target element is to be added, the electronic device can delete the existing map element and add the target element to that location.

[0307] In addition, during the map element update step, the observation status of existing map elements in the navigation map can be recorded. If there are map elements that have not been observed for a long time, they can be deleted.

[0308] In the solution provided in this application, after the electronic device determines that the target element has an accurate positioning pose and is indeed different from the corresponding map element in the map, it can fuse the same target element in each frame of positioning image based on the target positioning confidence, and then use the fused target element to update the map element, thereby improving the accuracy of map update.

[0309] As one implementation method of this application, such as Figure 9As shown, updating the mooring map based on the fused target elements can include:

[0310] S901, For each fused target element, calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element;

[0311] After completing multi-frame weighted fusion, in order to further improve the accuracy of the update, the electronic device can make a further judgment on whether the fused target element needs to be updated in the map.

[0312] The electronic device can first calculate the difference between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element for each fused target element.

[0313] For example, if the fused target element A is obtained by fusing target element a1, target element a2 and target element a3, then the electronic device can calculate the difference between the centroid coordinates of the fused target element A and the centroid coordinates of target element a1, the difference between the centroid coordinates of the fused target element A and the centroid coordinates of target element a2, and the difference between the centroid coordinates of the fused target element A and the centroid coordinates of target element a3, respectively.

[0314] S902, search for map elements of the same type as the fused target element within a preset range in the berthing map, and calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element;

[0315] The electronic device can also search for map elements of the same type as the merged target element within a preset range in the navigation map, and calculate the distance between the centroid coordinates of the merged target element and the centroid coordinates of the searched map element.

[0316] In one implementation, when there are multiple map elements of the same type as the fused target element within a preset range in the navigation map, the electronic device can determine the distance between the centroid coordinates of each map element and the centroid coordinates of the fused target element, and then take the minimum distance among the above distances as the distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element.

[0317] S903, based on the second distance and the fusion error, determine the update probability corresponding to the fused target element;

[0318] After determining the first distance between each target element and the centroid coordinates of the fused target element, the electronic device can average the first distances and use the average value as the fusion error. Then, based on the fusion error and the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element, the update probability corresponding to the fused target element can be determined.

[0319] In one implementation, the electronic device can determine the update probability p2 based on the aforementioned distance and fusion error, according to the following update probability calculation formula:

[0320]

[0321] Where α is a preset coefficient, n represents the total number of frames included in the target positioning image, d represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element, and e represents the fusion error.

[0322] S904, if the update probability is greater than the second preset threshold, the fused target element is used to update the parking map.

[0323] If the update probability is greater than the second preset threshold, it means that the fused target element needs to be updated in the map. At this time, the electronic device can use the fused target element to update the navigation map.

[0324] In the solution provided in this application embodiment, after the electronic device completes the weighted fusion of multiple frames, in order to further improve the accuracy of the update, it can further determine whether the fused target element needs to be updated in the map. On the one hand, when the error is small, the probability of map update can be reduced and the workload can be reduced. On the other hand, it can also accurately determine whether the map needs to be updated.

[0325] The following is based on Figures 10-11 For example, let's introduce a method for constructing a vehicle parking map provided in an embodiment of this application:

[0326] To implement the vehicle parking map construction method provided in this application embodiment, the vehicle can be equipped with, for example... Figure 10 The sensors shown mainly include: four surround-view cameras, one forward-looking camera, four millimeter-wave radars, one inertial navigation system consisting of an IMU (Inertial Measurement Unit) / RTK / wheel speed measurement unit, and one lidar. The types and quantities of each camera, radar, and inertial navigation system can be configured according to actual usage requirements and are not specifically limited here. Figure 10This is just an example configuration, and LiDAR is optional, meaning it can be omitted. However, the introduction of LiDAR can improve the robustness of vehicle parking map construction.

[0327] The vehicle parking map provided in this application mainly includes two major steps: mapping and positioning. Its data processing flow is as follows: Figure 11 As shown. The inertial navigation system, composed of surround-view cameras, forward-looking cameras, millimeter-wave radar, and IMU / RTK / wheel speed sensors, collects various sensor data. After being input into the electronic equipment, the electronic equipment can process this data systematically. Specifically, it first performs time synchronization and invalid data filtering through a data preprocessing module. Then, based on the user-defined modeling or positioning mode, it performs teaching mapping or real-time positioning. In mapping mode, the electronic equipment can comprehensively utilize the information from various sensors to extract and generate vector information of necessary elements in the scene, ultimately completing map construction and outputting a vector map. In positioning mode, the electronic equipment can combine the constructed vector map with the input sensor data to calculate the current positioning pose and confidence level, outputting the positioning result. Simultaneously, in positioning mode, the electronic equipment can also activate an update module, detecting and updating changed elements in the map based on the current pose and observation data, updating the map, and outputting the updated map to ensure its timeliness.

[0328] The following is based on Figures 12-14 Taking the above as an example, we will introduce the data processing flow of electronic devices in mapping mode, positioning mode and update mode respectively.

[0329] In mapping mode, electronic devices can enable heuristic mapping functionality. The algorithm processing flow mainly includes two steps: real-time processing and backend optimization. Figure 12 As shown, sensor data can be transmitted to electronic devices. When the user issues a teaching mapping start command, the electronic devices initiate the real-time processing flow of teaching mapping, which mainly includes four modules: odometer calculation, single-frame element extraction, driving and parking boundary detection, and multi-frame element fusion. Among them, odometer calculation is based on the inertial navigation information recorded by the inertial navigation device, and obtains the relative vehicle pose between the acquired mapping images through matching or recursion. Single-frame element extraction is to obtain visual feature elements in the mapping images. Driving and parking boundary detection is to determine the driving area and parking area in the initial map. Multi-frame element fusion is to weight and optimize the same visual feature elements in multiple mapping images according to the distance of observation to obtain optimized visual feature elements.

[0330] Since the odometer pose is obtained through recursive calculation, it will gradually diverge as the vehicle moves, especially when the vehicle passes the same location multiple times. This will result in discrepancies in the calculated positions of the same element in the scene each time. Therefore, when the user issues a teaching mapping end command or triggers a mapping end event, such as when the vehicle parks in a parking space, the real-time processing ends. At this point, the electronic device can initiate a backend optimization process to correct these discrepancies. This process mainly includes three modules: loop closure detection and pose optimization, element pose adjustment, and parking area optimization. After the system completes backend optimization, a high-precision vector map can be output, and the teaching mapping ends.

[0331] Loop closure detection and pose optimization refer to using RTK, vision, or LiDAR data to perform loop closure detection, thereby constructing a corresponding pose map and employing graph optimization algorithms to eliminate accumulated odometer errors, thus improving the overall consistency of the map pose. Afterward, the position and orientation of each element need to be adjusted according to the optimized map pose, and duplicate elements are fused using vector fusion, i.e., element pose adjustment. Driving and parking area optimization refers to optimizing the map elements included in the driving and parking areas based on the vehicle's driving and parking characteristics, respectively, after determining the initial driving and parking areas in the map.

[0332] In positioning mode, electronic devices can activate the positioning function, match the current observation with the established map, and output the vehicle's position and attitude on the map. The algorithm processing flow mainly includes two processes: initial positioning and continuous positioning. Initial positioning refers to positioning when the vehicle enters the map without knowing its position and attitude, and the positioning accuracy is required to reach the meter level. Continuous positioning, on the other hand, is performed when the approximate position and attitude of the vehicle are known, and the positioning accuracy is required to reach the centimeter level.

[0333] The algorithm flow for continuous localization is as follows: Figure 13 As shown, the electronic device can first acquire sensor data and map data. Then, based on odometry calculations and initial positioning results, it can perform a predicted pose calculation step. Following this, it can extract and transform map data based on the predicted pose; that is, based on the predicted pose, it selects a subset of map elements corresponding to the predicted pose from the various map elements included in the berthing map, and discretizes these map elements into a feature point cloud. Alternatively, it can extract single-frame features from the sensor data, and then fuse the results of each single-frame feature extraction based on odometry calculations, i.e., it performs a multi-frame feature fusion step.

[0334] The electronic device can then perform feature matching between the fused features and the feature point cloud discretized from map elements to determine the localization pose and output a confidence estimate. Based on this confidence estimate, the localization confidence can be determined.

[0335] In update mode, electronic devices can enable map update functionality. The algorithm processing flow mainly includes five modules: single-frame element extraction, initial data screening, multi-frame weighted fusion, update probability calculation, and map element update. Figure 14 As shown, the electronic device can acquire sensor data, map data, positioning pose, and positioning confidence. Then, the electronic device can extract elements from a single frame, perform initial data screening on the extracted elements, determine the target elements whose positioning pose is accurate and which do indeed differ from the corresponding map elements in the navigation map, and then perform weighted fusion of multiple frames, that is, merge the same target elements in each frame of the target positioning image to obtain the fused target elements. Then, the update probability is calculated, and when the update probability is greater than a preset threshold, the map elements are updated to obtain the updated map.

[0336] The solution provided in this application proposes an integrated mapping and positioning scheme applicable to all scenarios. The user drives the vehicle for instruction, and the system automatically constructs a vector map. After instruction, high-precision real-time positioning is achieved. During this process, map updates are simultaneously initiated to ensure consistency between the map and the real-world scene. Specifically, a multi-sensor-based driving and parking area detection algorithm addresses the issue of differences in driving and parking scenarios by distinguishing between driving and parking areas to achieve differentiated mapping for different regions. A feature-based location reliability estimation algorithm improves the security of the positioning system by evaluating the current location reliability to prevent unknowingly positioning errors. A confidence-weighted map update algorithm addresses the issue of scene changes by fusing location reliability to improve map update accuracy.

[0337] Corresponding to the above-described method for constructing a vehicle parking map, this application also provides a device for constructing a vehicle parking map. The following describes the device for constructing a vehicle parking map provided by this application.

[0338] like Figure 15 As shown, a vehicle parking map construction device includes:

[0339] The mapping data acquisition module 1510 is used to acquire mapping images of the surrounding environment during the vehicle teaching mapping process and acquire mapping data corresponding to each frame of mapping image, wherein the mapping data includes visual feature elements and vehicle pose.

[0340] The initial map construction module 1520 is used to perform teaching map construction based on the map construction data to obtain an initial map;

[0341] The boundary location determination module 1530 is used to determine whether the vehicle is located at the parking boundary location for each frame of the mapping image based on the mapping data corresponding to the target mapping image. The target mapping image includes the current frame of the mapping image and a preset number of previous frames of the mapping image.

[0342] The driving and parking area determination module 1540 is used to determine the driving area and parking area in the initial map based on the speed of the vehicle before it is at the driving and parking boundary position and the speed of the vehicle after it is at the driving and parking boundary position when the judgment result of the boundary position determination module is yes.

[0343] The map element optimization module 1550 is used to optimize the map elements included in the driving area and the parking area based on the driving characteristics of the vehicle when driving and the driving characteristics of the vehicle when parking, respectively, to obtain the driving and parking map of the vehicle.

[0344] In the solution provided in this application embodiment, the electronic device can acquire mapping images of the surrounding environment during vehicle teaching mapping and obtain mapping data corresponding to each frame of mapping image. The mapping data includes visual feature elements and vehicle pose. Based on the mapping data, teaching mapping is performed to obtain an initial map. For each frame of mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the boundary between driving and parking. The target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images. If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after the boundary between driving and parking. Based on the vehicle's driving characteristics and parking characteristics, the map elements included in the driving area and parking area are optimized respectively to obtain the vehicle's driving and parking map. Because electronic devices can first construct an initial map for both driving and parking areas using the same method, and then determine the boundary between driving and parking areas based on this initial map, thus distinguishing between them, and then optimize the map elements included in the driving and parking areas separately according to the different driving characteristics of vehicles during driving and parking. Therefore, the constructed map can not only include both driving and parking areas simultaneously, achieving integrated construction of driving and parking maps, but also optimize the map elements according to the different driving characteristics during driving and parking, further improving the accuracy of the map.

[0345] As one embodiment of this application, the mapping data may further include satellite solution data;

[0346] The boundary location determination module 1530 may include:

[0347] The existence probability determination unit is used to determine the existence probability of a preset category element in each frame of the target mapping image based on the visual feature elements corresponding to each frame of the target mapping image, wherein the preset category element is the feature element of the parking area entrance;

[0348] The vehicle slope information determination unit is used to determine the vehicle slope information corresponding to the frame of the mapping image based on the vehicle pose corresponding to the target mapping image.

[0349] The satellite solution result determination unit is used to determine the satellite solution result corresponding to the frame of the map image based on the number of satellite solutions corresponding to the target map image;

[0350] The boundary location determination unit is used to determine whether the vehicle is located at the boundary of the parking area based on the existence probability, the vehicle slope information, and the satellite calculation results.

[0351] As one embodiment of this application, the boundary position determination unit may include:

[0352] The boundary probability determination subunit is used to determine the probability that the vehicle is located at the boundary between parking and berthing based on the existence probability, the vehicle slope information and the satellite calculation results, as the parking boundary probability;

[0353] The boundary probability judgment subunit is used to determine whether the boundary probability of the berthing is greater than a first preset threshold.

[0354] The boundary location determination subunit is used to determine that the vehicle is located at the boundary location when the probability of the parking boundary is greater than the first preset threshold.

[0355] As one embodiment of this application, the boundary probability determination subunit can be specifically used to calculate the probability p1 that the vehicle is located at the boundary of parking and berthing according to the following parking boundary probability calculation formula based on the existence probability, the vehicle slope information, and the satellite calculation results:

[0356]

[0357] in, This represents the probability of the existence of a preset category element j in the i-th frame of the target mapping image. and σ θ Let represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image, respectively; Δn represents the difference between the number of satellite solutions corresponding to that frame and the number of satellite solutions corresponding to the first frame of the target mapping image; α v α r and α gThese are the visual probability coefficient, the slope probability coefficient, and the satellite probability coefficient, respectively.

[0358] As one embodiment of this application, the apparatus may further include:

[0359] The positioning feature point cloud acquisition module is used to collect positioning images of the surrounding environment during vehicle operation and acquire the positioning feature point cloud corresponding to each frame of positioning image. The positioning feature point cloud includes rod-shaped object feature point cloud and area-shaped object feature point cloud.

[0360] The matching module is used to match the positioning feature point cloud with map elements in the mooring map to obtain the matching result;

[0361] The positioning module is used to determine the positioning pose of the vehicle based on the matching results.

[0362] As one embodiment of this application, the apparatus may further include:

[0363] The transformation matrix determination module is used to construct a local coordinate system corresponding to each target positioning feature point cloud for each target positioning feature point cloud included in each frame positioning image, and determine the transformation matrix between the local coordinate system and the coordinate system of the berthing map, wherein the target positioning feature point cloud is a positioning feature point cloud that successfully matches the map elements in the berthing map;

[0364] The target local positioning confidence determination module is used to determine the target local positioning confidence corresponding to each target positioning feature point cloud based on a preset correspondence between local positioning confidence and feature point cloud type; wherein, the local positioning confidence represents the constraint of the corresponding feature point cloud on positioning in the local coordinate system.

[0365] The global positioning confidence determination module is used to determine the global positioning confidence of the target positioning feature point cloud in the coordinate system of the mooring map based on the transformation matrix and the target local positioning confidence;

[0366] The target location confidence determination module is used to fuse the global location confidence corresponding to each target location feature point cloud to obtain the target location confidence corresponding to the frame location image.

[0367] As one embodiment of this application, the transformation matrix determination module can be specifically used to calculate the rotation transformation matrix between the local coordinate system and the coordinate system of the mooring map based on the coordinate axis direction vector of the local coordinate system and according to the following rotation relationship matrix calculation formula.

[0368]

[0369] Where, n x n represents the direction vector of the x-axis of the local coordinate system. y n represents the direction vector of the y-axis of the local coordinate system. z This represents the direction vector of the z-axis of the local coordinate system;

[0370] Based on the rotation transformation matrix Using the coordinates of feature points in the target location feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the mooring map according to the following transformation matrix calculation formula.

[0371]

[0372] Where t represents the coordinates of the feature point in the target localization feature point cloud.

[0373] As one embodiment of this application, the global location confidence determination module may include:

[0374] The first-order approximate Jacobian matrix calculation unit is used to calculate, based on the transformation matrix and according to the following Jacobian matrix calculation formula, the first-order approximate Jacobian matrix used for error propagation.

[0375]

[0376] Among them, J r Denote the left and right Jacobian functions on Lie algebras. Transformation matrix The Lie algebra representation of Ad, where Ad denotes the computation of the adjoint matrix function;

[0377] The global location confidence calculation unit is used to calculate the confidence level based on the first-order approximate Jacobian matrix. Based on the target's local location confidence, the global location confidence Ω of the target's location feature point cloud in the coordinate system of the mooring map is calculated according to the following global location confidence calculation formula. M :

[0378]

[0379] Among them, Ω L The matrix representation of the local location confidence of the target.

[0380] As one embodiment of this application, the apparatus may further include:

[0381] The localization pose determination module is used to determine whether the localization pose corresponding to each frame of localization image is accurate based on the target localization confidence level corresponding to each frame of localization image.

[0382] The target element determination module is used to extract the visual feature elements of the positioning image frame with accurate positioning pose, and use them as target elements.

[0383] The difference judgment module is used to determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map;

[0384] The map element update module is used to update the mooring map based on the target location confidence and the target element when the difference judgment module determines that the result is yes.

[0385] As one embodiment of this application, the map element update module may include:

[0386] The centroid coordinate determination unit is used to determine the centroid coordinates of the target element based on the feature point cloud corresponding to the target element.

[0387] The confidence level rotation and translation matrix determination unit is used to perform Lie algebra vector transformation on the target location confidence to obtain the confidence level rotation and translation matrix;

[0388] The observation error calculation unit is used to calculate the observation error corresponding to the target element based on the centroid coordinates and the confidence rotation and translation matrix.

[0389] The fusion weight calculation unit is used to calculate the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element.

[0390] The target element fusion unit is used to fuse the target element in each frame of the target positioning image based on the fusion weight corresponding to the target element in each frame of the target positioning image, so as to obtain the fused target element. The target positioning image includes the positioning image of the frame and a preset number of positioning images before the positioning image of the frame.

[0391] The map element update unit is used to update the mooring map based on the fused target elements.

[0392] As one embodiment of this application, the observation error calculation unit can be specifically used to calculate the observation error e corresponding to the target element based on the centroid coordinates and the confidence rotation and translation matrix, according to the following observation error calculation formula. m :

[0393] em =||T err P||

[0394] Among them, T err Let P be the confidence level rotation and translation matrix, and let P be the centroid coordinates.

[0395] The fusion weight calculation unit is specifically used to calculate the fusion weight ω corresponding to the target element based on the observation error and the detection probability corresponding to the target element, according to the following fusion weight calculation formula. m :

[0396] ω m =a m / e m

[0397] Among them, a m The detection probability is the probability corresponding to the target element.

[0398] As one embodiment of this application, the map element updating unit may include:

[0399] The centroid coordinate difference calculation subunit is used to calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element for each fused target element.

[0400] The centroid coordinate distance calculation subunit is used to search for map elements of the same type as the fused target element within a preset range in the berthing map, and to calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element.

[0401] The update probability determination subunit is used to determine the update probability corresponding to the fused target element based on the second distance and the fusion error, wherein the fusion error is the mean of the first distance corresponding to each target element;

[0402] The map element update subunit is used to update the mooring map using the fused target elements when the update probability is greater than a second preset threshold.

[0403] As one embodiment of this application, the update probability determination subunit can be specifically used to determine the update probability p2 based on the distance and the fusion error amount, according to the following update probability calculation formula:

[0404]

[0405] Where α is a preset coefficient, n represents the total number of frames included in the target positioning image, d represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element, and e represents the fusion error.

[0406] This application also provides an electronic device, such as... Figure 16 As shown, it includes:

[0407] Memory 1601 is used to store computer programs;

[0408] The processor 1602, when executing the program stored in the memory 1601, implements the vehicle parking map construction method described in any of the above embodiments.

[0409] In the solution provided in this application embodiment, the electronic device can acquire mapping images of the surrounding environment during vehicle teaching mapping and obtain mapping data corresponding to each frame of mapping image. The mapping data includes visual feature elements and vehicle pose. Based on the mapping data, teaching mapping is performed to obtain an initial map. For each frame of mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the boundary between driving and parking. The target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images. If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after the boundary between driving and parking. Based on the vehicle's driving characteristics and parking characteristics, the map elements included in the driving area and parking area are optimized respectively to obtain the vehicle's driving and parking map. Because electronic devices can first construct an initial map for both driving and parking areas using the same method, and then determine the boundary between driving and parking areas based on this initial map, thus distinguishing between them, and then optimize the map elements included in the driving and parking areas separately according to the different driving characteristics of vehicles during driving and parking. Therefore, the constructed map can not only include both driving and parking areas simultaneously, achieving integrated construction of driving and parking maps, but also optimize the map elements according to the different driving characteristics during driving and parking, further improving the accuracy of the map.

[0410] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1602, the communication interface, and the memory 1601 communicating with each other via the communication bus.

[0411] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0412] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0413] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0414] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0415] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the vehicle parking map construction method described in any of the above embodiments.

[0416] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the vehicle parking map construction method described in any of the above embodiments.

[0417] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0418] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0419] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0420] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for constructing a vehicle parking map, characterized in that, The method includes: During the vehicle teaching mapping process, mapping images of the surrounding environment are collected, and mapping data corresponding to each frame of the mapping image is obtained. The mapping data includes visual feature elements and vehicle pose, or visual feature elements, vehicle pose, and satellite solution data. Based on the mapping data, a teaching mapping is performed to obtain an initial map; For each frame of the mapping image, based on the mapping data corresponding to the target mapping image, it is determined whether the vehicle is located at the intersection of driving and parking, wherein the target mapping image includes the current frame of mapping image and a preset number of previous frames of mapping images; If the vehicle is located at the boundary between driving and parking, the driving area and parking area in the initial map are determined based on the vehicle's speed before and after it is located at the boundary between driving and parking. Based on the driving characteristics of the vehicle during driving and the driving characteristics of the vehicle during parking, the map elements included in the driving area and the parking area are optimized respectively to obtain the driving and parking map of the vehicle. The step of determining whether the vehicle is located at the intersection of parking and driving zones based on the mapping data corresponding to the target mapping image includes: Perform at least two of the following steps: determine the existence probability of a preset category element in each frame of the target mapping image based on the visual feature elements corresponding to each frame of the target mapping image, wherein the preset category element is the feature element possessed by the parking area entrance; determine the vehicle slope information corresponding to the frame of the mapping image based on the vehicle pose corresponding to the target mapping image; and determine the satellite solution result corresponding to the frame of the mapping image based on the number of satellite solutions corresponding to the target mapping image. Based on at least two of the existence probability, the vehicle slope information, and the satellite calculation results, it is determined whether the vehicle is located at the intersection of parking and driving.

2. The method as described in claim 1, characterized in that, Based on the existence probability, the vehicle slope information, and the satellite calculation results, determining whether the vehicle is located at the intersection of parking and driving areas includes: Based on the existence probability, the vehicle slope information, and the satellite calculation results, the probability that the vehicle is located at the boundary between parking and berthing is determined as the boundary probability between parking and berthing. Determine whether the probability of the berth boundary is greater than a first preset threshold; If the probability of the parking boundary is greater than the first preset threshold, the vehicle is determined to be located at the parking boundary.

3. The method as described in claim 2, characterized in that, The determination of the probability that the vehicle is located at the intersection of parking and driving zones based on the existence probability, the vehicle slope information, and the satellite calculation results includes: Based on the existence probability, the vehicle slope information, and the satellite calculation results, the probability that the vehicle is located at the intersection of parking and berthing is calculated according to the following formula for calculating the probability of parking and berthing intersection. : ； in, Indicates the first [item] in the target mapping image Preset category elements in frame mapping images The probability of its existence. and These represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image. This represents the difference between the number of satellite solutions corresponding to the current frame of the mapped image and the number of satellite solutions corresponding to the first frame of the target mapped image. , and These are the visual probability coefficient, the slope probability coefficient, and the satellite probability coefficient, respectively.

4. The method according to any one of claims 1-3, characterized in that, After obtaining the parking map of the vehicle, the method further includes: During vehicle operation, positioning images of the surrounding environment are collected, and positioning feature point clouds corresponding to each frame of positioning image are obtained. The positioning feature point clouds include rod-shaped feature point clouds and area-shaped feature point clouds. The location feature point cloud is matched with map elements in the berthing map to obtain the matching result; Based on the matching results, the positioning pose of the vehicle is determined.

5. The method as described in claim 4, characterized in that, After determining the vehicle's positioning pose based on the matching result, the method further includes: For each target positioning feature point cloud included in each frame of positioning image, a local coordinate system corresponding to each target positioning feature point cloud is constructed, and the transformation matrix between the local coordinate system and the coordinate system of the berthing map is determined, wherein the target positioning feature point cloud is the positioning feature point cloud that successfully matches the map element in the berthing map; For each target localization feature point cloud, based on the preset correspondence between local positioning confidence and feature point cloud type, the target local positioning confidence corresponding to the target localization feature point cloud is determined; wherein, the local positioning confidence represents the constraint of the corresponding feature point cloud on positioning in the local coordinate system; Based on the transformation matrix and the target local positioning confidence, the global positioning confidence of the target positioning feature point cloud in the coordinate system of the mooring map is determined; The global positioning confidence corresponding to each target localization feature point cloud is fused to obtain the target positioning confidence corresponding to the localization image of that frame.

6. The method as described in claim 5, characterized in that, Determining the transformation matrix between the local coordinate system and the coordinate system of the mooring map includes: Based on the coordinate axis direction vectors of the local coordinate system, the rotation transformation matrix between the local coordinate system and the coordinate system of the mooring map is calculated according to the following rotation relationship matrix calculation formula. : ； in, This represents the direction vector of the x-axis of the local coordinate system. This represents the direction vector of the y-axis of the local coordinate system. This represents the direction vector of the z-axis of the local coordinate system; Based on the rotation transformation matrix Using the coordinates of feature points in the target location feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the mooring map according to the following transformation matrix calculation formula. : ； Where t represents the coordinates of the feature point in the target localization feature point cloud.

7. The method as described in claim 5, characterized in that, The step of determining the global positioning confidence of the target's localization feature point cloud in the coordinate system of the mooring map based on the transformation matrix and the target's local positioning confidence includes: Based on the transformation matrix, calculate the first-order approximate Jacobian matrix used for error propagation according to the following Jacobian matrix calculation formula. : ； in, Denote the left and right Jacobian functions on Lie algebras. Transformation matrix Lie algebras indicate that This indicates the computation of the adjoint matrix function; Based on the first-order approximate Jacobian matrix Based on the target's local location confidence, the global location confidence of the target's location feature point cloud in the coordinate system of the berthing map is calculated according to the following global location confidence calculation formula. : ； in, The matrix representation of the local location confidence of the target.

8. The method as described in claim 5, characterized in that, After obtaining the target location confidence score corresponding to the frame location image, the method further includes: Based on the target positioning confidence corresponding to each frame of positioning image, determine whether the positioning pose corresponding to each frame of positioning image is accurate; For a positioning image with accurate pose, extract the visual feature elements of that positioning image frame as the target element; Determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map; If discrepancies exist, the mooring map is updated based on the target location confidence and the target elements.

9. The method as described in claim 8, characterized in that, The update of the mooring map based on the target location confidence and the target elements includes: Based on the feature point cloud corresponding to the target element, determine the centroid coordinates of the target element; The target location confidence is transformed by Lie algebra vector transformation to obtain the confidence rotation and translation matrix; The observation error corresponding to the target element is calculated based on the centroid coordinates and the confidence level rotation and translation matrix. Based on the observation error and the detection probability corresponding to the target element, the fusion weight corresponding to the target element is calculated. For each target element, based on the fusion weight corresponding to the target element in each frame of the target localization image, the target element in each frame of the target localization image is fused to obtain the fused target element. The target localization image includes the current frame localization image and a preset number of previous frames of localization images. The berthing map is updated based on the fused target elements.

10. The method as described in claim 9, characterized in that, The step of calculating the observation error corresponding to the target element based on the centroid coordinates and the confidence level rotation / translation matrix includes: Based on the centroid coordinates and the confidence level rotation / translation matrix, the observation error corresponding to the target element is calculated according to the following observation error calculation formula. : ； in, Let be the confidence level rotation and translation matrix. The coordinates of the centroid; The calculation of the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element includes: Based on the observation error and the detection probability corresponding to the target element, the fusion weight corresponding to the target element is calculated according to the following fusion weight calculation formula. : ； in, The detection probability is the probability corresponding to the target element.

11. The method as described in claim 9, characterized in that, The update of the mooring map based on the fused target elements includes: For each fused target element, calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element. Search for map elements of the same type as the fused target element within a preset range in the berthing map, and calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element; Based on the second distance and the fusion error, the update probability corresponding to the fused target element is determined, wherein the fusion error is the mean of the first distance corresponding to each target element; If the update probability is greater than the second preset threshold, the fused target elements are used to update the parking map.

12. The method as described in claim 11, characterized in that, The step of determining the update probability corresponding to the fused target element based on the distance and the fusion error includes: Based on the distance and fusion error, the update probability is determined according to the following update probability calculation formula. : ； in, The preset coefficients, This indicates the total number of frames included in the target location image. This represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element. This indicates the amount of fusion error.

13. A device for constructing a vehicle parking map, characterized in that, The device includes: The mapping data acquisition module is used to collect mapping images of the surrounding environment during the vehicle teaching mapping process and acquire mapping data corresponding to each frame of mapping image. The mapping data includes visual feature elements and vehicle pose, or visual feature elements, vehicle pose and satellite solution data. An initial map construction module is used to perform teaching map construction based on the map construction data to obtain an initial map; The boundary location determination module is used to determine whether the vehicle is located at the parking boundary location for each frame of the mapping image based on the mapping data corresponding to the target mapping image. The target mapping image includes the current frame of the mapping image and a preset number of previous frames of the mapping image. The driving and parking area determination module is used to determine the driving area and parking area in the initial map based on the speed of the vehicle before it is at the driving and parking boundary position and the speed of the vehicle after it is at the driving and parking boundary position when the judgment result of the boundary position determination module is yes. The map element optimization module is used to optimize the map elements included in the driving area and the parking area based on the driving characteristics of the vehicle when driving and the driving characteristics of the vehicle when parking, respectively, to obtain the driving and parking map of the vehicle. The boundary location determination module includes: The determining unit is configured to perform at least two of the following steps: determining the probability of the presence of a preset category element in each frame of the target mapping image based on the visual feature elements corresponding to each frame of the target mapping image, wherein the preset category element is a feature element possessed by the entrance of the parking area; determining the vehicle slope information corresponding to the frame of the mapping image based on the vehicle pose corresponding to the target mapping image; and determining the satellite solution result corresponding to the frame of the mapping image based on the number of satellite solutions corresponding to the target mapping image. The boundary location determination unit is used to determine whether the vehicle is located at the boundary of parking and driving based on at least two of the existence probability, the vehicle slope information, and the satellite calculation results.

14. The apparatus as claimed in claim 13, characterized in that, The boundary location determination unit includes: The boundary probability determination subunit is used to determine the probability that the vehicle is located at the boundary between parking and berthing based on the existence probability, the vehicle slope information and the satellite calculation results, as the parking boundary probability; The boundary probability judgment subunit is used to determine whether the boundary probability of the berthing is greater than a first preset threshold. The boundary position determination subunit is used to determine that the vehicle is located at the parking boundary position when the probability of the parking boundary is greater than the first preset threshold; or, The boundary probability determination subunit is specifically used to calculate the probability that the vehicle is located at the parking boundary based on the existence probability, the vehicle slope information, and the satellite calculation results, according to the following parking boundary probability calculation formula. : ； in, Indicates the first [item] in the target mapping image Preset category elements in frame mapping images The probability of its existence. and These represent the mean and variance of the slope at the vehicle's location in each frame of the target mapping image. This represents the difference between the number of satellite solutions corresponding to the current frame of the mapped image and the number of satellite solutions corresponding to the first frame of the target mapped image. , and These are the visual probability coefficient, the ramp probability coefficient, and the satellite probability coefficient, respectively; or, The device further includes: The positioning feature point cloud acquisition module is used to collect positioning images of the surrounding environment during vehicle operation and acquire the positioning feature point cloud corresponding to each frame of positioning image. The positioning feature point cloud includes rod-shaped object feature point cloud and area-shaped object feature point cloud. The matching module is used to match the positioning feature point cloud with map elements in the mooring map to obtain the matching result; A positioning module is used to determine the positioning pose of the vehicle based on the matching result; or, The device further includes: The transformation matrix determination module is used to construct a local coordinate system corresponding to each target positioning feature point cloud for each target positioning feature point cloud included in each frame positioning image, and determine the transformation matrix between the local coordinate system and the coordinate system of the berthing map, wherein the target positioning feature point cloud is a positioning feature point cloud that successfully matches the map elements in the berthing map; The target local positioning confidence determination module is used to determine the target local positioning confidence corresponding to each target positioning feature point cloud based on a preset correspondence between local positioning confidence and feature point cloud type; wherein, the local positioning confidence represents the constraint of the corresponding feature point cloud on positioning in the local coordinate system. The global positioning confidence determination module is used to determine the global positioning confidence of the target positioning feature point cloud in the coordinate system of the mooring map based on the transformation matrix and the target local positioning confidence; The target location confidence determination module is used to fuse the global location confidence corresponding to each target location feature point cloud to obtain the target location confidence corresponding to the frame of the location image; or, The transformation matrix determination module is specifically used to calculate the rotation transformation matrix between the local coordinate system and the coordinate system of the mooring map based on the coordinate axis direction vectors of the local coordinate system, according to the following rotation relationship matrix calculation formula. : ； in, This represents the direction vector of the x-axis of the local coordinate system. This represents the direction vector of the y-axis of the local coordinate system. This represents the direction vector of the z-axis of the local coordinate system; Based on the rotation transformation matrix Using the coordinates of feature points in the target location feature point cloud, calculate the transformation matrix between the local coordinate system and the coordinate system of the mooring map according to the following transformation matrix calculation formula. : ； Where t represents the coordinates of the feature point in the target localization feature point cloud; or, The global location confidence determination module includes: The first-order approximate Jacobian matrix calculation unit is used to calculate, based on the transformation matrix and according to the following Jacobian matrix calculation formula, the first-order approximate Jacobian matrix used for error propagation. : ； in, Denote the left and right Jacobian functions on Lie algebras. Transformation matrix Lie algebras indicate that This indicates the computation of the adjoint matrix function; The global location confidence calculation unit is used to calculate the confidence level based on the first-order approximate Jacobian matrix. Based on the target's local location confidence, the global location confidence of the target's location feature point cloud in the coordinate system of the berthing map is calculated according to the following global location confidence calculation formula. : ； in, The matrix representation of the target local location confidence; or, The device further includes: The localization pose determination module is used to determine whether the localization pose corresponding to each frame of localization image is accurate based on the target localization confidence level corresponding to each frame of localization image. The target element determination module is used to extract the visual feature elements of the positioning image frame with accurate positioning pose, and use them as target elements. The difference judgment module is used to determine whether there is a difference between the target element and a map element in the same position as the target element in the berthing map; A map element update module is used to update the mooring map based on the target location confidence and the target elements when the difference judgment module determines that the difference is true; or, The map element update module includes: The centroid coordinate determination unit is used to determine the centroid coordinates of the target element based on the feature point cloud corresponding to the target element. The confidence level rotation and translation matrix determination unit is used to perform Lie algebra vector transformation on the target location confidence to obtain the confidence level rotation and translation matrix; The observation error calculation unit is used to calculate the observation error corresponding to the target element based on the centroid coordinates and the confidence rotation and translation matrix. The fusion weight calculation unit is used to calculate the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element. The target element fusion unit is used to fuse the target element in each frame of the target positioning image based on the fusion weight corresponding to the target element in each frame of the target positioning image, so as to obtain the fused target element. The target positioning image includes the positioning image of the frame and a preset number of positioning images before the positioning image of the frame. A map element update unit is used to update the mooring map based on the fused target elements; or, The observation error calculation unit is specifically used to calculate the observation error corresponding to the target element based on the centroid coordinates and the confidence level rotation / translation matrix, according to the following observation error calculation formula. : ； in, Let be the confidence level rotation and translation matrix. The coordinates of the centroid; The fusion weight calculation unit is specifically used to calculate the fusion weight corresponding to the target element based on the observation error and the detection probability corresponding to the target element, according to the following fusion weight calculation formula. : ； in, The detection probability corresponding to the target element; or, The map element update unit includes: The centroid coordinate difference calculation subunit is used to calculate the first distance between the centroid coordinates of the fused target element and the centroid coordinates of each target element corresponding to the fused target element for each fused target element. The centroid coordinate distance calculation subunit is used to search for map elements of the same type as the fused target element within a preset range in the berthing map, and to calculate the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element. The update probability determination subunit is used to determine the update probability corresponding to the fused target element based on the second distance and the fusion error, wherein the fusion error is the mean of the first distance corresponding to each target element; A map element update subunit is used to update the mooring map using the fused target elements when the update probability is greater than a second preset threshold; or... The update probability determination subunit is specifically used to determine the update probability based on the distance and the fusion error, according to the following update probability calculation formula. : ； in, The preset coefficients, This indicates the total number of frames included in the target location image. This represents the second distance between the centroid coordinates of the fused target element and the centroid coordinates of the searched map element. This indicates the amount of fusion error.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-12.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-12.

Images