Parking navigation positioning method, device, electronic equipment and storage medium

Abstract

The present application provides a parking navigation positioning method, device, electronic device and storage medium, the method comprising: obtaining the current image frame in front of the vehicle and the previous image frame corresponding to the current image frame; based on the previous image frame, predicting the theoretical position of the vehicle corresponding to the current image frame, and determining the target map point cloud that matches the current image frame; reprojecting the target map point cloud and the current image frame to determine the reprojection error; determining the prediction error based on the reprojection error and the odometer error of the vehicle; determining the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error. In this embodiment, the theoretical position of the vehicle is verified to be reasonable by matching feature points to improve the accuracy of the positioning result; and the reprojection error and the odometer error are fused to comprehensively consider the situation that causes the positioning error, thereby improving the positioning accuracy of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of smart city technology, and more specifically, to a parking navigation positioning method, device, electronic device and storage medium. Background Art

[0002] As the number of vehicles increases, more and more traditional parking lots are being transformed into smart parking lots to accommodate more vehicles more efficiently. Smart parking lots can provide positioning and navigation services for vehicles, making it easier for vehicles to quickly and accurately drive to the target parking space.

[0003] In the related art, since most parking lots are indoor parking lots, the positioning signal will be interfered by buildings in indoor parking lots, making the positioning signal very poor. In order to enhance the positioning signal, the existing solution is to install multiple Bluetooth, wireless broadcasting devices or ultra-wideband (UWB) devices in the parking lot so that the positioning signal covers the entire parking lot. However, the vehicle positioning accuracy using Bluetooth or wireless broadcasting is poor, and the high-precision UWB equipment is expensive, making the parking lot renovation cost too high, which is not conducive to popularization in many parking lots. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a parking navigation positioning method, device, electronic device and storage medium, aiming to solve the problem of poor positioning accuracy in the current parking navigation process.

[0005] In a first aspect, an embodiment of the present application provides a parking navigation positioning method, comprising:

[0006] Obtain the current image frame in front of the vehicle and the previous image frame corresponding to the current image frame;

[0007] Based on the previous image frame, predict the theoretical position of the vehicle corresponding to the current image frame, and determine the target map point cloud that matches the current image frame;

[0008] Reproject the target map point cloud and the current image frame to determine the reprojection error;

[0009] Determining a prediction error based on the reprojection error and the vehicle's odometry error;

[0010] Based on the theoretical position of the vehicle and the prediction error, the actual position of the vehicle corresponding to the current image frame is determined.

[0011] In this embodiment, by acquiring the current image frame in front of the vehicle and the previous image frame corresponding to the current image frame, based on the previous image frame, the theoretical position of the vehicle corresponding to the current image frame is predicted, and the target map point cloud matching the current image frame is determined; and by reprojecting the target map point cloud and the current image frame, the reprojection error is determined, so as to verify whether the theoretical position of the vehicle is reasonable by means of feature point matching, thereby improving the accuracy of the positioning result; and based on the reprojection error and the odometer error of the vehicle, the prediction error is determined, so as to fuse the reprojection error and the odometer error to comprehensively consider the situation causing the positioning error, thereby further improving the accuracy of the positioning result; finally, based on the theoretical position of the vehicle and the prediction error, the actual position of the vehicle corresponding to the current image frame is determined, so as to verify whether the theoretical position of the vehicle is the actual position of the vehicle through the prediction error, thereby improving the positioning accuracy.

[0012] In one embodiment, predicting a theoretical vehicle position corresponding to a current image frame based on a previous image frame, and determining a target map point cloud matching the current image frame, includes:

[0013] Based on the actual position of the previous vehicle corresponding to the previous image frame, according to the preset uniform motion model, the theoretical position of the vehicle corresponding to the current image frame is predicted;

[0014] Based on the previous target map point cloud corresponding to the previous image frame, a local map point cloud of the parking lot where the vehicle is located is constructed;

[0015] The local map point cloud is projected onto the current image frame, and a target map point cloud matching the current image in the local map point cloud is determined.

[0016] In this embodiment, the previous image frame is combined with a uniform motion model to predict the theoretical position of the vehicle corresponding to the current image frame, and to determine the target map point cloud that matches the current image frame.

[0017] Further, based on the last target map point cloud corresponding to the last image frame, a local map point cloud of the parking lot where the vehicle is located is constructed, including:

[0018] Determine all key image frames that can observe the previous target map point cloud;

[0019] Determining a common-view key image frame of all key image frames;

[0020] A local map point cloud is constructed based on the first map point clouds of all key image frames and the second map point clouds of the common view key image frames.

[0021] In this embodiment, the common view key image frame is observed through the previous target map point cloud, so as to construct a local map point cloud of the vehicle's location based on the previous target map point cloud, so that the target map point cloud can be matched in a lightweight manner during vehicle positioning, thereby reducing the map point cloud loading amount and improving positioning efficiency.

[0022] Further, the local map point cloud is projected onto the current image frame, and a target map point cloud matching the current image in the local map point cloud is determined, including:

[0023] Projecting the first feature point in the local map point cloud to the current image frame to obtain a first projection point;

[0024] Extracting image feature points of the current image frame to obtain second feature points;

[0025] Matching the first projection point with the second feature point to obtain a matching value;

[0026] Based on the highest matching value, the target map point cloud in the local map point cloud that matches the current image frame is determined.

[0027] In this embodiment, the local map point cloud is matched with the current image frame by projection to achieve the determination of the target map point cloud.

[0028] Furthermore, the previous image frame is an initial image frame, the previous actual position of the vehicle is an initial position of the vehicle, and based on the previous actual position of the vehicle corresponding to the previous image frame, according to a preset uniform motion model, before predicting the theoretical position of the vehicle corresponding to the current image frame, the method further includes:

[0029] Obtain an initial image frame of the vehicle at the entrance of the parking lot;

[0030] Based on the image features of the initial image frame, the initial image frame is matched with a plurality of preset key image frames of the parking lot entrance to obtain a matching degree between the initial image frame and the plurality of preset key image frames;

[0031] Based on the preset n-point perspective algorithm, the initial image frame with the highest matching degree and the preset key image frame are fused and solved;

[0032] If the fusion solution is successful, the solution result of the fusion solution is the initial position of the vehicle.

[0033] In this embodiment, the vehicle is initially positioned at the entrance of the parking lot, and the positioning result is used as the positioning basis for subsequent visual tracking. The optimal positioning result is determined by the n-point perspective algorithm to improve the positioning accuracy.

[0034] In one embodiment, reprojecting the target map point cloud and the current image frame to determine the reprojection error includes:

[0035] Extract image feature points of the current image frame;

[0036] Projecting the third feature point in the target map point cloud to the current image frame to obtain the corresponding relationship between the image feature point and the third feature point;

[0037] Determine the three-dimensional feature point corresponding to the image feature point based on the corresponding relationship and the preset three-dimensional feature point corresponding to the third feature point;

[0038] Reprojecting the three-dimensional feature points to the current image frame to obtain a second projection point;

[0039] Based on a preset reprojection error function, a reprojection error between the second projection point and the image feature point is determined.

[0040] In this embodiment, the target map point cloud and the current image frame are reprojected to obtain the three-dimensional coordinates of the image feature points of the current image frame, and the error is determined based on the three-dimensional coordinates, thereby taking into account the three-dimensional positioning error of the vehicle and further improving the vehicle positioning accuracy.

[0041] In one embodiment, determining a prediction error based on a reprojection error and an odometer error of the vehicle includes:

[0042] According to the preset prediction error function, the reprojection error and the vehicle's odometer error are calculated to obtain the prediction error; the calculation formula of the prediction error function is:

[0043] E(T)=aE p (T)+bE o (T)

[0044] Where E(T) is the prediction error, a is the reprojection error factor, and E p (T) is the reprojection error, b is the odometer error factor, E o (T) is the odometer error.

[0045] In this embodiment, the error influence ratio is adjusted by the error factor so that the error function can be applicable to more visual tracking scenarios.

[0046] Further, after determining the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error, the method further includes:

[0047] If the image feature points of the current image frame fail to match the target map point cloud, the reprojection error factor is adjusted to a preset value;

[0048] After the reprojection error factor is adjusted, the prediction error is re-determined;

[0049] Based on the theoretical position of the vehicle and the re-determined prediction error, the actual position of the vehicle corresponding to the current image frame is determined.

[0050] In this embodiment, the reprojection error factor is adjusted to maintain the inter-frame relationship based on the odometer error when visual tracking fails.

[0051] In one embodiment, determining the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error includes:

[0052] Based on the preset gradient descent algorithm, the current image frame and the previous image frame are iterated until the prediction error is less than the preset value, and the theoretical position of the vehicle corresponding to the prediction error is the actual position of the vehicle.

[0053] In this embodiment, the actual position of the vehicle is determined in an iterative manner to further improve the positioning accuracy of the vehicle.

[0054] In a second aspect, an embodiment of the present application provides a parking navigation positioning device, comprising:

[0055] An acquisition module, used to acquire a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame;

[0056] A first determination module is used to predict the theoretical position of the vehicle corresponding to the current image frame based on the previous image frame, and to determine the target map point cloud matching the current image frame;

[0057] A reprojection module is used to reproject the target map point cloud and the current image frame and determine the reprojection error;

[0058] A second determination module is used to determine a prediction error based on the reprojection error and an odometer error of the vehicle;

[0059] The third determination module is used to determine the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error.

[0060] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the parking navigation positioning method according to the first aspect above.

[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the parking navigation positioning method as described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A flowchart of a parking navigation positioning method provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of image feature composition of an image frame provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of the projection process provided by an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of a parking navigation and positioning device provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0069] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0070] As described in the relevant technology, since most parking lots are indoor parking lots, the positioning signal will be interfered by buildings in indoor parking lots, making the positioning signal very poor. In order to enhance the positioning signal, the existing solution is to install multiple Bluetooth, wireless broadcasting devices or ultra-wideband (UWB) devices in the parking lot so that the positioning signal covers the entire parking lot. However, the vehicle positioning accuracy using Bluetooth or wireless broadcasting is poor, and the use of high-precision UWB equipment is expensive, making the cost of parking lot renovation too high, which is not conducive to popularization in many parking lots.

[0071] In response to the problems in the above-mentioned prior art, the present application provides a parking navigation positioning method, which obtains a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame, predicts the vehicle theoretical position corresponding to the current image frame based on the previous image frame, and determines the target map point cloud that matches the current image frame; and reprojects the target map point cloud and the current image frame to determine the reprojection error, thereby verifying whether the vehicle theoretical position is reasonable by means of feature point matching, thereby improving the accuracy of the positioning result; and based on the reprojection error and the odometer error of the vehicle, determines the prediction error, thereby fusing the reprojection error and the odometer error to comprehensively consider the situation that causes the positioning error, thereby further improving the accuracy of the positioning result; finally, based on the theoretical position of the vehicle and the prediction error, determines the actual position of the vehicle corresponding to the current image frame, thereby verifying whether the theoretical position of the vehicle is the actual position of the vehicle through the prediction error, thereby improving the positioning accuracy.

[0072] See also Figure 1 , Figure 1 The following flowchart of a parking navigation positioning method provided by an embodiment of the present application is shown. The parking navigation positioning method described below in the embodiment of the present application can be applied to electronic devices, including a vehicle-mounted computer. The parking navigation positioning method in the embodiment of the present application includes steps S101 to S105, which are described in detail as follows:

[0073] Step S101, obtaining a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame.

[0074] In this step, the current image frame (or the current frame hereinafter) is an image frame captured at the current moment during the vehicle's driving process, and the previous image frame (or the previous frame hereinafter) is an image frame used for navigation and positioning relative to the current image frame. Both the current image frame and the previous image frame are captured by the vehicle's onboard camera.

[0075] Step S102, based on the previous image frame, predicting the theoretical position of the vehicle corresponding to the current image frame, and determining the target map point cloud matching the current image frame.

[0076] In this embodiment, the theoretical position of the vehicle is the vehicle position predicted based on the previous image frame. It can be understood that since the image captured by the vehicle camera is used for visual tracking, the vehicle position is the position of the vehicle camera. The target map point cloud is a set of image feature points corresponding to the preset parking lot point cloud map.

[0077] Optionally, the parking lot point cloud map includes two parts of data: keyframe data and point cloud data. Keyframe data is the key image frame (or hereinafter referred to as keyframe) selected by the algorithm in the visual mapping process according to conditions such as displacement distance and number of new feature points. Each keyframe data includes the following information: keyframe ID, keyframe position, image features observed by it, point cloud ID list observed by it, correspondence between image features observed by it and point cloud, keyframe ID with which it is co-viewed, map block ID to which it belongs, and other information.

[0078] Point cloud data is the three-dimensional points obtained from visual features using triangulation algorithms during visual mapping. Exemplarily, the steps of the triangulation algorithm are as follows: First, define the camera matrix P as P=KR[I|-C], where K is the camera intrinsic parameter, R is the camera rotation, and C is the camera translation. Then, for the same visual feature, let (u1, v1) be its coordinates in the first frame image, and the camera matrix of the first frame is P1; let (u2, v2) be its coordinates in the second frame image, and the camera intrinsic matrix of the second frame is P2. Then the triangulation formula can be described by the homogeneous equation: AX=0; where: The superscript in the formula indicates the row number of the matrix. Finally, after solving the triangulation formula, the three-dimensional coordinates of the visual feature can be expressed as: Each feature point in the point cloud includes: x-axis coordinate, y-axis coordinate, z-axis coordinate, the point ID and other information.

[0079] Optionally, the last image frame has determined the vehicle position during the last visual tracking operation of the vehicle, so the current theoretical position of the vehicle can be predicted based on the last position of the vehicle, the vehicle moving speed, and the time difference between the acquisition time point of the current image frame and the acquisition time point of the last image frame. Furthermore, the last image frame has been matched to a preset key image frame, so the corresponding map point cloud can be loaded according to the last matched key image frame to determine the target map point cloud of the current image frame.

[0080] In one embodiment, based on the actual position of the previous vehicle corresponding to the previous image frame, according to a preset uniform motion model, the theoretical position of the vehicle corresponding to the current image frame is predicted; based on the previous target map point cloud corresponding to the previous image frame, a local map point cloud of the parking lot where the vehicle is located is constructed; the local map point cloud is projected to the current image frame, and the target map point cloud in the local map point cloud that matches the current image is determined.

[0081] In this embodiment, the uniform motion model is a preset model that describes the vehicle's driving process, and the model parameters of the model can be adjusted based on the vehicle's current speed to obtain a motion model suitable for this visual tracking. The previous target map point cloud is the target map point cloud matched to the previous image frame during the previous visual tracking operation. The local map point cloud is the point cloud data of the local map of the parking lot where the vehicle is located.

[0082] Optionally, constructing a local map point cloud of the parking lot where the vehicle is located includes: determining all key image frames that can observe the previous target map point cloud; determining the common view key image frames of all key image frames; and constructing a local map point cloud based on the first map point clouds of all key image frames and the second map point clouds of the common view key image frames.

[0083] In this embodiment, all key frames that can observe the point cloud of the previous target map are obtained from the preset point cloud map of the parking lot; the common view key frames of the key frames are obtained from the preset point cloud map; the point clouds that can be observed by all the above key frames are obtained from the point cloud map, so as to construct a local map point cloud. The key image frame is the key frame data of the preset point cloud map, and the common view key image frame is another key image frame containing part or all of the data of the key image frame. Therefore, the local map point cloud can be obtained based on the previous target map point cloud and fused with the point cloud data corresponding to the common view key frame.

[0084] Optionally, the local map point cloud is projected onto the current image frame to determine a target map point cloud in the local map point cloud that matches the current image, including: projecting a first feature point in the local map point cloud onto the current image frame to obtain a first projection point; extracting image feature points of the current image frame to obtain a second feature point; matching the first projection point with the second feature point to obtain a matching value; and determining a target map point cloud in the local map point cloud that matches the current image based on the highest matching value.

[0085] In this embodiment, the image features of the current image frame include traditional visual features and deep learning features. Traditional visual features include, but are not limited to, SURF features, SIFT features, ORB features, and AprilTag features arranged in the parking lot. Each feature point data includes the following information: the position of the feature on the image, the number of layers of the image pyramid where the feature is located, the algorithm ID for feature acquisition (for example, 1 is SURF, 2 is SIFT, 3 is ORB, and 4 is AprilTag), and the description vector of the feature (different types of feature description vectors have different algorithms and different vector lengths).

[0086] For deep learning features, such as Figure 2As shown in the figure, the model includes a multi-layer image encoder, a multi-layer decoder for detecting feature points, and a multi-layer decoder for detecting feature description vectors. After the input image is encoded and decoded, its corresponding feature intensity map and feature description vector matrix are obtained. Among them, the feature intensity map is an image of the same size as the input image, and the intensity of each pixel is used to determine whether the point is selected as an image feature. The feature description vector is a three-dimensional matrix of the image size multiplied by the length of the description vector, giving the description vector corresponding to each pixel.

[0087] The training of the model is divided into two stages: For the first stage of training, a feasible method is to use virtual structures such as polygons and grid graphics. The corner points of these structures are known and can be used as feature points to supervise the training of feature intensity maps. The error function is:

[0088] L p =||X-Y||2

[0089] Among them, X is the feature intensity map output, and Y is the feature intensity map constructed by the virtual structure corner points.

[0090] For the second stage of training, a feasible method is to use self-supervision. For a real image I, use the model trained in the first stage to extract feature point X, perform an affine transformation on the image to obtain image J, and perform the same affine transformation on feature point X to obtain image feature point Y. The error term of the description vector is constructed as the error between the feature description vectors of the feature point pair:

[0091]

[0092] Among them, (X, Y) is the corresponding feature point pair, and D(·) is the description vector corresponding to the feature point. The total error of the second stage is the sum of the feature point supervision error and the feature vector error:

[0093] Optionally, the current frame and the key frame matching method can select any form of image matching method. One feasible method is to calculate the Euclidean distance between all feature description vectors of the current frame and the key frame, determine the one-to-one relationship between the vectors based on the distance, and the number of vector pairs with closer distances reflects the degree of matching.

[0094] Furthermore, the previous image frame is the initial image frame, the previous actual position of the vehicle is the initial position of the vehicle, and based on the previous actual position of the vehicle corresponding to the previous image frame, according to a preset uniform motion model, before predicting the theoretical position of the vehicle corresponding to the current image frame, it also includes: obtaining the initial image frame of the vehicle at the entrance of the parking lot; based on the image features of the initial image frame, matching the initial image frame with multiple preset key image frames of the entrance of the parking lot to obtain the matching degree between the initial image frame and the multiple preset key image frames; based on a preset n-point perspective algorithm, fusing and solving the initial image frame and the preset key image frame with the highest matching degree; if the fusion solution is successful, the solution result of the fusion solution is the initial position of the vehicle.

[0095] In this embodiment, the n-point perspective algorithm is a visual PnP algorithm, which is a common algorithm in multi-view geometry. The forms that can be adopted include but are not limited to direct linear transformation or minimizing reprojection errors. A feasible method is a PnP algorithm based on minimizing reprojection errors. For example, Figure 3 As shown in the figure, firstly, the correspondence between the feature points of the current frame and the feature points of the key frame is found through feature description vector matching; then the correspondence between the image features and the point cloud stored in the key frame is read to obtain the 3D point cloud points corresponding to the feature points in the key frame; thus, the 3D point cloud points corresponding to the feature points of the current frame can be obtained. The projection points can be obtained by reprojecting the 3D point cloud points to the current frame, and the reprojection errors between these projection points and the corresponding feature points are formed.

[0096] Step S103, reprojecting the target map point cloud and the current image frame to determine a reprojection error.

[0097] In one embodiment, the target map point cloud and the current image frame are reprojected to determine the reprojection error, including: extracting image feature points of the current image frame; projecting a third feature point in the target map point cloud to the current image frame to obtain a correspondence between the image feature point and the third feature point; determining a three-dimensional feature point corresponding to the image feature point based on the correspondence and a preset three-dimensional feature point corresponding to the third feature point; reprojecting the three-dimensional feature point to the current image frame to obtain a second projection point; and determining a reprojection error between the second projection point and the image feature point based on a preset reprojection error function.

[0098] In this embodiment, if Figure 3 As shown in the figure, firstly, the correspondence between the feature points of the current frame and the feature points of the key frame is found through feature description vector matching; then the correspondence between the image features and the point cloud stored in the key frame is read to obtain the 3D point cloud points corresponding to the feature points in the key frame; thus, the 3D point cloud points corresponding to the feature points of the current frame can be obtained. The projection points can be obtained by reprojecting the 3D point cloud points to the current frame, and the reprojection errors are formed between these projection points and the corresponding feature points.

[0099] Reprojection error formula:

[0100]

[0101] Where: i is the feature index, u is the two-dimensional coordinate of the feature on the image, P is the three-dimensional point coordinate corresponding to the feature, K is the camera intrinsic parameter, T is the camera extrinsic parameter, and s is the normalization factor.

[0102] Step S104, determining a prediction error based on the reprojection error and the odometer error of the vehicle.

[0103] In one embodiment, the prediction error is determined based on the reprojection error and the odometer error of the vehicle, including: calculating the reprojection error and the odometer error of the vehicle according to a preset prediction error function to obtain the prediction error; the calculation formula of the prediction error function is:

[0104] E(T)=aE p (T)+bE o (T)

[0105] Where E(T) is the prediction error, a is the reprojection error factor, and E p (T) is the reprojection error, b is the odometer error factor, E o (T) is the odometer error.

[0106] In this embodiment, the odometer error formula is:

[0107]

[0108] Where: T is the current frame position, T′ is the previous frame position, and odom is the odometer information given by the sensor. odom can be obtained from the vehicle chassis information:

[0109]

[0110] Where: v is the vehicle speed, theta is the steering wheel angle, dt is the time interval between the previous frame and the current frame, and r is the ratio of the steering wheel angle to the front wheel angle.

[0111] odom can also be obtained through the IMU sensor:

[0112] Among them: a is acceleration, v is velocity, o is angular acceleration, w is angular velocity, and dt is the time interval between the previous frame and the current frame.

[0113] Furthermore, after determining the actual position of the vehicle corresponding to the current image frame based on the vehicle theoretical position and the prediction error, it also includes: if the image feature points of the current image frame fail to match the target map point cloud, adjusting the reprojection error factor to a preset value; after the reprojection error factor is adjusted, redetermining the prediction error; based on the vehicle theoretical position and the redetermined prediction error, determining the actual position of the vehicle corresponding to the current image frame.

[0114] In this embodiment, if the image feature points of the current image frame fail to match the target map point cloud, it means that the visual tracking has failed. At this time, the system adjusts the back projection error factor in the fusion error to 0, resulting in the back projection error having no effect on the optimization result, and the fusion error only contains the odometer error. During the visual tracking failure process, the system optimizes the position according to the odometer error. When the visual tracking is restored, the optimized position of the odometer error is used as the position of the first frame image during the restoration, that is, the positional relationship between the image frames before and after the visual tracking is maintained.

[0115] Step S105, based on the theoretical position of the vehicle and the prediction error, determining the actual position of the vehicle corresponding to the current image frame.

[0116] In one embodiment, based on the theoretical position of the vehicle and the prediction error, the actual position of the vehicle corresponding to the current image frame is determined, including: based on a preset gradient descent algorithm, iterative operations are performed on the current image frame and the previous image frame until the prediction error is less than a preset value, and the theoretical position of the vehicle corresponding to the prediction error is the actual position of the vehicle.

[0117] In this embodiment, the current frame position T is solved using a gradient descent method. A feasible method is to use Lie algebraic derivation and the Levenberg-Marquardt algorithm.

[0118] In order to execute the method corresponding to the above method embodiment and achieve the corresponding functions and technical effects, a parking navigation positioning device is provided below. Figure 4 , Figure 4 : is a structural block diagram of a parking navigation positioning device provided in an embodiment of the present application. For the sake of convenience, only the parts related to the present embodiment are shown. The parking navigation positioning device provided in an embodiment of the present application includes:

[0119] An acquisition module 401 is used to acquire a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame;

[0120] A first determination module 402 is used to predict the theoretical position of the vehicle corresponding to the current image frame based on the previous image frame, and to determine the target map point cloud matching the current image frame;

[0121] A reprojection module 403 is used to reproject the target map point cloud and the current image frame and determine the reprojection error;

[0122] A second determination module 404, configured to determine a prediction error based on a reprojection error and an odometer error of the vehicle;

[0123] The third determination module 405 is used to determine the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error.

[0124] In one embodiment, the first determining module 402 includes:

[0125] A prediction unit, configured to predict a theoretical position of a vehicle corresponding to a current image frame based on an actual position of a previous vehicle corresponding to a previous image frame and a preset uniform motion model;

[0126] A construction unit, configured to construct a local map point cloud of the parking lot where the vehicle is located based on a previous target map point cloud corresponding to a previous image frame;

[0127] The first determination unit is used to project the local map point cloud to the current image frame and determine the target map point cloud that matches the current image in the local map point cloud.

[0128] Furthermore, the construction unit comprises:

[0129] A first determination subunit is used to determine all key image frames that can observe the previous target map point cloud;

[0130] A second determining subunit, used to determine a common-view key image frame of all key image frames;

[0131] The construction subunit is used to construct a local map point cloud based on the first map point cloud of all key image frames and the second map point cloud of the common view key image frame.

[0132] Furthermore, the first determining unit includes:

[0133] A projection subunit, used to project a first feature point in the local map point cloud to a current image frame to obtain a first projection point;

[0134] The extraction subunit is used to extract the image feature points of the current image frame to obtain the second feature points;

[0135] A matching subunit, used for matching the first projection point with the second feature point to obtain a matching value;

[0136] The third determination subunit is used to determine a target map point cloud that matches the current image in the local map point cloud based on the highest matching value.

[0137] Furthermore, the above device also includes:

[0138] A second acquisition module is used to acquire an initial image frame of the vehicle at the entrance of the parking lot;

[0139] The matching module is used to match the initial image frame with a plurality of preset key image frames of the parking lot entrance based on the image features of the initial image frame, and obtain the matching degree between the initial image frame and the plurality of preset key image frames:

[0140] The solution module is used to perform fusion solution on the initial image frame with the highest matching degree and the preset key image frame based on the preset n-point perspective algorithm; if the fusion solution is successful, the solution result of the fusion solution is the initial position of the vehicle.

[0141] In one embodiment, the reprojection module 403 includes:

[0142] An extraction unit, used for extracting image feature points of the current image frame;

[0143] A projection unit, used to project the third feature point in the target map point cloud to the current image frame to obtain a corresponding relationship between the image feature point and the third feature point;

[0144] A second determining unit, configured to determine the three-dimensional feature point corresponding to the image feature point based on the corresponding relationship and the preset three-dimensional feature point corresponding to the third feature point;

[0145] A reprojection unit, used to reproject the three-dimensional feature points to the current image frame to obtain a second projection point;

[0146] The third determining unit is used to determine the reprojection error between the second projection point and the image feature point based on a preset reprojection error function.

[0147] In one embodiment, the second determining module 404 includes:

[0148] The calculation unit is used to calculate the reprojection error and the odometer error of the vehicle according to a preset prediction error function to obtain the prediction error; the calculation formula of the prediction error function is:

[0149] E(T)=aE p (T)+bE o (T)

[0150] Where E(T) is the prediction error, a is the reprojection error factor, and E p (T) is the reprojection error, b is the odometer error factor, E o (T) is the odometer error.

[0151] Furthermore, the above device also includes:

[0152] An adjustment module is used to adjust the reprojection error factor to a preset value if the actual position of the vehicle fails to be determined:

[0153] A fourth determination module, used to re-determine the prediction error after the reprojection error factor is adjusted;

[0154] The fifth determination module is used to determine the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the re-determined prediction error.

[0155] In one embodiment, the third determination module 405 includes:

[0156] The iteration unit is used to perform iterative operations on the current image frame and the previous image frame based on a preset gradient descent algorithm until the prediction error is less than a preset value, and the theoretical position of the vehicle corresponding to the prediction error is the actual position of the vehicle.

[0157] The above-mentioned parking navigation positioning device can implement the parking navigation positioning method of the above-mentioned method embodiment. The optional items in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of this application can refer to the contents of the above-mentioned method embodiment and will not be described in detail in this embodiment.

[0158] Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown in the figure) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, and when the processor 50 executes the computer program 52, the steps in any of the above method embodiments are implemented.

[0159] The electronic device 5 may be a computing device such as a smart phone, a tablet computer, a desktop computer, a supercomputer, a personal digital assistant, a physical server, a cloud server, etc. The electronic device may include but is not limited to a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 5 and does not constitute a limitation on the electronic device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0160] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the electronic device 5. The memory 51 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0162] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments can be implemented.

[0163] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0164] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0165] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0166] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0169] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A parking navigation positioning method, characterized in that: include: Acquire a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame; Based on the previous image frame, predicting a theoretical position of the vehicle corresponding to the current image frame, and determining a target map point cloud matching the current image frame; Reprojecting the target map point cloud and the current image frame to determine a reprojection error; determining a prediction error based on the reprojection error and an odometer error of the vehicle; Based on the theoretical position of the vehicle and the prediction error, an actual position of the vehicle corresponding to the current image frame is determined.

2. The parking navigation positioning method according to claim 1, characterized in that: The predicting, based on the previous image frame, the theoretical position of the vehicle corresponding to the current image frame, and determining the target map point cloud matching the current image frame includes: Based on the actual position of the previous vehicle corresponding to the previous image frame, according to a preset uniform motion model, predicting the theoretical position of the vehicle corresponding to the current image frame; Constructing a local map point cloud of the parking lot where the vehicle is located based on the previous target map point cloud corresponding to the previous image frame; The local map point cloud is projected onto the current image frame, and a target map point cloud matching the current image frame is determined in the local map point cloud.

3. The parking navigation positioning method according to claim 2, characterized in that: The step of constructing a local map point cloud of the parking lot where the vehicle is located based on the previous target map point cloud corresponding to the previous image frame includes: Determine all key image frames capable of observing the previous target map point cloud; Determining a common-view key image frame of all the key image frames; The local map point cloud is constructed based on the first map point clouds of all the key image frames and the second map point cloud of the common view key image frame.

4. The parking navigation positioning method according to claim 2, characterized in that: The projecting the local map point cloud to the current image frame and determining a target map point cloud in the local map point cloud that matches the current image frame includes: Projecting a first feature point in the local map point cloud to the current image frame to obtain a first projection point; Extracting image feature points of the current image frame to obtain second feature points; Matching the first projection point with the second feature point to obtain a matching value; Based on the highest matching value, a target map point cloud in the local map point cloud that matches the current image frame is determined.

5. The parking navigation positioning method according to claim 2, characterized in that: The previous image frame is an initial image frame, the previous actual position of the vehicle is an initial position of the vehicle, and before predicting the theoretical position of the vehicle corresponding to the current image frame based on the previous actual position of the vehicle corresponding to the previous image frame and according to a preset uniform motion model, the method further includes: Acquire an initial image frame of the vehicle at the entrance of the parking lot; Based on the image features of the initial image frame, matching the initial image frame with a plurality of preset key image frames of the parking lot entrance to obtain a matching degree between the initial image frame and the plurality of preset key image frames; Based on a preset n-point perspective algorithm, a fusion solution is performed on the initial image frame with the highest matching degree and the preset key image frame; If the fusion solution is successful, the solution result of the fusion solution is the initial position of the vehicle.

6. The parking navigation positioning method according to claim 1, characterized in that: The reprojecting the target map point cloud and the current image frame to determine a reprojection error includes: Extracting image feature points of the current image frame; Projecting the third feature point in the target map point cloud to the current image frame to obtain a corresponding relationship between the image feature point and the third feature point; Determine the three-dimensional feature point corresponding to the image feature point based on the corresponding relationship and the preset three-dimensional feature point corresponding to the third feature point; Reprojecting the three-dimensional feature points to the current image frame to obtain second projection points; Based on a preset reprojection error function, a reprojection error between the second projection point and the image feature point is determined.

7. The parking navigation positioning method according to claim 1, characterized in that: The determining of a prediction error based on the reprojection error and an odometer error of the vehicle comprises: According to a preset prediction error function, the reprojection error and the odometer error of the vehicle are calculated to obtain the prediction error; the calculation formula of the prediction error function is: E(T)=aE p (T)+bE o (T) Where E(T) is the prediction error, a is the reprojection error factor, and E p (T) is the reprojection error, b is the odometer error factor, E o (T) is the odometer error.

8. The parking navigation positioning method according to claim 7, characterized in that: After determining the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error, the method further includes: If the image feature points of the current image frame fail to match the target map point cloud, adjusting the reprojection error factor to a preset value; re-determining the prediction error after the reprojection error factor is adjusted; Based on the theoretical position of the vehicle and the re-determined prediction error, the actual position of the vehicle corresponding to the current image frame is determined.

9. The parking navigation positioning method according to claim 1, characterized in that: The determining, based on the theoretical position of the vehicle and the prediction error, the actual position of the vehicle corresponding to the current image frame comprises: Based on a preset gradient descent algorithm, an iterative operation is performed on the current image frame and the previous image frame until the prediction error is less than a preset value, and the theoretical position of the vehicle corresponding to the prediction error is the actual position of the vehicle.

10. A parking navigation positioning device, characterized in that: include: An acquisition module, used to acquire a current image frame in front of the vehicle and a previous image frame corresponding to the current image frame; A first determination module, configured to predict a theoretical vehicle position corresponding to the current image frame based on the previous image frame, and determine a target map point cloud matching the current image frame; A reprojection module, used to reproject the target map point cloud and the current image frame to determine a reprojection error; A second determination module, configured to determine a prediction error based on the reprojection error and an odometer error of the vehicle; The third determination module is used to determine the actual position of the vehicle corresponding to the current image frame based on the theoretical position of the vehicle and the prediction error.

11. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the parking navigation positioning method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the parking navigation positioning method as claimed in any one of claims 1 to 9.

Images