Parking positioning method and device based on geometric constraint

Abstract

The invention relates to the technical field of parking positioning, and discloses a parking positioning method and device based on geometric constraints. The method comprises the following steps: on the basis of vehicle state information obtained by a vehicle-mounted sensor of a vehicle, determining an initial pose of the vehicle by adopting an error state Kalman filter; based on a parking environment aerial view image of the vehicle acquired by a visual input component of the vehicle, determining semantic features contained in the parking environment aerial view image; based on the initial pose and the semantic features, constructing a parking map of the vehicle in a standard coordinate system; the semantic features in the parking map serve as geometric constraints, the initial pose of the vehicle is optimized, and the optimized pose of the vehicle is determined; the optimized pose is used for updating the parking map. According to the method, the semantic features in the parking map are used as geometric constraints, and the initial pose determined based on the error state Kalman filter during vehicle parking is optimized, so that the accuracy during parking positioning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of parking positioning, and in particular to a parking positioning method and device based on geometric constraints. Background Art

[0002] With the development of autonomous driving technology, parking positioning systems have been widely used in intelligent vehicles. In the prior art, the positioning of vehicles is often achieved by relying on the Global Positioning System (GPS).

[0003] However, in complex parking environments such as underground parking lots, due to the inability to receive GPS signals, there may be a problem of insufficient positioning accuracy during parking positioning, and a better solution is needed. Summary of the invention

[0004] In view of this, the present invention provides a parking positioning method and device based on geometric constraints to solve the problem of insufficient parking positioning accuracy when there is a lack of GPS signals.

[0005] In one aspect, the present invention provides a parking positioning method based on geometric constraints, the method comprising determining an initial position and posture of the vehicle using an error state Kalman filter based on vehicle state information acquired by an onboard sensor of the vehicle;

[0006] Determining semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment acquired by the visual input component of the vehicle;

[0007] Based on the initial position and the semantic features, construct a parking map of the vehicle in a standard coordinate system;

[0008] The semantic features in the parking map are used as geometric constraints to optimize the initial posture of the vehicle and determine the optimized posture of the vehicle; the optimized posture is used to update the parking map.

[0009] Another aspect of the present invention provides a parking positioning device based on geometric constraints, the device comprising: an initial posture evaluation module, for determining the initial posture of the vehicle based on vehicle state information acquired by an onboard sensor of the vehicle and using an error state Kalman filter;

[0010] a feature extraction module, configured to determine semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment acquired by the visual input component of the vehicle;

[0011] A mapping module, configured to construct a parking map of the vehicle in a standard coordinate system based on the initial posture and the semantic features;

[0012] The posture optimization module is used to optimize the initial posture of the vehicle by taking the semantic features in the parking map as geometric constraints, and determine the optimized posture of the vehicle; the optimized posture is used to update the parking map.

[0013] On the other hand, the present invention further provides a computer device, which includes: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned parking positioning method based on geometric constraints by executing the computer instructions.

[0014] Another aspect of the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned parking positioning method based on geometric constraints.

[0015] In this process, the dynamic information of the vehicle is obtained based on the on-board sensors to determine the initial position of the vehicle. The semantic features in the parking map are used as geometric constraints to optimize the initial position of the vehicle during parking, thereby improving the accuracy of parking positioning. By continuously optimizing the initial position of the vehicle during parking positioning based on semantic features without using GPS, the accuracy of parking positioning can be improved in complex environments without GPS. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of a parking positioning method based on geometric constraints provided by an embodiment of the present invention;

[0018] Figure 2 is an IPM schematic diagram of a parking positioning method based on geometric constraints provided by an embodiment of the present invention;

[0019] Figure 3 is a flow chart of another parking positioning method based on geometric constraints provided by an embodiment of the present invention;

[0020] Figure 4 1 is a schematic diagram of the structure of a parking positioning device based on geometric constraints provided by an embodiment of the present invention;

[0021] Figure 5It is a schematic structural diagram of another parking positioning device based on geometric constraints provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] With the continuous advancement of autonomous driving technology, the importance of parking positioning systems in smart vehicles has become increasingly prominent. The core task of the parking positioning system is to ensure that the vehicle can be accurately positioned in various environments, especially in parking lots with limited space and various obstacles. However, traditional positioning methods such as GPS, inertial measurement units (IMUs) and wheel speed meters often face challenges in these complex environments. Especially in underground parking lots, due to building obstructions and the inability of GPS signals to penetrate, vehicles cannot receive reliable satellite signals, which leads to the problem of insufficient positioning accuracy in application scenarios where GPS signals are lacking.

[0023] To solve the above problems, various embodiments of the present invention provide a parking positioning method based on geometric constraints, including: using an error state Kalman filter to determine the initial posture of the vehicle; determining the semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the vehicle's visual input component; constructing a parking map of the vehicle in a standard coordinate system based on the initial posture and the semantic features; using the semantic features in the parking map as geometric constraints to optimize the initial posture of the vehicle and determine the optimized posture of the vehicle; and using the optimized posture to update the parking map.

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] According to an embodiment of the present invention, a parking positioning method based on geometric constraints is provided. Figure 1 is a flow chart of a parking positioning method based on geometric constraints provided by an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0026] Step S101, based on vehicle state information acquired by the vehicle's onboard sensor, an error state Kalman filter is used to determine the initial position and posture of the vehicle;

[0027] Step S102, determining semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment acquired by the visual input component of the vehicle;

[0028] Step S103, constructing a parking map of the vehicle in a standard coordinate system based on the initial position and semantic features;

[0029] Step S104, using the semantic features in the parking map as geometric constraints, optimizing the initial posture of the vehicle, and determining the optimized posture of the vehicle; the optimized posture is used to update the parking map.

[0030] In one possible implementation, a vehicle's on-board sensors may refer to sensor devices installed on the vehicle for monitoring and collecting vehicle status information; for example, the on-board sensors may include but are not limited to: an inertial measurement unit, a wheel speed meter, a camera, etc.; the vehicle status information obtained by the on-board sensors may include but is not limited to: vehicle acceleration, angular velocity, wheel speed, vehicle travel distance, etc.

[0031] Furthermore, an Error-State Kalman Filter (ESKF) that determines an initial posture based on vehicle state information may refer to a Kalman filter designed to handle problems with a nonlinear relationship between system states and observations.

[0032] Here, the format of the vehicle's initial posture may be a posture vector, which may include the vehicle's position information and posture information. The posture vector may represent the specific position and orientation of the vehicle in three-dimensional space.

[0033] In one possible implementation, the visual input component of the vehicle may refer to a surround view camera of the vehicle, which may provide the vehicle with a panoramic view of the surrounding environment; the surround view camera may be installed in different directions of the vehicle, front, back, left, and right.

[0034] Furthermore, the bird's-eye view image of the parking environment of the vehicle obtained based on the visual input component may refer to a panoramic image of the vehicle, and the semantic features contained in the bird's-eye view image of the parking environment may refer to identifiable elements that can provide rich environmental information. For example, the semantic features may include but are not limited to: parking spaces, lane lines, pillars, walls, etc.

[0035] Among them, the bird's-eye view image (Inverse Perspective Mapping, IPM) can refer to the overhead view image of the vehicle's surrounding environment generated by performing geometric transformation processing on the environmental image obtained by the vehicle-mounted camera; the geometric transformation processing can refer to projecting the coordinates in the standard coordinate system into the IPM plane based on a preset conversion relationship.

[0036] In a possible implementation, constructing geometric constraints based on semantic features in the parking map may include:

[0037] Determine the geometric relationship between the vehicle and the parking space lines;

[0038] Determine the relative position and angle between the lane line and the vehicle.

[0039] The geometric relationship between the vehicle and the parking space lines may mean that the vehicle should remain parallel to the parking space lines and the vehicle should be within the range of the parking space lines.

[0040] Furthermore, using geometric constraints to optimize the initial posture of the vehicle and determining the optimized posture of the vehicle is a common technical means and will not be described in detail here.

[0041] In a possible implementation, the parking map may be further updated based on the optimized position and posture of the vehicle, and the semantic features in the updated parking map may continue to construct geometric constraints for optimizing the position and posture of the vehicle at the next moment.

[0042] Here, the updating of the parking map and the optimization of the vehicle posture can be a closed-loop feedback optimization mechanism with a two-way relationship; a globally consistent parking map can be generated through the closed-loop optimization process, and the vehicle posture in the parking map can be strictly aligned with the map.

[0043] Through the above method, by using the semantic features in the parking map as geometric constraints to optimize the initial posture of the vehicle, the positioning error can be reduced and the accuracy and robustness of the positioning results can be improved; high-precision positioning and optimization are achieved using visual semantic features and geometric constraints. This multi-sensor fusion method can reduce the uncertainty and drift caused by a single sensor, allowing the vehicle to maintain high positioning accuracy in complex environments.

[0044] In a specific embodiment, based on the vehicle state information obtained by the vehicle-mounted sensor, an error state Kalman filter is used to determine the initial position of the vehicle, which is implemented based on the following steps:

[0045] Based on the vehicle acceleration and vehicle angular velocity obtained by the inertial measurement unit in the vehicle-mounted sensor, and the vehicle wheel speed obtained by the wheel speed meter in the vehicle-mounted sensor, an error state Kalman filter is used to determine the initial position and posture of the vehicle.

[0046] Specifically, the distance moved by the vehicle within a preset time is determined based on the vehicle's acceleration, and the current position of the vehicle is obtained by combining the moving distance with the vehicle's initial position; the current rotation angle of the vehicle is determined based on the angular velocity provided by the IMU; the predicted posture of the vehicle is corrected based on the vehicle wheel speed provided by the speedometer, and the error state Kalman filter is used to dynamically adjust the posture based on the current observation error. After continuous circulation of the filter, the initial posture of the vehicle is obtained by convergence.

[0047] For example, the initial posture of the vehicle at time t can be determined by the posture matrix T t .

[0048] Through the above method, the use of multi-sensor fusion can effectively improve the estimation accuracy of the vehicle's initial posture and reduce the drift or error introduced by a single sensor; the prediction and update process of the error state Kalman filter can quickly respond to changes in the vehicle state. When the vehicle starts or changes its posture, the system can quickly converge to an accurate posture estimate, thereby improving the real-time performance of initial posture acquisition; the vehicle does not need to rely on a high-precision map to determine the vehicle's posture, but only needs to obtain it through the vehicle's own sensors, which improves the ease of use of parking positioning.

[0049] In a specific embodiment, based on the bird's-eye view image of the parking environment of the vehicle acquired by the visual input component of the vehicle, determining the semantic features contained in the bird's-eye view image of the parking environment is achieved based on the following steps:

[0050] The vehicle's visual input component is used to obtain the vehicle's surround view image, and the surround view image is processed by a preset stitching process to obtain a bird's-eye view image of the vehicle's parking environment;

[0051] A feature extraction network is used to identify the semantic features contained in the bird's-eye view image of the parking environment, and the semantic features include lane lines and parking spaces.

[0052] In a possible implementation, a visual input component of the vehicle is used to obtain a surround view image of the vehicle, and a preset stitching process is performed on the surround view image to obtain a bird's-eye view image of the parking environment of the vehicle, which is implemented based on one of the following steps:

[0053] Cameras at different positions of the vehicle are used to synchronously capture images of the vehicle's surrounding environment, and the fisheye distortion correction algorithm is used to pre-process the acquired surrounding environment images so that the images conform to the correct perspective projection;

[0054] The change matrix is ​​used to geometrically transform each pixel in the surrounding environment image and project the surrounding environment image into a unified plane coordinate system.

[0055] The weighted average method and the image registration algorithm based on feature point matching are used to fuse and stitch the overlapping areas of the images corresponding to each camera in the plane coordinate system.

[0056] The evaluated image is subjected to geometric optimization processing so that the objects around the vehicle present the correct shape and proportion in the image, so as to obtain a bird's-eye view image of the parking environment of the vehicle.

[0057] The image registration algorithm based on feature point matching may include, but is not limited to, a Scale-Invariant Feature Transform (SIFT) algorithm and a Speeded-Up Robust Features (SURF) algorithm.

[0058] In a possible implementation, a feature extraction network is used to identify semantic features contained in a bird's-eye view image of a parking environment, where the semantic features include lane lines and parking spaces, based on one of the following steps:

[0059] A feature extraction network is used to identify lane lines and parking spaces contained in a bird's-eye view image of a parking environment, as well as four intersection points corresponding to at least one control point on the lane line and the parking space.

[0060] Here, the feature extraction network may specifically refer to a neural network for extracting high-level features from an input bird's-eye view image of a parking environment; for example, the feature extraction network may refer to a convolutional neural network (CNN).

[0061] For example, the semantic feature geometry of lane lines and parking spaces obtained from the bird's-eye view image of the parking environment at the first moment using CNN can be in, It can refer to the corner point of the parking space, e can be the control point on the lane line, and ψ can refer to the parking space information of the parking space.

[0062] For example, Figure 2 is an IPM schematic diagram of a parking positioning method based on geometric constraints provided by an embodiment of the present invention; wherein, Figure 2 It is a bird's-eye view image of the parking environment that represents the vehicle's posture in a standard coordinate system. The black matrix can be used to represent the vehicle shape, and the white dots can be used to represent the corners of the parking space. The rectangular box surrounded by white dots can be used to represent the parking space information ψ of the parking space, the "x"-shaped point can represent the control point l on the lane line, and the dotted line formed by the control point can be used to represent the lane line.

[0063] In one possible implementation, the lane lines in the IPM map may be determined based on the following steps:

[0064] A preset number of control points are sampled in the 3D lane line in the standard coordinate system, and a cubic spline curve fitting equation is used to smoothly fit the 3D lane line based on the control points to determine the fitted lane line;

[0065] The preset optimization formula is used to optimize the fitted lane lines.

[0066] Among them, a preset number of control points are sampled in the 3D lane line in the standard coordinate system, and a cubic spline curve fitting equation is used based on the control points to smoothly fit the 3D lane line to determine the fitting lane line, including:

[0067] In the 3D lane line in the standard coordinate system, n control points C1, C2, ..., C n, set the interval λ∈(0,1) between two adjacent control points, for every 4 consecutive control points C m-1 ,C m ,C m+1 ,C m+2 , the following cubic spline curve fitting equation can be used to determine the fitting lane line:

[0068]

[0069] Here, λ represents the interval between two adjacent control points, that is, the interpolation position between adjacent control points. The first column vector can represent the basis function of the cubic spline curve, where each element represents a different polynomial part, λ 3 represents the cubic term, λ 2 represents the quadratic term, λ represents the linear term, and 1 represents the constant term; the second vector can represent the coefficient matrix of the spline curve, which represents the combination of the basis function and the control point vector. The τ in the matrix represents the degree of curvature during lane line fitting, and τ can be 0.5; the third vector can represent the control point vector, which represents 4 consecutive control points C m-1 ,C m ,C m+1 ,C m+2 .

[0070] Among them, a fitting lane line with smooth transition between control points is defined through the control points sampled in the 3D lane line and the cubic spline curve fitting equation.

[0071] Furthermore, a preset optimization formula is used to optimize the fitted lane line, including:

[0072] Assume that the set of 3D points in the 3D lane line in the standard coordinate system is Sample n control points in the fitted lane line in the IPM image, and use the following preset optimization formula to optimize the fitted lane line:

[0073]

[0074] here, is the set of 3D points in the 3D lane line in the standard coordinate system, that is, the actual measurement points obtained from the lane line detection algorithm; Indicates that the fitted lane line is in parameter The point at the position is used to represent the theoretical position corresponding to the fitted lane line; Represents the error between the 3D point on each 3D lane line and the theoretical position on the fitted lane line; |C m -C m-1 | indicates control point C m With C m-1The distance between them is used to represent the interval between control points in the fitted lane line; μ is the expected value of the control point interval, and μ can be taken as 1 meter; ‖|C m -C m-1 |-μ‖ 2 It can represent the error between the actual distance and the expected distance.

[0075] Among them, the purpose of presetting the optimization formula is to make the fitted lane line more accurately describe the 3D lane line, and to make the distribution spacing of the control points in the lane line also meet the expected spacing.

[0076] In one possible implementation, Figure 2 The lane line formed by the lane line control point l and the control point is determined by the above-mentioned cubic spline curve fitting equation and the preset optimization formula.

[0077] Through the above method, the cubic spline curve fitting equation and the preset optimization formula are used to determine the fitting lane line and the control points on the lane line, so that the fitting lane line can better match the actually detected lane line points, thereby improving the accuracy of lane line detection; optimizing the intervals between the control points so that the distribution of the control points on the curve is more uniform, which can improve the vehicle's lane line detection performance, thereby improving the accuracy of parking positioning.

[0078] In a specific embodiment, the semantic features in the parking map are used as geometric constraints to optimize the initial posture of the vehicle and determine the optimized posture of the vehicle, including:

[0079] Obtain parking space corner point information, sampling point information and parking space information contained in the semantic features of the parking map;

[0080] Determine a first residual function corresponding to the parking space corner point information, a second residual function corresponding to the sampling point, and a third residual function corresponding to the parking space information, and determine a geometric constraint corresponding to the error state Kalman filter based on the first residual function, the second residual function, and the third residual function;

[0081] The initial posture of the vehicle is optimized using geometric constraints to determine the optimized posture of the vehicle.

[0082] The residual function may be a function used to quantify the difference between the estimated value and the actual observed value.

[0083] In a possible implementation, the geometric constraints may include, but are not limited to: geometric relationships between parking space corner points, geometric relationships between lane line sampling points, and geometric relationships between parking space information.

[0084] Furthermore, the integrated geometric constraints are applied to the optimization process, and the initial posture of the vehicle is adjusted through ESKF iterative update so that the optimized posture can better match the semantic features in the parking map.

[0085] Through the above method, by introducing semantic features such as parking space corner points, lane line sampling points and parking space information into the optimization process, the initial posture of the vehicle can be corrected more accurately; in complex parking environments, the use of stable geometric constraints can significantly improve the robustness of the system; by utilizing existing sensor data and optimization algorithms with lower computational complexity, the computational burden and implementation cost of the system are reduced.

[0086] In a specific embodiment, the first residual function corresponding to the parking space corner point information is determined based on the following steps:

[0087] Obtaining the actual coordinates of the corner points of the parking space detected in the parking map in the vehicle center coordinate system, and the standard system corner point coordinates of the corner points of the parking space in the standard coordinate system;

[0088] Based on the current position and posture of the vehicle, the coordinates of the corner points of the standard system are transformed to the coordinates of the corner points of the vehicle system in the vehicle center coordinate system;

[0089] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the coordinates of the corner points of the vehicle system in the vehicle center coordinate system onto the parking map to obtain the predicted coordinates of the corner points of the parking space;

[0090] Based on the Euclidean distance between the actual coordinates and the predicted coordinates, a first residual function corresponding to the corner point of the parking space is determined.

[0091] Specifically, the first residual function corresponding to the corner point of the parking space is expressed by the following formula:

[0092]

[0093] Get the actual coordinates of the corner points of the parking space detected in the parking map in the vehicle center coordinate system And the standard coordinate system corner point coordinates of the parking space corner point in the standard coordinate system Based on the vehicle’s current position T t Determine the inverse matrix Coordinates of corner points of standard system Perform coordinate transformation and convert the standard system corner point coordinates Convert the vehicle corner coordinates to the vehicle center coordinate system Use the inverse matrix K of the preset camera intrinsic parameter matrix K -1 , the coordinates of the corner points of the vehicle system in the vehicle center coordinate system Project it onto the parking map to get the predicted coordinates of the corner points of the parking space Based on the Euclidean distance between the actual coordinates and the predicted coordinates Determine the first residual function corresponding to the corner point of the parking space.

[0094] The first residual function is used as a constraint to ensure that the vehicle's posture matches the actual parking space corner point as closely as possible during parking, so that the vehicle can accurately identify and locate the corner point of the parking space, thereby improving positioning accuracy.

[0095] Through the above method, the corner point constraint is introduced in the parking positioning, which can provide a stable positioning reference in complex environments and improve the overall effect and user experience of the automatic parking system.

[0096] In a specific embodiment, the second residual function corresponding to the sampling point is determined based on the following steps:

[0097] Obtaining the actual coordinates of the sampling points detected in the parking map in the vehicle center coordinate system, and the standard system sampling point coordinates determined by the lane line sampling function in the standard coordinate system;

[0098] Based on the current posture of the vehicle, the coordinates of the standard system sampling points are transformed to the coordinates of the vehicle system sampling points in the vehicle center coordinate system;

[0099] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the coordinates of the sampling points of the vehicle system in the vehicle center coordinate system onto the parking map to obtain the theoretical coordinates of the sampling points.

[0100] Based on the Euclidean distance between the actual coordinates and the theoretical coordinates of the sampling point, a second residual function corresponding to the sampling point is determined.

[0101] Specifically, the second residual function corresponding to the sampling point is expressed by the following formula:

[0102]

[0103] Get the actual coordinates of the sampling points detected in the parking map in the vehicle center coordinate system (i.e. the matching points of the control points in the fitted lane line) And the coordinates of the standard system sampling points (i.e. the control points in the fitted lane line) determined by the lane line sampling function in the standard coordinate system Based on the vehicle’s current position T t Determine the inverse matrix Coordinates of sampling points for the standard system Perform coordinate transformation and convert the sampling point coordinates of the standard system Convert the vehicle sampling point coordinates to the vehicle center coordinate system Use the inverse matrix K of the preset camera intrinsic parameter matrix K -1 , the coordinates of the sampling points of the vehicle system in the vehicle center coordinate system Project it onto the parking map to obtain the theoretical coordinates of the sampling points Based on the Euclidean distance between the actual coordinates and the theoretical coordinates Determine a second residual function corresponding to the sampling point.

[0104] The second residual function is used as a constraint to assist the lateral and longitudinal positioning of the vehicle during the parking process.

[0105] By introducing the sampling point constraints of the lane line in parking positioning through the above method, the position and direction of the vehicle relative to the lane line can be accurately determined, so that the vehicle can better plan the parking path, thereby improving the effect and efficiency of parking.

[0106] In a specific embodiment, the third residual function corresponding to the parking space information is determined based on the following steps:

[0107] Acquire the actual parking space information detected in the parking map in the vehicle center coordinate system, and the second parking space information in the standard coordinate system; the parking space information includes but is not limited to: parking space position, parking space shape;

[0108] Performing coordinate transformation on the second parking space information based on the current position and posture of the vehicle, and transforming the second parking space information into third parking space information in the center coordinate system of the vehicle;

[0109] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the third parking space information in the vehicle center coordinate system onto the parking map to obtain the theoretical parking space information;

[0110] Based on the Euclidean distance between the actual parking space information and the theoretical parking space information, a third residual function corresponding to the parking space information is determined.

[0111] Specifically, the third residual function corresponding to the parking space information is expressed by the following formula:

[0112]

[0113] Get the actual parking space information detected in the parking map in the vehicle center coordinate system And the second parking space information in the standard coordinate system Based on the vehicle’s current position T t Determine the inverse matrix Information about the second parking space Perform coordinate transformation and convert the sampling point coordinates of the standard system Convert the third parking space information into the vehicle center coordinate system Use the inverse matrix K of the preset camera intrinsic parameter matrix K -1 , the third parking space information in the vehicle center coordinate system Projected onto the parking map to obtain theoretical parking space information Based on the Euclidean distance between the actual parking space information and the theoretical parking space information Determine a third residual function corresponding to the parking space information.

[0114] Here, the position of the parking space may be determined according to the position of each corner point of the parking space, and the shape of the parking space may be determined according to the arrangement order of each corner point of the parking space.

[0115] Through the above method, by comparing the actual parking space information with the theoretical parking space information, the precise alignment of the parking space position and shape is ensured, thereby improving the parking positioning accuracy of the vehicle; combining the corner point alignment of the parking space with the overall alignment of the parking space, the parking efficiency of the vehicle can be improved.

[0116] Figure 3 FIG. 1 is a flow chart of another parking positioning method based on geometric constraints provided by an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0117] Step S301, obtaining vehicle status information;

[0118] Here, the vehicle acceleration and the vehicle angular velocity are obtained based on the inertial measurement unit in the vehicle-mounted sensor, and the vehicle wheel speed is obtained based on the wheel speed meter in the vehicle-mounted sensor;

[0119] Step S302, determining the initial posture;

[0120] Here, an error state Kalman filter is used to determine the initial position of the vehicle;

[0121] Step S303, obtaining a bird's-eye view image of the parking environment;

[0122] Here, a visual input component of the vehicle is used to obtain a surround view image of the vehicle, a preset splicing process is performed on the surround view image to obtain a bird's-eye view image of the parking environment of the vehicle, and semantic features in the bird's-eye view image of the parking environment are determined;

[0123] Step S304, creating a parking map;

[0124] Here, based on the initial posture and semantic features, a parking map of the vehicle in the standard coordinate system is constructed;

[0125] Step S305, determining geometric constraints;

[0126] Here, the parking space corner point information, sampling point information and parking space information included in the semantic features in the parking map are obtained; a first residual function corresponding to the parking space corner point information, a second residual function corresponding to the sampling point and a third residual function corresponding to the parking space information are determined, and based on the first residual function, the second residual function and the third residual function, the geometric constraint corresponding to the error state Kalman filter is determined;

[0127] Step S306, posture optimization;

[0128] Here, the geometric constraints are used to optimize the initial posture of the vehicle to determine the optimized posture of the vehicle; the process proceeds to step S304 to further optimize the parking map.

[0129] Figure 4 is a schematic diagram of a parking positioning device based on geometric constraints provided by an embodiment of the present invention, such as Figure 4 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: an initial posture evaluation module 401, a feature extraction module 402, a mapping module 403 and a posture optimization module 404;

[0130] The initial posture evaluation module 401 is used to determine the initial posture of the vehicle based on the vehicle state information obtained by the vehicle-mounted sensor of the vehicle by using an error state Kalman filter;

[0131] A feature extraction module 402 is used to determine semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment of the vehicle obtained by the visual input component of the vehicle;

[0132] A mapping module 403 is used to construct a parking map of the vehicle in a standard coordinate system based on the initial posture and semantic features;

[0133] The posture optimization module 404 is used to optimize the initial posture of the vehicle by taking the semantic features in the parking map as geometric constraints, and determine the optimized posture of the vehicle; the optimized posture is used to update the parking map.

[0134] In a specific embodiment, the initial posture evaluation module 401 is used to determine the initial posture of the vehicle based on the vehicle acceleration and vehicle angular velocity obtained by the inertial measurement unit in the vehicle-mounted sensor, and the vehicle wheel speed obtained by the wheel speed meter in the vehicle-mounted sensor, using an error state Kalman filter.

[0135] In a specific embodiment, the feature extraction module 402 is used to obtain a surround view image of the vehicle using a visual input component of the vehicle, and perform a preset splicing process on the surround view image to obtain a bird's-eye view image of the parking environment of the vehicle;

[0136] A feature extraction network is used to identify the semantic features contained in the bird's-eye view image of the parking environment, and the semantic features include lane lines and parking spaces.

[0137] In a specific embodiment, the posture optimization module 404 is used to obtain parking space corner point information, sampling point information and parking space information included in the semantic features in the parking map;

[0138] Determine a first residual function corresponding to the parking space corner point information, a second residual function corresponding to the sampling point, and a third residual function corresponding to the parking space information, and determine a geometric constraint corresponding to the error state Kalman filter based on the first residual function, the second residual function, and the third residual function;

[0139] The initial posture of the vehicle is optimized using geometric constraints to determine the optimized posture of the vehicle.

[0140] In a specific embodiment, the posture optimization module 404 is used to obtain the actual coordinates of the corner points of the parking space detected in the parking map in the vehicle center coordinate system, and the standard system corner point coordinates of the corner points of the parking space in the standard coordinate system;

[0141] Based on the current position and posture of the vehicle, the coordinates of the corner points of the standard system are transformed to the coordinates of the corner points of the vehicle system in the vehicle center coordinate system;

[0142] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the coordinates of the corner points of the vehicle system in the vehicle center coordinate system onto the parking map to obtain the predicted coordinates of the corner points of the parking space;

[0143] Based on the Euclidean distance between the actual coordinates and the predicted coordinates, a first residual function corresponding to the corner point of the parking space is determined.

[0144] In a specific embodiment, the posture optimization module 404 is used to obtain the actual coordinates of the sampling points detected in the parking map in the vehicle center coordinate system, and the standard system sampling point coordinates determined by the lane line sampling function in the standard coordinate system;

[0145] Based on the current posture of the vehicle, the coordinates of the standard system sampling points are transformed to the coordinates of the vehicle system sampling points in the vehicle center coordinate system;

[0146] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the coordinates of the sampling points of the vehicle system in the vehicle center coordinate system onto the parking map to obtain the theoretical coordinates of the sampling points.

[0147] Based on the Euclidean distance between the actual coordinates and the theoretical coordinates of the sampling point, a second residual function corresponding to the sampling point is determined.

[0148] In a specific embodiment, the posture optimization module 404 is used to obtain the actual parking space information detected in the parking map in the vehicle center coordinate system and the second parking space information in the standard coordinate system; the parking space information includes but is not limited to: the parking space position and the parking space shape;

[0149] Performing coordinate transformation on the second parking space information based on the current position and posture of the vehicle, and transforming the second parking space information into third parking space information in the center coordinate system of the vehicle;

[0150] The inverse matrix of the preset camera intrinsic parameter matrix is ​​used to project the third parking space information in the vehicle center coordinate system onto the parking map to obtain the theoretical parking space information;

[0151] Based on the Euclidean distance between the actual parking space information and the theoretical parking space information, a third residual function corresponding to the parking space information is determined.

[0152] It should be noted that: the parking positioning device based on geometric constraints provided in the above embodiment is only illustrated by the division of the above program modules when implementing the corresponding parking positioning method based on geometric constraints. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the device provided in the above embodiment and the corresponding Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0153] The embodiment of the present invention also provides a computer device having the above Figure 4 The parking positioning device based on geometric constraints is shown.

[0154] See also Figure 5 , Figure 5 is a schematic diagram of the structure of another parking positioning device based on geometric constraints provided by an embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0155] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0156] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0157] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0159] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0160] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0161] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0162] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A parking positioning method based on geometric constraints, characterized in that: The method comprises: Based on vehicle state information acquired by an onboard sensor of the vehicle, an error state Kalman filter is used to determine an initial position and posture of the vehicle; Determining semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment acquired by the visual input component of the vehicle; Based on the initial position and the semantic features, construct a parking map of the vehicle in a standard coordinate system; The semantic features in the parking map are used as geometric constraints to optimize the initial posture of the vehicle and determine the optimized posture of the vehicle; the optimized posture is used to update the parking map.

2. The method according to claim 1, characterized in that Based on the vehicle state information obtained by the vehicle's onboard sensors, an error state Kalman filter is used to determine the initial position of the vehicle, which is achieved based on the following steps: Based on the vehicle acceleration and vehicle angular velocity acquired by the inertial measurement unit in the vehicle-mounted sensor and the vehicle wheel speed acquired by the wheel speed meter in the vehicle-mounted sensor, an error state Kalman filter is used to determine the initial position and posture of the vehicle.

3. The method according to claim 2, characterized in that Determining semantic features contained in the bird's-eye view image of the parking environment of the vehicle acquired by the visual input component of the vehicle is based on the following steps: Using a visual input component of the vehicle to obtain a surround view image of the vehicle, performing a preset splicing process on the surround view image to obtain a bird's-eye view image of the parking environment of the vehicle; A feature extraction network is used to identify semantic features contained in the bird's-eye view image of the parking environment, wherein the semantic features include lane lines and parking spaces.

4. The method according to claim 3, characterized in that The method of using the semantic features in the parking map as geometric constraints to optimize the initial posture of the vehicle and determine the optimized posture of the vehicle includes: Obtain parking space corner point information, sampling point information and parking space information contained in the semantic features of the parking map; Determine a first residual function corresponding to the parking space corner point information, a second residual function corresponding to the sampling point, and a third residual function corresponding to the parking space information, and determine a geometric constraint corresponding to an error state Kalman filter based on the first residual function, the second residual function, and the third residual function; The geometric constraints are used to optimize the initial posture of the vehicle to determine the optimized posture of the vehicle.

5. The method according to claim 4, characterized in that The first residual function corresponding to the parking space corner point information is determined based on the following steps: Acquire the actual coordinates of the corner points of the parking space detected in the parking map in the vehicle center coordinate system, and the standard system corner point coordinates of the corner points of the parking space in the standard coordinate system; Based on the current posture of the vehicle, the coordinates of the corner point of the standard system are converted into the coordinates of the corner point of the vehicle system in the center coordinate system of the vehicle; Using the inverse matrix of the preset camera intrinsic parameter matrix, the coordinates of the corner points of the vehicle system in the vehicle center coordinate system are projected into the parking map to obtain the predicted coordinates of the corner points of the parking space; Based on the Euclidean distance between the actual coordinates and the predicted coordinates, a first residual function corresponding to the parking space corner point is determined.

6. The method according to claim 4, characterized in that The second residual function corresponding to the sampling point is determined based on the following steps: Obtaining the actual coordinates of the sampling points detected in the parking map in the vehicle center coordinate system, and the standard system sampling point coordinates determined by the lane line sampling function in the standard coordinate system; Based on the current posture of the vehicle, the coordinates of the standard system sampling points are converted into the coordinates of the vehicle system sampling points in the vehicle center coordinate system; Using the inverse matrix of the preset camera intrinsic parameter matrix, the coordinates of the sampling points of the vehicle system in the vehicle center coordinate system are projected into the parking map to obtain the theoretical coordinates of the sampling points; Based on the Euclidean distance between the actual coordinates of the sampling point and the theoretical coordinates, a second residual function corresponding to the sampling point is determined.

7. The method according to claim 4, characterized in that The third residual function corresponding to the parking space information is determined based on the following steps: Acquire the actual parking space information detected in the parking map in the vehicle center coordinate system, and the second parking space information in the standard coordinate system; the parking space information includes but is not limited to: parking space position, parking space shape; Performing coordinate transformation on the second parking space information based on the current position of the vehicle, and transforming the second parking space information into third parking space information in the center coordinate system of the vehicle; Using the inverse matrix of the preset camera intrinsic parameter matrix, projecting the third parking space information in the vehicle center coordinate system into the parking map to obtain theoretical parking space information; Based on the Euclidean distance between the actual parking space information and the theoretical parking space information, a third residual function corresponding to the parking space information is determined.

8. A parking positioning device based on geometric constraints, characterized in that: The device comprises: An initial posture evaluation module is used to determine the initial posture of the vehicle based on vehicle state information obtained by the vehicle-mounted sensor of the vehicle and using an error state Kalman filter; a feature extraction module, configured to determine semantic features contained in the bird's-eye view image of the parking environment of the vehicle based on the bird's-eye view image of the parking environment acquired by the visual input component of the vehicle; A mapping module, configured to construct a parking map of the vehicle in a standard coordinate system based on the initial posture and the semantic features; The posture optimization module is used to optimize the initial posture of the vehicle by taking the semantic features in the parking map as geometric constraints, and determine the optimized posture of the vehicle; the optimized posture is used to update the parking map.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the parking positioning method based on geometric constraints according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the parking positioning method based on geometric constraints according to any one of claims 1 to 7.

Images