Parking positioning method, parking method and parking device based on image key frame

Through the parking positioning method based on image keyframes, combined with GPS and high-precision maps, the problem of low automatic parking positioning accuracy is solved, and more efficient and accurate parking operations are achieved.

CN118494512BActive Publication Date: 2024-11-19Z-ONE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410961337.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-19
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The positioning accuracy of existing automatic parking technology is poor, and mismatch of similar scenarios is prone to occur, affecting the accuracy of parking routes.

Method used

Using the parking positioning method based on image keyframes, the parking positioning trajectory recording is triggered through GPS positioning and high-precision maps, the driving video image is obtained for inter-frame position estimation, the 3D map points are constructed and the 2D key point descriptor is recorded, and the storage is as Assisted parking positioning data, the positioning process is optimized using visual SLAM and beam adjustment method to improve the accuracy of parking routes.

Benefits of technology

It improves the driving accuracy and parking efficiency of the parking route, ensures that the vehicle can enter the parking space accurately according to the memory trajectory, and reduces mismatch situations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118494512B_ABST
    Figure CN118494512B_ABST
Patent Text Reader

Abstract

The present invention discloses a parking positioning method, a parking method and a parking device based on image key frames, wherein the parking method includes triggering automatic parking according to GPS positioning and high-precision maps, and determining the world coordinate origin of the parking positioning track according to the information of GPS positioning; recording the driving track, and performing image retrieval on the current frame video image in the driving track in the key frame image description subset; if the current frame video image matches the key frame image descriptor in the key frame image description subset, automatic parking is triggered; during the automatic parking process, the current video image is matched with the positioning data for auxiliary parking, and the vehicle is controlled to park in the parking space according to the parking positioning track in the auxiliary parking positioning data. The present invention performs multiple data matching on the driving video image and the stored auxiliary parking positioning data, so that the vehicle can complete the parking according to the memorized completed parking positioning track, and the driving route is more accurate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automatic parking, and in particular to a parking positioning method, a parking method and a parking device based on an image key frame. Background Art

[0002] With the continuous development of driverless technology, the Automated Valet Parking (AVP) function is also constantly improving. Automatic parking is a memory automatic parking function. When this function is turned on, the user needs to select a memory parking starting point, and then the vehicle will learn the driver's parking trajectory starting from the memory parking starting point. Then, when the vehicle reaches the vicinity of the memory parking starting point again, the vehicle will realize automatic parking according to the previously learned driver's parking trajectory.

[0003] In order to improve positioning accuracy, automatic parking in the prior art generally uses an extended Kalman filter algorithm to fuse the SLAM (simultaneous localization and mapping) output posture and the wheel speed odometer to calculate the current posture of the vehicle. However, this method has poor positioning effect and is prone to mismatching of similar scenes. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a parking positioning method, a parking method and a parking device based on image key frames, which can improve the driving accuracy of the parking route.

[0005] A first aspect of the present invention provides a parking positioning method based on image key frames, comprising:

[0006] Trigger parking location track recording based on GPS positioning and high-precision map, and determine the world coordinate origin of the parking location track based on GPS positioning information;

[0007] Acquire a driving video image, perform inter-frame pose estimation based on the driving video image, save a key frame image descriptor set, and record a parking positioning trajectory based on the inter-frame pose estimation;

[0008] Constructing 3D map points according to the driving video image, and recording 2D key point descriptors corresponding to the 3D map points;

[0009] The key frame image descriptor set, the parking positioning trajectory, the 3D map point and the 2D key point descriptor are stored as positioning data for assisting parking.

[0010] In an optional implementation, triggering parking location trajectory recording according to GPS positioning and high-precision map includes:

[0011] According to the high-precision map, it is determined that the current location is a parking area of ​​a building, and the signal strength of the GPS positioning is detected. When the signal strength of the GPS positioning is lower than a preset value, the parking positioning trajectory recording is triggered.

[0012] In an optional implementation manner, triggering the parking location track recording according to GPS positioning and high-precision map, and determining the world coordinate origin of the parking location track according to GPS positioning information; includes:

[0013] The IMU information of the vehicle is obtained, and the signal strength of the GPS positioning is detected when the upward acceleration in the IMU information disappears. When the signal strength of the GPS positioning is lower than a preset value, the coordinates of the position where the signal strength of the GPS positioning is lower than the preset value are used as the world coordinate origin of the parking positioning trajectory.

[0014] In an optional implementation manner, constructing 3D map points according to the driving video image includes:

[0015] Visual SLAM is used to construct 3D map points according to the driving video image, and the 3D map points are optimized using a bundle adjustment method.

[0016] A second aspect of the present invention provides an automatic positioning parking method based on the parking positioning method based on image key frames according to the first aspect of the present invention, comprising:

[0017] Trigger automatic parking based on GPS positioning and high-precision maps, and determine the world coordinate origin of the parking positioning trajectory based on GPS positioning information;

[0018] Record the driving trajectory, and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description subset;

[0019] If the current frame video image matches a key frame image descriptor in the key frame image descriptor set, automatic parking is triggered;

[0020] During the automatic parking process, the current video image is matched with the positioning data used for assisted parking, and the vehicle is controlled to park in the parking space according to the parking positioning trajectory in the positioning data for assisted parking.

[0021] In an optional implementation manner, matching the current video image with the positioning data for assisting parking during the automatic parking process includes:

[0022] Obtain target key points of the current video image, and perform pixel matching between the target key points and 2D key point descriptors in the positioning data of the auxiliary parking;

[0023] After the matching is successful, the 3D map point corresponding to the 2D key point descriptor is assigned to the target key point of the current video image.

[0024] In an optional implementation manner, the controlling the vehicle to park into a parking space according to the parking positioning trajectory in the positioning data of the assisted parking includes:

[0025] The current vehicle's posture is restored to be consistent with the inter-frame posture in the parking positioning trajectory according to the PnP posture solution algorithm, and the vehicle is parked in the parking space according to the parking positioning trajectory.

[0026] In an optional implementation, the pixel-matching of the target key point with the 2D key point descriptor in the positioning data of the assisted parking includes: using a bundle adjustment method to pixel-match the target key point with the 2D key point descriptor in the positioning data of the assisted parking.

[0027] A third aspect of the present invention provides an automatic positioning parking device, comprising:

[0028] The positioning module is used to trigger automatic parking according to GPS positioning and high-precision maps, and determine the world coordinate origin of the parking positioning trajectory according to the GPS positioning information;

[0029] A matching module is used to record the driving trajectory and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description subset;

[0030] a triggering module, configured to trigger automatic parking if the current frame video image matches a key frame image descriptor in the key frame image descriptor set;

[0031] The parking module is used to match the current video image with the positioning data for assisted parking during the automatic parking process, and control the vehicle to park in the parking space according to the parking positioning trajectory in the positioning data for assisted parking.

[0032] A fourth aspect of the present invention provides an electronic device, comprising:

[0033] At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the parking positioning method based on image key frames provided in the first aspect of the present invention or the automatic positioning parking method provided in the second aspect.

[0034] A fifth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, the parking positioning method based on image key frames provided in the first aspect of the present invention or the automatic positioning parking method provided in the second aspect thereof is executed.

[0035] After triggering the parking positioning trajectory through GPS positioning and high-precision map, the present invention determines the world coordinate origin of the parking positioning trajectory according to the GPS signal embedded end; then the inter-frame pose estimation of the video image recorded when driving on the parking positioning trajectory is performed to obtain a key frame image descriptor set; the map is constructed based on SLAM technology to obtain 3D map points and 2D key point descriptors; it is convenient to complete the parking positioning by matching multiple data during parking positioning, thereby improving the positioning accuracy during parking positioning.

[0036] After triggering the parking positioning trajectory through GPS positioning and high-precision map, the present invention determines the world coordinate origin of the parking positioning trajectory according to the GPS signal embedded end; multiple data matching is performed on the driving video image and the stored auxiliary parking positioning data, so that the vehicle can complete parking according to the memorized completed parking positioning trajectory, the parking efficiency is higher, and the driving route is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 The figure is a flow chart of a parking positioning method based on image key frames in an embodiment of the present invention.

[0039] Figure 2 The figure is a flow chart of an automatic positioning parking method in an embodiment of the present invention.

[0040] Figure 3 Schematic diagram of a module of an automatic positioning parking device in an embodiment of the present invention.

[0041] Figure 4 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] It should be understood that the terms "include" and "comprising" used in the description and claims of the present invention indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0044] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the claims, the singular forms of "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in the specification of the present invention and the claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0045] The problem of Simultaneous Localization and Mapping (SLAM) can be described as follows: a robot starts to move from an unknown position in an unknown environment, locates itself according to the position and map during the movement, and builds an incremental map based on its own positioning to achieve autonomous positioning and navigation of the robot. Simultaneous localization and mapping is divided into sparse mapping and dense mapping. The present invention is based on the simultaneous localization and mapping technology and is specifically implemented by the following method.

[0046] See also Figure 1 The parking positioning method based on image key frames provided by the present invention comprises the following steps:

[0047] Step 100: trigger parking location track recording according to GPS positioning and high-precision map, and determine the world coordinate origin of the parking location track according to GPS positioning information.

[0048] In this step, the user drives the vehicle into the parking lot, and the calculation and calibration are automatically performed. The world coordinate origin can be determined based on the GPS positioning information, while in the prior art it is usually determined by the user or by identifying a fixed reference object. However, if the fixed reference object is blocked or moved, it will affect the parking positioning.

[0049] The present invention determines whether a vehicle enters an underground parking lot by judging the signal strength of GPS positioning according to the actual application scenario. The signal strength of GPS positioning changes from strong to weak or disappears. Specifically, if the signal of GPS positioning disappears or it is detected that the vehicle enters the parking lot, the parking positioning function of the vehicle can be triggered.

[0050] In one implementation, it may be determined first whether the area is in a building, and then the signal strength of the GPS positioning may be determined to prevent signal interference.

[0051] For example, after determining that the current location is a parking area of ​​a building based on a high-precision map, the signal strength of the GPS positioning is detected. When the signal strength of the GPS positioning is lower than a preset value, the parking positioning trajectory recording is triggered.

[0052] The preset value can be set according to whether the signal strength of GPS positioning can meet the positioning requirements. It can be set according to international standards. Of course, each car company can also set it according to its own needs.

[0053] The high-precision map can determine that there is a parking lot at the current location. For example, if the current location is the Florentia Village Shanghai Premium Outlets - South Ground Parking Lot, it can be determined that the current location is in the parking area. When the vehicle drives to the South Ground Parking Lot, the signal strength of the GPS positioning is detected.

[0054] Furthermore, it is also possible to determine whether the vehicle has entered the parking lot area through the upward acceleration data of the vehicle during its travel.

[0055] For example, after the vehicle enters the parking lot area confirmed by the high-precision map, the IMU (inertial measurement unit) information of the vehicle is obtained, and the signal strength of the GPS positioning is detected when the upward acceleration in the IMU information disappears. When the signal strength of the GPS positioning is lower than a preset value, the coordinates of the position where the signal strength of the GPS positioning is lower than the preset value are used as the world coordinate origin of the parking positioning trajectory.

[0056] Step 200: Acquire a driving video image, perform inter-frame pose estimation based on the driving video image, save a key frame image descriptor set, and record a parking positioning trajectory based on the inter-frame pose estimation.

[0057] The driving video image can be acquired based on monocular vision SLAM, binocular vision SLAM, RGB-D or laser SLAM system. The inter-frame pose estimation of the driving video image can complete the vehicle driving trajectory, which corresponds to the parking positioning trajectory.

[0058] For example, the grayscale projection method is used to accumulate and sum the grayscale values ​​of pixels in the horizontal and vertical directions of the image, and construct the grayscale projection curves in the u and v directions for motion estimation. For video frames that are purely translated in the horizontal or vertical direction, the projection curve will move up or down accordingly, and the pixel offset values ​​between adjacent frames are calculated in reverse using the up, down, left, and right movement of the projection curve.

[0059] Another example is the use of feature matching-based methods to extract more obvious features in the image, such as feature points, edges, contours, etc. The extracted features are used to find the correspondence between the features according to certain matching rules, and then these correspondences are substituted into the image motion model to calculate the global motion vector.

[0060] The key frame image descriptor set is a set of descriptors of multiple key frame images. The image descriptor is used to describe the features in the image, and expresses the image by the distribution features of the pixels in the image. The key frame is a frame image containing more image information in the driving video image.

[0061] In this step, the driving trajectory and parking positioning trajectory are obtained according to the SLAM technology, and the key frame image description subset is recorded to facilitate data comparison and trajectory data use during subsequent parking.

[0062] Step 300: construct 3D map points according to the driving video image, and record 2D key point descriptors corresponding to the 3D map points.

[0063] Through the pose estimation content of the above SLAM technology, a pair of 3D points and 2D points can be obtained, that is, 3D map points with three-dimensional coordinates in the camera coordinate system, where the 2D key point descriptor is used to describe the key pixels of the video image, which are the coordinates in the pixel coordinate system of this frame. Since the pose of the two frames of driving video images before and after is unknown, the three-dimensional coordinates of the feature points in the coordinates of this frame are unknown. The pose between the two frames can be solved by the PnP (N-point perspective pose solution) algorithm. The absolute pose relationship between the camera coordinates and the world coordinates is obtained by using the coordinates of multiple control points in the three-dimensional scene and their perspective projection coordinates in the image.

[0064] Using visual SLAM to construct 3D map points based on the driving video image, it is also necessary to use the bundle adjustment method to optimize the 3D map points. For details, please refer to: Publication No. CN113269876 A, the invention name is map point coordinate optimization method and device, electronic device and storage medium. This solution proposes a solution for optimizing the coordinates of map points in map construction, specifically:

[0065] The camera is used to collect environmental video; features are extracted from the environmental video to obtain matching relationships between image points between frames; and the coordinates of the map points are determined using the matching relationships.

[0066] The map point coordinates are first optimized using a bundle adjustment method to obtain the first map point coordinates; a least squares problem of the bundle adjustment method is determined according to the pose and the map point coordinates, wherein the pose includes a camera pose or a laser pose. The least squares problem is solved using a Gauss-Newton method to obtain the first map point coordinates.

[0067] The relationship matrix between the pose error and the coordinate error is determined according to the first optimization.

[0068] An optimization model is established according to the matrix of the relationship matrix to perform a second optimization on the first map point coordinates to obtain the second map point coordinates. For example, an optimization model is established to maximize the value of the matrix trace; the optimization model is solved to obtain the second map point coordinates. When the map point coordinates are odometer map point coordinates, the odometer map point coordinates are first optimized using the bundle adjustment method to obtain the first map point coordinates; when the map point coordinates are the loop map point coordinates, the loop map point coordinates are first optimized using the bundle adjustment method to obtain the first map point coordinates.

[0069] The above-mentioned content cites and explains the optimization of the 3D map points using the bundle adjustment method. For any incomplete information, please refer to other contents recorded in CN113269876 A.

[0070] Step 400: storing the key frame image descriptor set, the parking positioning trajectory, the 3D map point and the 2D key point descriptor as positioning data for assisted parking.

[0071] This step stores the data when building a map based on SLAM so as to provide a method for automatic positioning parking. It should be understood that the method for automatic positioning parking in the present invention is similar to the parking positioning method based on image keyframes, that is, the map is built based on SLAM, but the data recorded in the parking positioning method is matched during the construction process, the vehicle's posture is adjusted to the same posture as the stored parking positioning, and then parking is completed according to the previously recorded parking positioning trajectory.

[0072] It can be seen from the signal strength of the GPS positioning that the parking positioning method based on the image key frame provided by the present invention is suitable for underground parking lots, such as Figure 2 As shown, the present invention also provides an automatic positioning parking method based on the above-mentioned parking positioning method based on image key frames, comprising the following steps:

[0073] Step 210: triggering automatic parking according to GPS positioning and high-precision map, and determining the world coordinate origin of the parking positioning trajectory according to the GPS positioning information.

[0074] As described in step 100 above, GPS positioning and high-precision maps are also used in the automatic positioning parking process to trigger scene functions and determine the world coordinate origin.

[0075] Step 220: record the driving trajectory, and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description sub-set.

[0076] During the parking process, SLAM is turned on to simultaneously locate and map the vehicle trajectory, that is, the inter-frame pose estimation records the trajectory. At the same time, the video images collected during the current parking process (i.e., the vehicle driving process) are obtained, and the current frame video image is searched in the key frame image description subset. The purpose is to perform image matching to synchronize the current driving trajectory with the stored parking positioning trajectory.

[0077] Step 230: If the current frame video image matches a key frame image descriptor in the key frame image descriptor set, automatic parking is triggered.

[0078] If the current frame video image matches the stored key frame image description subset, it means that the current parking route is consistent with the stored parking route, and the stored parking positioning trajectory can be used for parking.

[0079] In the present invention, the matching process may be: obtaining the target key points of the current video image, and matching the target key points with the 2D key point descriptors in the positioning data of the auxiliary parking. The target key points are the key points of the key frame, that is, at each candidate position, the position and scale are determined by a fine fitting model, and the selection of key points is based on their stability.

[0080] Step 240: During the automatic parking process, the current video image is matched with the positioning data for assisted parking, and the vehicle is controlled to park into the parking space according to the parking positioning trajectory in the positioning data for assisted parking.

[0081] Specifically, the target key point of the current video image is obtained, and the target key point is pixel-matched with the 2D key point descriptor in the positioning data of the auxiliary parking; the target key point is pixel-matched with the 2D key point descriptor in the positioning data of the auxiliary parking by using the bundle adjustment method. After the matching is successful, the 3D map point corresponding to the 2D key point descriptor is assigned to the target key point of the current video image.

[0082] As can be seen from the above, in the coordinate system, the 2D key point descriptor corresponds to the 3D map point. After the matching is successful, the 3D map point corresponding to the 2D key point descriptor is assigned to the target key point of the current video image.

[0083] In this step, PnP processing is also required to restore the current vehicle's posture. The posture between two frames can be solved by the PnP algorithm. The absolute posture relationship between the camera coordinates and the world coordinates is obtained by using the coordinates of multiple control points in the three-dimensional scene and their perspective projection coordinates in the image.

[0084] For example, feature extraction or optical flow calculation is performed on the captured video to obtain the matching relationship of image points between frames. Feature point matching can use optical flow tracking to obtain the matching of key points. At the same time, in order to eliminate mistracking, consistency detection can be used. If the amount of motion and appearance between the two frames change greatly, it is necessary to calculate the feature points and descriptors between the two frames and compare the distances between the descriptors. Due to the amount of calculation, a constant velocity model can also be used to search for potential correspondences in the expected area. If the depth of each pixel is calculated in binocular matching or a depth filter, polar line search and normalized cross-correlation or absolute error can be used to find matching points. After the feature match is determined, the basic matrix F or homography matrix H can be calculated using the matching relationship, and then F or H can be subjected to singular value decomposition to obtain the pose of the camera between the two frames. The coordinates of the map point can be obtained using the triangulation method based on the correspondence between the pose and the image point.

[0085] In summary, in this step, the posture of the current vehicle is restored to be consistent with the inter-frame posture in the parking positioning trajectory according to the PnP posture solution algorithm, and the vehicle is parked in the parking space according to the parking positioning trajectory.

[0086] It should be understood that the algorithms involved in the present invention are known technologies, and therefore the disclosed technologies are not fully described. The purpose of the present invention is to provide a new method for automatic parking positioning; that is, after the present invention triggers the parking positioning trajectory through GPS positioning and high-precision maps, the world coordinate origin of the parking positioning trajectory is determined according to the GPS signal embedded end; multiple data matching is performed on the driving video image and the stored auxiliary parking positioning data, so that the vehicle can complete parking according to the memorized completed parking positioning trajectory, the parking efficiency is higher, and the driving route is more accurate.

[0087] like Figure 3 As shown, the present invention provides an automatic positioning parking device, comprising:

[0088] The positioning module 31 is used to trigger automatic parking according to GPS positioning and high-precision maps, and determine the world coordinate origin of the parking positioning trajectory according to the GPS positioning information;

[0089] A matching module 32 is used to record the driving trajectory and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description subset;

[0090] A triggering module 33, configured to trigger automatic parking if the current frame video image matches a key frame image descriptor in the key frame image descriptor set;

[0091] The parking module 34 is used to match the current video image with the positioning data for auxiliary parking during the automatic parking process, and control the vehicle to park in the parking space according to the parking positioning trajectory in the positioning data for auxiliary parking.

[0092] For any incomplete description of the functions of the above modules, please refer to the contents of the automatic positioning parking method of the present invention.

[0093] like Figure 4 As shown, the present invention also provides an electronic device, including:

[0094] At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned parking positioning method based on image key frames or automatic positioning parking method.

[0095] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the parking positioning method based on image key frames or the automatic positioning parking method is implemented.

[0096] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. A computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0097] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0098] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0100] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A parking positioning method based on image key frames, characterized in that: include: Trigger parking location track recording based on GPS positioning and high-precision map, and determine the world coordinate origin of the parking location track based on GPS positioning information; Acquire a driving video image, perform inter-frame pose estimation based on the driving video image, save a key frame image descriptor set, and perform parking positioning trajectory recording based on the inter-frame pose estimation; the key frame image descriptor set is a set of descriptors of multiple key frame images, the descriptors are used to describe features in an image, and the image is expressed by the distribution features of pixel points in the image, and the key frame is an image frame containing more image information in the driving video image; Constructing 3D map points according to the driving video image, and recording 2D key point descriptors corresponding to the 3D map points; The key frame image descriptor set, the parking positioning trajectory, the 3D map point and the 2D key point descriptor are stored as positioning data for assisting parking; the 2D key point descriptor is used to describe key pixels of a video image.

2. The parking positioning method based on image key frames according to claim 1, characterized in that: The parking location trajectory recording is triggered according to GPS location and high-precision map, including: According to the high-precision map, it is determined that the current location is a parking area of ​​a building, and the signal strength of the GPS positioning is detected. When the signal strength of the GPS positioning is lower than a preset value, the parking positioning trajectory recording is triggered.

3. The parking positioning method based on image key frames according to claim 2, characterized in that: The triggering of parking location track recording according to GPS location and high-precision map, and determining the world coordinate origin of the parking location track according to GPS location information, includes: The IMU information of the vehicle is obtained, and the signal strength of the GPS positioning is detected when the upward acceleration in the IMU information disappears. When the signal strength of the GPS positioning is lower than a preset value, the coordinates of the position where the signal strength of the GPS positioning is lower than the preset value are used as the world coordinate origin of the parking positioning trajectory.

4. The parking positioning method based on image key frames according to claim 1, characterized in that: The step of constructing 3D map points according to the driving video image comprises: Visual SLAM is used to construct 3D map points according to the driving video image, and the 3D map points are optimized using a bundle adjustment method.

5. An automatic positioning parking method based on the parking positioning method based on image key frames according to any one of claims 1 to 4, characterized in that: include: Trigger automatic parking based on GPS positioning and high-precision maps, and determine the world coordinate origin of the parking positioning trajectory based on GPS positioning information; Record the driving trajectory, and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description subset; If the current frame video image matches a key frame image descriptor in the key frame image descriptor set, automatic parking is triggered; During the automatic parking process, the current video image is matched with the positioning data used for assisted parking, and the vehicle is controlled to park in the parking space according to the parking positioning trajectory in the positioning data for assisted parking.

6. The automatic positioning parking method according to claim 5, characterized in that: The automatic parking process includes matching the current video image with the positioning data for assisting parking, including: Obtain target key points of the current video image, and perform pixel matching between the target key points and 2D key point descriptors in the positioning data of the auxiliary parking; After the matching is successful, the 3D map point corresponding to the 2D key point descriptor is assigned to the target key point of the current video image.

7. The automatic positioning parking method according to claim 6, characterized in that: The controlling the vehicle to park in the parking space according to the parking positioning track in the positioning data of the auxiliary parking includes: The current vehicle's posture is restored to be consistent with the inter-frame posture in the parking positioning trajectory according to the PnP posture solution algorithm, and the vehicle is parked in the parking space according to the parking positioning trajectory.

8. The automatic positioning parking method according to claim 6, characterized in that: The pixel matching of the target key point with the 2D key point descriptor in the positioning data of the auxiliary parking includes: using a bundle adjustment method to pixel match the target key point with the 2D key point descriptor in the positioning data of the auxiliary parking.

9. An automatic positioning parking device, used to execute the automatic positioning parking method according to any one of claims 5 to 8, characterized in that: include: The positioning module is used to trigger automatic positioning parking according to GPS positioning and high-precision maps, and determine the world coordinate origin of the parking positioning trajectory according to the GPS positioning information; A matching module is used to record the driving trajectory and perform image retrieval on the current frame video image in the driving trajectory in the key frame image description subset; A triggering module, configured to trigger automatic parking if a current frame video image matches a key frame image descriptor in the key frame image descriptor set; The parking module is used to match the current video image with the positioning data for assisted parking during the automatic parking process, and control the vehicle to park in the parking space according to the parking positioning trajectory in the positioning data for assisted parking.

10. An electronic device, characterized in that: include: at least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the parking positioning method according to any one of claims 1 to 4 or the automatic positioning parking method according to any one of claims 5 to 8.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the parking positioning method according to any one of claims 1 to 4 or the automatic positioning parking method according to any one of claims 5 to 8 is executed.