Visual repositioning method and system and storage medium
Image features are extracted through deep learning models and pixel-level feature detection algorithms, map models are constructed and pose recovery is carried out, and combined with high-precision positioning pose constraint model, the accuracy problem of traditional visual repositioning methods under the influence of environmental factors is solved, and the accuracy problem of traditional visual repositioning methods is realized. Higher pose recovery accuracy and success rate.
Patent Information
- Application Number
- CN202410990911.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-26
AI Technical Summary
The traditional visual repositioning method cannot achieve accurate pose recovery under the low semantic map accuracy and the characteristic map is susceptible to environmental factors, and cannot meet the functions of memory parking and valet parking.
The deep learning model and pixel-level feature detection algorithm are used to extract image features, build a map model, and pose recovery is performed through the matching of image features of candidate frames, and the relocation success rate is improved by combining the high-precision positioning pose constraint model.
It improves the accuracy and robustness of pose recovery, enhances the success rate of visual repositioning, and solves the mismatch problems caused by scene degradation and lighting changes.
Smart Images

Figure CN119027500A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer vision technology, and specifically relates to a visual repositioning method, system and storage medium. Background Art
[0002] With the development of science and technology, the technology used for automatic parking of automobiles has developed rapidly. Computer vision technology is often used in the process of automatic parking of automobiles. Through image retrieval, the vehicle's posture is restored according to image features and visual relocation is performed. However, in the traditional relocation method, the constructed prior map includes semantic maps and feature maps. Among them, the semantic map has low accuracy and cannot achieve accurate posture recovery. The feature map has high accuracy but is easily affected by factors such as environmental visual scene degradation and lighting changes, resulting in the position and posture recovery accuracy cannot meet the requirements of memory parking and valet parking functions. Summary of the invention
[0003] In order to solve the above technical problems, this application proposes a visual relocalization method, system and storage medium. According to the data features extracted from the acquired image information by the deep learning model and the constructed map model, candidate frame image features are selected, and the vehicle posture is restored according to the obtained candidate frame image features and the relocalization posture information is output.
[0004] Specifically, the present application proposes a visual relocalization method, comprising: S1: Obtain image information of the vehicle's environment at the current moment; S2: extracting features from the image information to obtain pixel-level feature points, local features, and global features; wherein the local features and global features are extracted based on a deep learning model, and the pixel-level feature points are extracted based on a pixel-level feature detection algorithm; S3: Matching the pixel-level feature points, local features and global features according to the map model, and selecting candidate frame image features; S4: Recovering the posture of the vehicle in the map model according to the candidate frame image features, and outputting visual relocation posture information.
[0005] This application uses a deep learning model to extract features from the acquired image information, solves the mismatch problem of visual relocalization caused by scene degradation, changes in lighting perspective and other factors, makes the selection of candidate frames robust, and improves the accuracy of posture recovery based on the extraction of deep learning models and pixel-level feature detection algorithms.
[0006] The deep learning model is one of a neural network model, a convolutional neural network model or a recurrent neural network model.
[0007] Furthermore, the map model is specifically: Obtain image information of the vehicle's environment at historical moments; Extract features of the image information of the historical moment according to the deep learning model to obtain local features and global features of the historical moment; Extracting features from the image information of the historical moment according to a pixel-level feature detection algorithm to obtain pixel-level feature points of the historical moment; The map model is constructed according to the local features, global features and pixel-level feature points of the historical moment.
[0008] Feature extraction of image information at historical moments is performed based on the deep learning model and pixel-level feature detection algorithm, which improves the accuracy of posture recovery.
[0009] Furthermore, before constructing the map model according to the local features, global features and pixel-level feature points of the historical moment, the method further includes: The global features and local features of the historical moment are subjected to dimensionality reduction processing, and then the feature vectors in the global features and local features after the dimensionality reduction processing are rounded, compressed and stored.
[0010] Through the method of dimensionality reduction processing and integer compression storage, the problem of time-consuming traditional feature extraction scheme and large memory occupation of the generated map model is solved, and the memory of the map model is optimized.
[0011] Furthermore, the dimension reduction process of the global features and local features of the historical moments is specifically as follows: Constructing a feature matrix according to the global features and local features of the historical moment; Performing zero mean processing on the feature matrix; Solve the covariance matrix corresponding to the feature matrix after zero mean processing; Obtaining eigenvalues and eigenvectors of the covariance matrix, and constructing a dimensionality reduction matrix according to the eigenvalues and eigenvectors; The global features and local features are reduced in dimension according to the dimension reduction matrix.
[0012] Furthermore, the selecting of candidate frame image features includes: Performing posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model; Calculate and obtain the pose error and maximum similarity score between the current frame image features and the candidate frame image features; Determine whether the pose error is within the preset pose error threshold, whether the maximum similarity score is greater than the preset similarity score threshold, and whether the candidate frame image features selected N times are continuous. If all are satisfied, the selection of the candidate frame image features is successful, otherwise the selection of the candidate frame image features fails.
[0013] The pose constraints on the current frame image features and the candidate frame image features are performed according to the high-precision positioning pose constraint model, specifically: The high-precision positioning posture constraint model includes absolute position and absolute posture information; The relocation ranges of the current frame image features and the candidate frame image features are constrained according to the absolute position and absolute posture information.
[0014] The relocation range of the current frame image features and the candidate frame image features is constrained according to the high-precision positioning pose constraint model, which reduces the relocation range in the map scene and improves the relocation success rate.
[0015] Furthermore, the step S4 further includes: Whether the posture restoration is successful is judged according to the number of feature points after removing abnormal points after posture optimization. When the number of matching points between the current frame image features and the candidate frame image features is greater than the matching point number threshold, the posture restoration is successful.
[0016] Based on the same inventive concept, the present application also proposes a system for a visual repositioning method, the system comprising: Data acquisition module: used to obtain image information of the vehicle's environment at the current moment; Feature extraction module: used to extract features from the image information to obtain pixel-level feature points, local features and global features; wherein the local features and global features are extracted based on the deep learning model, and the pixel-level feature points are extracted based on the pixel-level feature detection algorithm; Candidate frame selection module: used to match the pixel-level feature points, local features and global features according to the map model, and select candidate frame image features; And, a posture recovery module: used to restore the posture of the vehicle in the map model according to the candidate frame image features, and output visual relocation posture information.
[0017] Furthermore, the system also includes a posture constraint module; The posture constraint module is used to perform posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model.
[0018] Wherein, the posture constraint module includes a global navigation satellite system, an inertial measurement unit and a computing chip.
[0019] Based on the same inventive concept, the present application also proposes a computer-readable storage medium, wherein the executable instructions of the computer-readable storage medium implement the visual repositioning method when being executed by a control processor.
[0020] In summary, the present application performs feature extraction on the acquired image information through a deep learning model and a pixel-level feature detection algorithm, and selects candidate frame image features by matching the acquired feature data with a map model; restores the posture of the vehicle in the map model according to the candidate frame image features, and outputs visual relocation posture information.
[0021] Compared with the prior art, this application has at least the following beneficial effects: This application extracts features through a deep learning model and a pixel-level feature detection algorithm, solves the mismatch problem of visual relocalization caused by scene degradation, changes in lighting perspective, and other factors, and improves the accuracy of pose recovery, making the selection of candidate frames robust; constrains the image features of candidate frames according to the pose constraint model, thereby improving the success rate of visual relocalization. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a flow chart of the visual relocation method shown in an embodiment of the present application.
[0023] Figure 2 It is a flow chart of the process of selecting candidate frame image features shown in an embodiment of the present application.
[0024] Figure 3 It is a diagram of the visual relocalization system architecture shown in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. Embodiment 1:
[0026] Reference Figure 1 , is a flow chart of the visual repositioning method shown in an embodiment of the present application.
[0027] This application proposes a visual relocalization method, comprising: S1: Obtain image information of the vehicle's environment at the current moment; S2: extracting features from the image information to obtain pixel-level feature points, local features, and global features; wherein the local features and global features are extracted based on a deep learning model, and the pixel-level feature points are extracted based on a pixel-level feature detection algorithm; Preferably, the pixel-level feature detection algorithm may be an ORB feature point, a SIFT pixel-level feature or a SURT pixel-level feature detection algorithm. Taking the ORB feature point detection algorithm as an example, the pixel-level feature point is obtained by detecting the key points in the image information and calculating its descriptor. The pixel-level feature point is a point in image processing or computer vision that has significant features and can be accurately described by pixel-level information. For example, a corner point, an edge, or other significant image structure. The ORB feature point detection algorithm combines FAST corner point detection and BRIEF descriptor, and has real-time and robustness. A deep learning model is used to extract more advanced local features, and the local features are feature information that describes the image information or data information that is smaller and localized relative to the whole. For example, a single-frame image is divided into multiple blocks, and feature information is extracted from each block based on a deep learning model. Local features may include the position, local structure, and local appearance of an object in any block image, such as the window position of a building, the local structure of an object, and the local shape. Deep learning models can learn feature representations by training on large amounts of image information data, such as using autoencoders or convolutional neural networks for unsupervised or supervised training.
[0028] The global features of the entire image information are extracted using a deep learning model. The global features can represent the overall features of the image information or data information, as opposed to local features. The global features cover the entire data set or complete image information and reflect the overall properties and characteristics.
[0029] S3: Match the pixel-level feature points, local features and global features according to the map model, and select candidate frame image features; the candidate frame image features describe the image features of a specific frame selected from the entire video image sequence. The pixel-level feature points, local features and global features are matched with the map model to obtain a similarity score between any frame image information and the pixel-level feature points, local features and global features, and the image information whose similarity score meets a preset threshold is taken as a specific frame. The similarity score is a numerical evaluation score used to measure the similarity between two objects, such as text, images, etc. In different application scenarios, the similarity score can be calculated using different measurement methods and algorithms based on the specific requirements of the task and data characteristics.
[0030] S4: Recovering the posture of the vehicle in the map model according to the candidate frame image features, and outputting visual relocation posture information.
[0031] Preferably, the posture recovery of the vehicle can be realized by the PNP algorithm. According to the known vehicle posture information in the map model and the corresponding pixel coordinates of the vehicle posture information in the candidate frame image features, the traditional PNP algorithm is used to obtain the posture of the camera. According to the posture of the camera, the position and orientation of the vehicle in the map model are converted. Finally, the posture information of visual relocation is output, including position and direction. The position can be the three-dimensional coordinates of the vehicle in the world coordinate system, and the direction can be expressed in azimuth.
[0032] The deep learning model is one of a neural network model, a convolutional neural network model or a recurrent neural network model.
[0033] Preferably, the deep learning model can be a Netvlad or SFRS model, and the global features of the single-frame image data are extracted according to the deep learning model to obtain the global features of the historical moment; the single-frame image is divided into multiple blocks, and the local features of the single-frame image are generated for each block based on the deep learning model; wherein each block can be defined by a rectangular area, which can be a square or rectangle of a fixed size, or an area of different sizes and shapes. When performing block segmentation, overlapping or non-overlapping methods can be selected. Overlapping block segmentation can improve the richness and robustness of features.
[0034] Furthermore, the map model is specifically: Obtain image information of the vehicle's environment at historical moments; Extract features of the image information of the historical moment according to the deep learning model to obtain local features and global features of the historical moment; Extracting features from the image information of the historical moment according to a pixel-level feature detection algorithm to obtain pixel-level feature points of the historical moment; The map model is constructed according to the local features, global features and pixel-level feature points of the historical moment.
[0035] Feature extraction of image information at historical moments is performed based on the deep learning model and pixel-level feature detection algorithm, which improves the accuracy of posture recovery.
[0036] Preferably, the constructed map model includes pixel-level feature points, local features and global features. The global features and local features are extracted by a deep learning model, thereby solving the problem in which the feature maps in traditional feature maps are highly accurate but easily affected by environmental factors, resulting in the position and posture recovery accuracy failing to meet the functional requirements of memory parking HPP or valet parking AVP.
[0037] Preferably, the mapping function is turned on when the vehicle reaches the entrance of the parking lot, and the vehicle drives to a parking space at a speed of less than 20 km / h. In this process, the vehicle obtains image information of the surrounding environment through the sensors it carries, such as a camera or a lidar, and extracts feature data from the image information based on a pixel-level feature detection algorithm and a deep learning model, and constructs a map model based on the obtained pixel-level feature points, local features, and global features; once the vehicle reaches a parking space, the mapping is completed and the map information is stored for subsequent relocation.
[0038] When the vehicle returns to the mapped driving path again, it performs candidate frame selection, feature matching, and pose recovery to finally obtain the precise pose in the local reference coordinate system. When the vehicle needs to be relocated, the system selects appropriate candidate frame image features based on the previously stored map information, and matches the features of the current position to restore the precise pose information of the vehicle.
[0039] Furthermore, before constructing the map model according to the local features, global features and pixel-level feature points of the historical moment, the method further includes: The global features and local features of the historical moment are subjected to dimensionality reduction processing, and then the feature vectors in the global features and local features after the dimensionality reduction processing are rounded, compressed and stored.
[0040] Through the method of dimensionality reduction processing and integer compression storage, the problem of time-consuming traditional feature extraction scheme and large memory occupation of the generated map model is solved, and the memory of the map model is optimized.
[0041] Furthermore, the dimension reduction process of the global features and local features of the historical moments is specifically as follows: Constructing a feature matrix according to the global features and local features of the historical moment; Performing zero mean processing on the feature matrix; Solve the covariance matrix corresponding to the feature matrix after zero mean processing; Obtaining eigenvalues and eigenvectors of the covariance matrix, and constructing a dimensionality reduction matrix according to the eigenvalues and eigenvectors; The global features and local features are reduced in dimension according to the dimension reduction matrix.
[0042] Preferably, the global features and local features can be reduced in dimension by principal component analysis; a feature matrix is constructed according to the global features and local features of the historical moment; wherein the feature matrix is a matrix of n rows and m columns, wherein n is the dimension of the feature vector and m is the number of the feature vectors; The constructed feature matrix is processed with zero mean and its covariance matrix is solved; Solve the eigenvalues and eigenvectors of the covariance matrix and sort them according to the size of the eigenvalues, take its k rows to construct a dimensionality reduction matrix; finally, realize global and local feature dimensionality reduction based on the dimensionality reduction matrix.
[0043] Preferably, the feature vectors in the global features and local features after dimensionality reduction are rounded and compressed for storage, which can be: the feature vectors of the global features and local features after dimensionality reduction according to the principal component analysis method are floating-point k-dimensional vectors. Storing one image feature alone requires 8*k bytes, which has too large a memory requirement. In order to reduce memory, the k-dimensional feature vectors are rounded and compressed for storage, such as "10110101010", similar to binary data, which can optimize the memory to k / 8 bytes.
[0044] Furthermore, the selecting of candidate frame image features includes: Performing posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model; Calculate and obtain the pose error and maximum similarity score between the current frame image features and the candidate frame image features; Determine whether the pose error is within the preset pose error threshold, whether the maximum similarity score is greater than the preset similarity score threshold, and whether the candidate frame image features selected N times are continuous. If all of them are satisfied, the selection of the candidate frame image features is successful, otherwise the selection of the candidate frame image features fails.
[0045] Reference Figure 2 , is a flow chart of the process of selecting candidate frame image features shown in an embodiment of the present application.
[0046] Preferably, the pixel-level feature points, local features and global features are matched according to the map model to select candidate frame image features; whether the state of the high-precision positioning pose constraint model is within the range of 1 to 4 is judged according to the current frame image features and the candidate frame image features to perform pose constraints on the current frame image features and the candidate frame image features; Among them, the status of the high-precision positioning posture constraint model includes: 0 means invalid; 1 means that the confidence of the high-precision positioning posture constraint model is 5m; 2 means that the RTK confidence of the high-precision positioning posture constraint model is 10cm; 3 means that the RTK / INS confidence of the high-precision positioning posture constraint model is 5cm; 4 means that the confidence of the high-precision positioning posture constraint model is unreliable.
[0047] Preferably, calculations are performed based on the current frame image features and candidate frame image features after pose constraints to obtain pose errors and maximum similarity scores. When the pose error is within a preset pose error threshold and the maximum similarity score is greater than a preset similarity score threshold, and the candidate frame image features are selected three times in a row, it is determined that the candidate frame image features have been successfully selected.
[0048] The pose constraints on the current frame image features and the candidate frame image features are performed according to the high-precision positioning pose constraint model, specifically: The high-precision positioning posture constraint model includes absolute position and absolute posture information; The relocation ranges of the current frame image features and the candidate frame image features are constrained according to the absolute position and absolute posture information.
[0049] The relocation range of the current frame image features and the candidate frame image features is constrained according to the high-precision positioning pose constraint model, which reduces the relocation range in the map scene and improves the relocation success rate.
[0050] In step S4, whether the posture restoration is successful is judged based on the number of feature points after removing abnormal points after the posture restoration. When the number of matching points between the current frame image features and the candidate frame image features is greater than the matching point number threshold, the posture restoration is successful.
[0051] Preferably, the vehicle position is relocated by combining the PNP algorithm, and the vehicle posture is calculated by the PNP algorithm; the two-dimensional coordinates of the feature points matched in the current frame in the image, the three-dimensional coordinates of these feature points in the map model, and the camera internal parameter matrix are obtained; since there may be mismatches in feature point matching, the use of the RANSAC algorithm can improve the robustness of the PNP estimation. RANSAC will randomly sample a portion of the matching points for PNP calculation, and through iterative selection, make more points conform to the model results, thereby eliminating the impact of mismatches.
[0052] After the preliminary pose estimation is completed, the BA optimization method can be further performed to fine-tune the pose estimation results to reduce the projection error and improve the estimation accuracy.
[0053] Preferably, according to the pose recovery algorithm, the estimated camera pose information is obtained; the feature points of the current frame and the candidate frame are obtained; the feature points of the current frame are matched with the feature points of the candidate frame to obtain matching point pairs; an outlier removal method, such as the RANSAC algorithm, is used to remove outliers in the matching point pairs; and it is determined whether the number of matching points after removing the outliers is greater than the matching point number threshold. If the number of matching points is greater than the threshold, the pose recovery is deemed to be successful, and the estimated pose information is output; otherwise, a prompt message indicating that the relocalization failed is output.
[0054] By combining pixel-level feature detection algorithms and deep learning models to extract global features, local features, and pixel-level feature points, the visual repositioning method proposed in this application can significantly improve the positioning accuracy and stability of vehicles in complex environments. At the same time, through the construction of the map model, the visual repositioning system of this application has high real-time and robustness. It is not only suitable for the field of autonomous driving, but can also be extended to other scenarios that require precise positioning, such as robot navigation and drone positioning. This application realizes multi-level and multi-dimensional fusion of feature extraction, providing more comprehensive and accurate information support for visual repositioning. Embodiment 2:
[0055] Reference Figure 3 , is a diagram of the visual relocalization system architecture shown in an embodiment of the present application.
[0056] The present application also proposes a system for a visual repositioning method, the system comprising: Data acquisition module: used to obtain image information of the vehicle's environment at the current moment; Feature extraction module: used to extract features from the image information to obtain pixel-level feature points, local features and global features; wherein the local features and global features are extracted based on the deep learning model, and the pixel-level feature points are extracted based on the pixel-level feature detection algorithm; Candidate frame selection module: used to match the pixel-level feature points, local features and global features according to the map model, and select candidate frame image features; And, a posture recovery module: used to restore the posture of the vehicle in the map model according to the candidate frame image features, and output visual relocation posture information.
[0057] Furthermore, the system also includes a posture constraint module; The posture constraint module is used to perform posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model.
[0058] The posture constraint module includes a global navigation satellite system, an inertial measurement unit and a computing chip.
[0059] The repositioning range in the map model is reduced according to the absolute position of the global navigation satellite system and the absolute attitude information of the inertial measurement unit. The computing chip is the core part of the posture constraint module, which is responsible for receiving and processing data from the global navigation satellite system and the inertial measurement unit, and performing a sensor fusion algorithm to estimate the optimal position and attitude. The fusion algorithm can optimize and integrate the data of the global navigation satellite system and the inertial measurement unit according to a specific strategy, such as a Kalman filter or a particle filter, to provide a more accurate and robust positioning result.
[0060] Preferably, local features, global features and pixel-level feature points extracted by deep learning models and pixel-level feature detection algorithms are matched with candidate frame image features selected by the map model to restore the posture of the vehicle in the map model; deep learning scene recognition is combined with the traditional PNP algorithm to achieve vehicle position relocation, image retrieval based on the deep learning model, relocation position constraints, candidate frame determination, feature matching, and posture recovery; mismatching of visual relocation is resolved; for example: abnormal candidate frame selection and posture recovery accuracy in scenarios such as visual scene degradation and lighting changes; when the relocation accuracy is insufficient or even wrong, HPP cruise will produce abnormalities, such as scratches, collisions, and inability to cruise to the corresponding end point. Embodiment three:
[0061] The present application also proposes a computer-readable storage medium, the computer-readable storage medium comprising: The computer-readable storage medium stores computer-executable instructions; The computer executable instructions implement the visual relocalization method when being executed by a control processor.
[0062] In the computer-readable storage medium, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.
[0063] In summary, this application uses a deep learning model and a pixel-level feature detection algorithm to extract features from the acquired image information, and matches the acquired feature data with the map model to select candidate frame image features; based on the candidate frame image features, the vehicle in the map model is restored to its posture, and visual relocation posture information is output. This solves the problem of mismatching in visual relocation due to factors such as scene degradation and changes in lighting perspective, and improves the accuracy of posture recovery, making the selection of candidate frames robust; the candidate frame image features are constrained according to the posture constraint model, which improves the success rate of visual relocation.
[0064] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
[0065] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for an electronic device to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0066] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.
Claims
1. A visual relocalization method, characterized in that: include: S1: Obtain image information of the vehicle's environment at the current moment; S2: extracting features from the image information to obtain pixel-level feature points, local features, and global features; wherein the local features and global features are extracted based on a deep learning model, and the pixel-level feature points are extracted based on a pixel-level feature detection algorithm; S3: Matching the pixel-level feature points, local features and global features according to the map model, and selecting candidate frame image features; S4: Recovering the posture of the vehicle in the map model according to the candidate frame image features, and outputting visual relocation posture information.
2. The visual relocation method according to claim 1, characterized in that: The method for constructing the map model comprises: Obtain image information of the vehicle's environment at historical moments; Extract features of the image information of the historical moment according to the deep learning model to obtain local features and global features of the historical moment; Extracting features from the image information of the historical moment according to a pixel-level feature detection algorithm to obtain pixel-level feature points of the historical moment; The map model is constructed according to the local features, global features and pixel-level feature points of the historical moment.
3. The visual relocation method according to claim 2, characterized in that: Before constructing the map model according to the local features, global features and pixel-level feature points at the historical moment, the method further includes: The global features and local features of the historical moment are subjected to dimensionality reduction processing, and then the feature vectors in the global features and local features after the dimensionality reduction processing are rounded, compressed and stored.
4. The visual relocation method according to claim 3, characterized in that: The dimension reduction process of the global features and local features of the historical moment is specifically as follows: Constructing a feature matrix according to the global features and local features of the historical moment; Performing zero mean processing on the feature matrix; Solve the covariance matrix corresponding to the feature matrix after zero mean processing; Obtaining eigenvalues and eigenvectors of the covariance matrix, and constructing a dimensionality reduction matrix according to the eigenvalues and eigenvectors; The global features and local features are reduced in dimension according to the dimension reduction matrix.
5. The visual relocation method according to claim 4, characterized in that: The selecting of candidate frame image features comprises: Performing posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model; Calculate and obtain the pose error and maximum similarity score between the current frame image features and the candidate frame image features; Determine whether the pose error is within the preset pose error threshold, whether the maximum similarity score is greater than the preset similarity score threshold, and whether the candidate frame image features selected N times are continuous. If all of them are satisfied, the selection of the candidate frame image features is successful, otherwise the selection of the candidate frame image features fails.
6. The visual relocation method according to claim 5, characterized in that: The pose constraints on the current frame image features and the candidate frame image features are performed according to the high-precision positioning pose constraint model, specifically: The high-precision positioning posture constraint model includes absolute position and absolute posture information; The relocation ranges of the current frame image features and the candidate frame image features are constrained according to the absolute position and absolute posture information.
7. The visual relocation method according to claim 6, characterized in that: The deep learning model is one of a neural network model, a convolutional neural network model or a recurrent neural network model.
8. A system according to any one of claims 1 to 7, the system comprising: Data acquisition module: used to obtain image information of the vehicle's environment at the current moment; Feature extraction module: used to extract features from the image information to obtain pixel-level feature points, local features and global features; wherein the local features and global features are extracted based on the deep learning model, and the pixel-level feature points are extracted based on the pixel-level feature detection algorithm; Candidate frame selection module: used to match the pixel-level feature points, local features and global features according to the map model, and select candidate frame image features; And, a posture recovery module: used to restore the posture of the vehicle in the map model according to the candidate frame image features, and output visual relocation posture information.
9. The system of the visual relocalization method according to claim 8, characterized in that: The system further comprises: The posture constraint module is used to perform posture constraints on the current frame image features and the candidate frame image features according to the high-precision positioning posture constraint model.
10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by the control processor, the visual repositioning method as described in any one of claims 1-7 is implemented.
Citation Information
Cited By
Vision and satellite navigation fused unmanned aerial vehicle inspection accurate positioning method and device
CN121594854A
