Visual positioning method, device and computer equipment

Abstract

Embodiments of the present invention disclose a visual positioning method, apparatus, and computer equipment. The method includes: acquiring a camera image; acquiring feature points corresponding to the image, and determining an initial sampling template corresponding to the feature points, the initial sampling template including a first preset number of pixel pairs; determining normalized coordinates and pixel centroid coordinates corresponding to the feature points; determining an error angle based on the normalized coordinates and pixel centroid coordinates; adjusting the initial sampling template based on the error angle to obtain a corrected sampling template, and determining a descriptor corresponding to the feature points based on the corrected sampling template; and matching the feature points corresponding to any other frame of camera image based on the descriptors to determine the camera's pose. This method can reduce changes in feature point descriptors caused by camera distortion, improve the accuracy of feature point matching, obtain more accurate feature point descriptors, and accurately match the corresponding feature points with the descriptors to determine the camera's pose, thereby improving the accuracy and robustness of the positioning of the entire system.

Description

Technical Field

[0001] The present invention relates to the field of computer software, and in particular to a visual positioning method, device and computer equipment. Background Art

[0002] Visual simultaneous localization and mapping (VSLAM) systems use cameras to capture images of the environment and, based on the relative changes in pixel or regional features at different times, infer the camera's motion within the environment, calculating the relative pose between images. Compared to conventional cameras, fisheye cameras offer a wider field of view, capturing more information about the environment and improving the robustness of VSLAM algorithms.

[0003] However, images captured by fisheye cameras are significantly distorted, and feature-based VSLAM algorithms typically use feature point descriptors for feature point matching. Due to distortion, the same feature point will produce different descriptors at different imaging locations, affecting the accuracy of feature point matching. This, in turn, can severely impact the positioning accuracy and robustness of the entire VSLAM system. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present application provides a visual positioning method, device and computer equipment to accurately match descriptors for image feature points acquired by a monocular fisheye camera, thereby improving the accuracy and robustness of VSLAM system positioning.

[0005] In a first aspect, the present application provides a visual positioning method, the method comprising:

[0006] Get camera image;

[0007] Acquire feature points corresponding to the image, and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs;

[0008] Determine the normalized coordinates and pixel centroid coordinates corresponding to the feature points;

[0009] determining an error angle according to the normalized coordinates and the pixel centroid coordinates;

[0010] Adjusting the initial sampling template according to the error angle to obtain a modified sampling template;

[0011] Determining a descriptor corresponding to the feature point according to the modified sampling template;

[0012] The camera's position and posture are determined by matching the feature points corresponding to any other frame of camera image according to the descriptor.

[0013] Optionally, determining the initial sampling template corresponding to the feature point includes:

[0014] Randomly sampling on a pixel plane with the feature point as the center to obtain the first preset number of pixel pairs;

[0015] The first preset number of pixel pairs is used as an initial sampling template for the feature points.

[0016] Optionally, determining the normalized coordinates and pixel centroid coordinates corresponding to the feature point includes:

[0017] Back-projecting the pixel coordinates of the feature point onto a normalized plane of a preset camera coordinate system to obtain the normalized coordinates corresponding to the feature point on the normalized plane;

[0018] A second preset number of coordinate points are selected within a preset area with the normalized coordinates as the center, and the pixel centroid coordinates are determined according to the coordinate points.

[0019] Optionally, obtaining the normalized coordinates corresponding to the feature point on the normalized plane includes:

[0020] Determining a coordinate conversion relationship between the pixel coordinates of the feature point and the normalized coordinates;

[0021] The normalized coordinates are determined according to the camera intrinsic parameters, the camera distortion coefficient and the coordinate transformation relationship.

[0022] Optionally, determining the pixel centroid coordinates according to the coordinate point includes:

[0023] Projecting the coordinate point into a pixel coordinate system to obtain the pixel coordinates corresponding to the coordinate point;

[0024] The pixel centroid coordinates are determined according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.

[0025] Optionally, adjusting the initial sampling template according to the error angle to obtain a modified sampling template includes:

[0026] Back-projecting the initial sampling template onto a normalized plane of a preset camera coordinate system, and determining an initial projected sampling template corresponding to the initial sampling template on the normalized plane;

[0027] The initial projection sampling template is rotated clockwise around the origin by the error angle with the normalized coordinate as the origin to obtain the modified sampling template.

[0028] Optionally, determining the descriptor corresponding to the feature point according to the modified sampling template includes:

[0029] Acquire a first preset number of corrected sampling point pairs according to the corrected sampling template;

[0030] Projecting the corrected sampling point pair into a pixel coordinate system to determine the pixel value of each corrected sampling point in the corrected sampling point pair;

[0031] The descriptor corresponding to the feature point is determined according to the size relationship of the pixel values of the corrected sampling points in the corrected sampling point pair.

[0032] In a second aspect, the present application provides a visual positioning device, comprising:

[0033] An image acquisition module, used to acquire camera images;

[0034] a feature point determination module, configured to obtain feature points corresponding to the image and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs;

[0035] A coordinate determination module, used to determine the normalized coordinates and pixel centroid coordinates corresponding to the feature points;

[0036] an error angle determination module, configured to determine an error angle according to the normalized coordinates and the pixel centroid coordinates;

[0037] An adjustment module, configured to adjust the initial sampling template according to the error angle to obtain a modified sampling template;

[0038] a descriptor determination module, configured to determine a descriptor corresponding to the feature point according to the modified sampling template;

[0039] The posture determination module matches the feature points corresponding to any other frame of camera image according to the descriptor to determine the posture of the camera.

[0040] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program executes any one of the above-described visual positioning methods when the processor is running.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which executes the visual positioning method according to any one of the above items when running on a processor.

[0042] The present application provides a visual positioning method, apparatus, and computer equipment, the method comprising: obtaining feature points corresponding to a camera image and determining an initial sampling template corresponding to the feature points; determining normalized coordinates and pixel centroid coordinates corresponding to the feature points; determining an error angle based on the normalized coordinates and the pixel centroid coordinates; adjusting the initial sampling template based on the error angle to obtain a revised sampling template, determining a descriptor corresponding to the feature points based on the revised sampling template; and matching the feature points corresponding to any other frame of camera image based on the descriptor to determine the camera's pose. This method can reduce the variation in feature point descriptors caused by fisheye camera distortion, obtain more accurate feature point descriptors, improve the accuracy of feature point matching, accurately locate the camera's pose by matching feature points with descriptors, achieve accurate visual positioning, and thereby improve the accuracy and robustness of the entire VSLAM system's positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0044] Figure 1 A schematic diagram of a process of visual positioning method is shown;

[0045] Figure 2 A schematic diagram of a back-projection model of a visual positioning method is shown;

[0046] Figure 3 A schematic diagram of a projection model of a visual positioning method is shown;

[0047] Figure 4 A module block diagram of a visual positioning device is shown;

[0048] Figure 5 A structural diagram of a computer device is shown. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0050] The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the figures is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without inventive effort are intended to be within the scope of protection of the present invention.

[0051] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0052] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0053] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present invention pertain. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present invention.

[0054] Example 1

[0055] The present application embodiment provides a visual positioning method, such as Figure 1 The method for generating an image feature point descriptor comprises:

[0056] S101, acquiring a camera image;

[0057] Specifically, a camera's camera captures a photograph of the real environment to produce a camera image. Image processing software on a computer then uses these multiple frames for subsequent computational processing. In practical applications, images captured by a monocular fisheye camera are distorted, and the same feature point will generate different descriptors at different imaging locations, affecting the accuracy of feature point matching.

[0058] S102, acquiring feature points corresponding to each frame image, and determining an initial sampling template corresponding to the feature points, the initial sampling template including a first preset number of pixel pairs;

[0059] Specifically, the feature points in each frame of the image are first extracted using a feature point extraction algorithm. Optional image feature point extraction algorithms include the FAST feature point extraction algorithm, the Harris corner detection method, the SIFT algorithm, the SURF algorithm, and the like. In order to improve the running speed of the algorithm, the FAST feature point extraction algorithm is preferred in this embodiment. Feature points refer to points where the grayscale value of an image changes dramatically or points with a larger curvature on the edge of an image (i.e., the intersection of two edges). The feature points of an image can reflect the essential features of the image and can identify target objects in the image. The purpose of feature point extraction is to match the same feature points between different images in order to solve the positional relationship between different images. With the feature point as the center, a first preset number of pixel pairs are sampled around the feature point to obtain the first preset number of pixel pairs. The first preset number of pixel pairs is the initial sampling template for the feature point, and the pixel pair is a feature descriptor used to describe the feature point.

[0060] S103, obtaining normalized coordinates and pixel centroid coordinates corresponding to the feature points;

[0061] Specifically, the normalized coordinates are the coordinates of the feature point on the normalized plane, and the pixel barycentric coordinates are the barycentric coordinates of the sampling point of the feature point on the normalized plane. The feature point in the pixel coordinate system is projected onto the normalized coordinate system to obtain the normalized coordinates, and the pixel barycentric coordinates corresponding to the feature point are calculated to facilitate the subsequent calculation of the error angle.

[0062] S104, determining an error angle according to the normalized coordinates and the pixel centroid coordinates;

[0063] Specifically, the error angle is calculated based on the normalized coordinates of the feature point on the normalized plane and the pixel centroid coordinates of the feature point on the normalized plane, so as to facilitate subsequent correction based on the error angle.

[0064] S105: Adjust the initial sampling template according to the error angle to obtain a modified sampling template.

[0065] After the error angle is determined, the initial sampling template is adjusted by the error angle according to the error angle, and a corrected sampling template is obtained after the adjustment.

[0066] S106: Determine a descriptor corresponding to the feature point according to the modified sampling template.

[0067] The descriptor corresponding to the feature point is determined based on the obtained modified sampling template. This feature point descriptor is the accurate descriptor of the feature point in this embodiment. Specifically, the feature point descriptor is a simplified representation of the image, which only contains the most important information about the image and is sufficient to identify the image.

[0068] S107 , matching feature points corresponding to any other frame of camera image according to the descriptor to determine the camera's position and posture.

[0069] Specifically, the feature point descriptor does not change with the changes in various changing conditions. The descriptor describes the pixel information near the feature point, and feature points with similar appearance in the image have similar descriptors. For example, a camera shoots a picture of a house. The feature point in the initial frame shot by the camera is the upper right corner point A of the house. After the camera moves and rotates and shoots again, the descriptor of the upper right corner point A of the house in the next frame is similar to the descriptor of the upper right corner point A of the house in the initial frame. The similar descriptors can be used to locate the current camera shooting posture. Therefore, when the camera moves and rotates and the perspective changes, the feature point descriptors that remain unchanged can be used to match the same object in images from different perspectives, determine the camera's posture, and achieve accurate visual positioning.

[0070] Furthermore, the camera's position change, i.e., its movement and rotation, is calculated based on the camera's initial and current position. This position change then determines the camera's motion trajectory, and accurate mapping is achieved based on the camera's motion trajectory and feature points.

[0071] In practical applications, due to the distortion of monocular fisheye cameras, the same feature point will produce different descriptors at different imaging positions, thus affecting the accuracy of feature point matching. In this embodiment, after calculating the error angle by using the normalized coordinates of the feature point and the pixel centroid coordinates, the initial sampling template is adjusted by the angle of the error angle. The feature point descriptor corresponding to the corrected sampling template obtained after adjustment is the accurate descriptor of the feature point in this embodiment. This solves the change of feature point descriptors caused by the distortion of the fisheye camera, accurately matches descriptors for feature points, and obtains more accurate feature point descriptors, thereby improving the accuracy of feature point matching, achieving accurate visual positioning and mapping functions, and thus improving the accuracy and robustness of positioning of the entire VSLAM system.

[0072] In a specific embodiment, obtaining the initial sampling template corresponding to the feature point includes:

[0073] Randomly sampling on a pixel plane with the feature point as the center to obtain the first preset number of pixel pairs;

[0074] The first preset number of pixel pairs is used as an initial sampling template for the feature points.

[0075] Specifically, a Gaussian distribution is used to randomly sample the pixel plane where the feature point is located on the image. A certain number of pixels are randomly sampled with the feature point as the center, and then grouped into pairs to obtain a first preset number of pixel pairs, where the value of the first preset number includes but is not limited to 128, 256, and 512. This embodiment is described using 256 pairs of pixel pairs as an example. Random sampling is performed around the feature point with the feature point as the center to obtain 256 pairs of pixel pairs, where these 256 pairs of pixel pairs serve as the initial sampling template corresponding to the feature point. When another feature point needs to be calculated, it is compared with the pixels in the initial sampling template to obtain 256 pairs of pixel pairs surrounding the other feature point.

[0076] The number of pixel pairs corresponds to the number of bits in the descriptor. Each pair of pixels corresponds to a value, and the final generated descriptor is a 256-bit binary string.

[0077] In a specific embodiment, obtaining the normalized coordinates and pixel centroid coordinates corresponding to the feature point includes:

[0078] Back-projecting the pixel coordinates of the feature point onto a normalized plane of a preset camera coordinate system to obtain the normalized coordinates corresponding to the feature point on the normalized plane;

[0079] A second preset number of coordinate points are selected within a preset rectangular area with the normalized coordinates as the center, and the pixel centroid coordinates are obtained according to the coordinate points.

[0080] Specifically, if Figure 2 As shown in the figure, the pre-established camera coordinate system is a camera coordinate system established with the camera position corresponding to the first frame image as the origin, with the optical center of the camera as the origin, the Z axis coincides with the camera optical axis and is perpendicular to the image imaging plane, and the photographic direction is taken as the positive direction of the Z axis; the X and Y axes are parallel to the X and Y axes of the image coordinate axis.

[0081] The pixel coordinates of an image are established as follows: The image's pixel plane has many pixels. The origin is the upper left of each pixel. Pixel coordinates represent the pixel's position on the image. The u-axis is parallel to the positive direction of the image plane and points horizontally to the right. The v-axis is perpendicular to the u-axis and points downward. The unit of the pixel coordinate system is pixel. The normalized plane of the camera coordinate system is assumed to be a coordinate plane parallel to the XOY plane with a Z-axis value of 1.

[0082] Among them, the plane where the pixel coordinate system is located and the imaging plane are on the same plane, but their respective origins and measurement units are different.

[0083] In a specific embodiment, obtaining the normalized coordinates corresponding to the feature point on the normalized plane includes:

[0084] Determining a coordinate conversion relationship between the pixel coordinates and the normalized coordinates;

[0085] The normalized coordinates are determined according to the camera intrinsic parameters, the camera distortion coefficient and the coordinate transformation relationship.

[0086] For example, if the coordinates of feature point p are (u, v), the feature point p(u, v) is projected onto the normalized plane S1, with the coordinates P = (x, y, 1.0), and the calculation method is as follows:

[0087] Formula (1):

[0088] Among them, P d is the coordinate expression relationship of the pixel coordinate p(u, v) in the imaging plane according to the pinhole camera imaging model. d The coordinates on the imaging plane are (x d ,y d ).

[0089] u and v correspond to the elements in the uth column and vth row in the pixel coordinates. k1, k2, k3, k4 are the camera distortion coefficients, c x 、c y 、f x 、f y It is the camera's internal parameter. It is generally believed that the camera's internal parameter is fixed after leaving the factory and will not change during use.

[0090] Formula (2):

[0091] r is the projection length, r d is the projection length after distortion.

[0092] Formula (3):

[0093] After solving the expression of r using the Newton iteration method, substitute it into the above formula to obtain the normalized coordinates.

[0094] Specifically, a second preset number of coordinate points are selected within the preset area with the normalized coordinates as the center. For example, the preset area can be a rectangular area of any size centered on the feature point, and the second preset number can be the number of m×n coordinate points selected in any rectangular area. In this embodiment, on the normalized plane point, with P=(x, y, 1.0) as the center, a second preset number of 20*20 coordinate points are equidistantly taken within a rectangle with a preset area of 0.1×0.1.

[0095] In a specific embodiment, obtaining the pixel centroid coordinates according to the coordinate point includes:

[0096] Projecting the coordinate point into a pixel coordinate system to obtain the pixel coordinates corresponding to the coordinate point;

[0097] The pixel centroid coordinates are determined according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.

[0098] like Figure 3 As shown, any point P i Project to the camera pixel plane to obtain the pixel value, where the projection method is:

[0099] Formula (4):

[0100] Formula (5):

[0101] Among them, formula (5) corresponds to Figure 3 in Angle. Formula (4) and formula (5) combined with formula (2) yield:

[0102] Formula (6):

[0103] According to the above calculation process, the pixel values corresponding to the above 20×20 coordinate points after being projected onto the camera imaging plane are calculated. The coordinates of the center of gravity are obtained as I i is the pixel value corresponding to the coordinate point.

[0104] In a specific embodiment, adjusting the initial sampling template according to the error angle to obtain a modified sampling template includes:

[0105] Back-projecting the initial sampling template onto a normalized plane of a preset camera coordinate system, and determining an initial projected sampling template corresponding to the initial sampling template on the normalized plane;

[0106] The initial projection sampling template is rotated clockwise around the origin by the error angle with the normalized coordinate as the origin to obtain the modified sampling template.

[0107] Specifically, the error angle is calculated using the center of gravity C = (x c ,y c , 1.0) and the coordinates of P are calculated using the following formula:

[0108] Formula (7): θ=atan2(y c -y,x c -y)

[0109] As shown in Figure 2, D1 is the area corresponding to the normalized plane S1 before projection, and S0 is the area corresponding to the projection onto the pixel plane. During projection, the feature point P and the initial sampling template D0 area of the feature point are back-projected onto the normalized plane of the preset camera coordinate system to obtain the initial projection sampling template D1. The initial projection sampling template D1 is rotated clockwise around the origin by the error angle θ with the normalized coordinates of the feature point as the origin. The error angle θ is expressed as formula (7), and the corrected sampling template can be obtained, so that the descriptor can be calculated based on the corrected sampling template.

[0110] In a specific embodiment, obtaining the descriptor corresponding to the feature point according to the modified sampling template includes:

[0111] Acquire a first preset number of corrected sampling point pairs according to the corrected sampling template;

[0112] Projecting the corrected sampling point pair onto the pixel coordinate system to obtain the pixel value of each corrected sampling point in the corrected sampling point pair;

[0113] The descriptor corresponding to the feature point is determined according to the size relationship of the pixel values of the corrected sampling points in the corrected sampling point pair.

[0114] Specifically, if there are 256 pixel pairs in the initial projection sampling template D1, for any pair of points (p i , p j ), which is calculated as follows: i , p j Project it onto the pixel plane and get the corresponding pixel value. If p i The corresponding pixel value is greater than p j , the corresponding bit data is set to 1, otherwise it is set to 0. Each pixel pair gets a binary digit, and 256 pixel pairs get corresponding 256-bit binary data. The 256-bit binary data is the descriptor of the image feature point.

[0115] Example 2

[0116] The embodiment of the present application provides a visual positioning device 400, such as Figure 4 As shown, the visual positioning device 400 includes:

[0117] Image acquisition module 401, used to acquire camera images;

[0118] A feature point determination module 402 is configured to obtain feature points corresponding to each frame image and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs;

[0119] A coordinate determination module 403 is used to determine the normalized coordinates and pixel centroid coordinates corresponding to the feature points;

[0120] an error angle determination module 404, configured to determine an error angle based on the normalized coordinates and the pixel centroid coordinates;

[0121] An adjustment module 405 is configured to adjust the initial sampling template according to the error angle to obtain a modified sampling template;

[0122] a descriptor determination module 406, configured to determine a descriptor corresponding to the feature point according to the modified sampling template;

[0123] The posture determination module 407 matches the feature points corresponding to any other frame of camera image according to the descriptor to determine the posture of the camera.

[0124] Optionally, the feature point acquisition module 402 is further configured to randomly sample on a pixel plane with the feature point as the center to obtain the first preset number of pixel pairs; and use the first preset number of pixel pairs as an initial sampling template for the feature point.

[0125] The coordinate acquisition module 403 is used to back-project the pixel coordinates of the feature point onto the normalized plane of the preset camera coordinate system to obtain the normalized coordinates corresponding to the feature point on the normalized plane; select a second preset number of coordinate points in a preset area with the normalized coordinates as the center, and determine the pixel center of gravity coordinates based on the coordinate points.

[0126] The adjustment module 405 is further configured to back-project the initial sampling template onto a normalized plane of a preset camera coordinate system, determine an initial projected sampling template corresponding to the initial sampling template on the normalized plane, and rotate the initial projected sampling template clockwise around the origin by the error angle with the normalized coordinate as the origin to obtain the corrected sampling template.

[0127] The descriptor determination module 406 is further configured to obtain a first preset number of corrected sampling point pairs based on the corrected sampling template; project the corrected sampling point pairs into the pixel coordinate system to determine the pixel value of each corrected sampling point in the corrected sampling point pairs; and determine the descriptor corresponding to the feature point based on the magnitude relationship of the pixel values of each corrected sampling point in the corrected sampling point pairs.

[0128] For the specific implementation steps, please refer to the relevant description of the visual positioning method provided in the above embodiment 1. To avoid repetition, they are not described here.

[0129] The visual positioning device 400 of this embodiment can reduce the changes in feature point descriptors caused by the distortion of the fisheye camera, obtain more accurate feature point descriptors, improve the accuracy of feature point matching, accurately locate the camera's posture by matching feature points with descriptors, achieve accurate visual positioning, and thereby improve the accuracy and robustness of the entire VSLAM system positioning.

[0130] Example 3

[0131] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is run by the processor, the computer program executes the visual positioning method described in any one of the above-mentioned embodiments 1.

[0132] In addition, if Figure 5 An embodiment of the present disclosure provides a computer device 500 including a memory 501 and a processor 502 . The memory 501 stores a computer program. When the computer program runs on the processor 502 , the computer device 500 executes the visual positioning method described in the above method embodiment 1.

[0133] Processor 502, used to obtain camera images;

[0134] Acquire feature points corresponding to the image, and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs;

[0135] Determine the normalized coordinates and pixel centroid coordinates corresponding to the feature points;

[0136] determining an error angle according to the normalized coordinates and the pixel centroid coordinates;

[0137] Adjusting the initial sampling template according to the error angle to obtain a revised sampling template, and determining the descriptor corresponding to the feature point according to the revised sampling template;

[0138] The camera's position and posture are determined by matching the feature points corresponding to any other frame of camera image according to the descriptor.

[0139] The processor 502 is further configured to randomly sample on a pixel plane with the feature point as the center to obtain the first preset number of pixel pairs;

[0140] The first preset number of pixel pairs is used as an initial sampling template for the feature points.

[0141] The processor 502 is further configured to back-project the feature point onto a normalized plane of a preset camera coordinate system to obtain the normalized coordinates corresponding to the feature point on the normalized plane;

[0142] A second preset number of coordinate points are selected within a preset area with the normalized coordinates as the center, and the pixel centroid coordinates are determined according to the coordinate points.

[0143] The processor 502 is further configured to determine a coordinate conversion relationship between the pixel coordinates and the normalized coordinates;

[0144] The normalized coordinates are determined according to the camera intrinsic parameters, the camera distortion coefficient and the coordinate transformation relationship.

[0145] The processor 502 is further configured to project the coordinate point into a pixel coordinate system to obtain pixel coordinates corresponding to the coordinate point;

[0146] The pixel centroid coordinates are determined according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.

[0147] The processor 502 is further configured to back-project the initial sampling template onto a normalized plane of a preset camera coordinate system, and determine an initial projected sampling template corresponding to the initial sampling template on the normalized plane;

[0148] The initial projection sampling template is rotated clockwise around the origin by the error angle with the normalized coordinate as the origin to obtain the modified sampling template.

[0149] The processor 502 is further configured to obtain a first preset number of corrected sampling point pairs according to the corrected sampling template;

[0150] Projecting the corrected sampling point pair onto the pixel coordinate system to determine the pixel value of each corrected sampling point in the corrected sampling point pair;

[0151] The descriptor corresponding to the feature point is determined according to the size relationship of the pixel values of the corrected sampling points in the corrected sampling point pair.

[0152] For the specific implementation steps, reference may be made to the relevant description of the method for generating image feature point descriptors provided in the above-mentioned embodiment 1, which will not be described here in detail to avoid repetition.

[0153] The computer device provided in this embodiment can accurately obtain the descriptors of the image feature points of a monocular fisheye camera, reduce the changes in the feature point descriptors caused by the distortion of the fisheye camera, obtain more accurate feature point descriptors, improve the accuracy of feature point matching, and accurately locate the camera's posture by matching feature points with descriptors, thereby achieving accurate visual positioning, thereby improving the accuracy and robustness of the entire VSLAM system positioning.

[0154] Example 4

[0155] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is run on a processor, the computer program executes the visual positioning method according to any one of the above-mentioned embodiments 1.

[0156] For the specific implementation steps, please refer to the relevant description of the visual positioning method provided in the above embodiment 1. To avoid repetition, they are not described here.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0158] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0159] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0160] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A visual positioning method, characterized in that: The method comprises: Get camera image; Acquire feature points corresponding to the image, and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs; Determine the normalized coordinates and pixel centroid coordinates corresponding to the feature points; determining an error angle according to the normalized coordinates and the pixel centroid coordinates; Adjusting the initial sampling template according to the error angle to obtain a modified sampling template; Determining a descriptor corresponding to the feature point according to the modified sampling template; Match the feature points corresponding to any other frame of camera image according to the descriptor to determine the camera's position and posture; The determining of the normalized coordinates and pixel centroid coordinates corresponding to the feature points includes: Back-projecting the feature point onto a normalized plane of a preset camera coordinate system to obtain the normalized coordinates corresponding to the feature point on the normalized plane; Selecting a second preset number of coordinate points within a preset area with the normalized coordinates as the center, and determining the pixel centroid coordinates according to the coordinate points; The determining the pixel centroid coordinates according to the coordinate points includes: The coordinate point is projected into a pixel coordinate system to obtain pixel coordinates corresponding to the coordinate point, and the pixel centroid coordinates are obtained according to the pixel coordinates.

2. The method according to claim 1, characterized in that The determining of the initial sampling template corresponding to the feature point includes: Randomly sampling on a pixel plane with the feature point as the center to obtain the first preset number of pixel pairs; The first preset number of pixel pairs is used as an initial sampling template for the feature points.

3. The method according to claim 1, characterized in that Obtaining the normalized coordinates corresponding to the feature point on the normalized plane includes: Determining a coordinate conversion relationship between the pixel coordinates of the feature point and the normalized coordinates; The normalized coordinates are determined according to the camera intrinsic parameters, the camera distortion coefficient and the coordinate transformation relationship.

4. The method according to claim 1, wherein The obtaining the pixel centroid coordinates according to the pixel coordinates includes: The pixel centroid coordinates are determined according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.

5. The method according to claim 1, wherein The adjusting the initial sampling template according to the error angle to obtain a modified sampling template includes: Back-projecting the initial sampling template onto a normalized plane of a preset camera coordinate system, and determining an initial projected sampling template corresponding to the initial sampling template on the normalized plane; The initial projection sampling template is rotated clockwise around the origin by the error angle with the normalized coordinate as the origin to obtain the modified sampling template.

6. The method according to claim 1, characterized in that The determining the descriptor corresponding to the feature point according to the modified sampling template includes: Acquire a first preset number of corrected sampling point pairs according to the corrected sampling template; Projecting the corrected sampling point pair into a pixel coordinate system to determine the pixel value of each corrected sampling point in the corrected sampling point pair; The descriptor corresponding to the feature point is determined according to the size relationship of the pixel values of the corrected sampling points in the corrected sampling point pair.

7. A visual positioning device, characterized in that: The device comprises: An image acquisition module, used to acquire camera images; a feature point determination module, configured to obtain feature points corresponding to the image and determine an initial sampling template corresponding to the feature points, wherein the initial sampling template includes a first preset number of pixel pairs; A coordinate determination module, configured to determine normalized coordinates and pixel centroid coordinates corresponding to the feature points; wherein the feature points are back-projected onto a normalized plane of a preset camera coordinate system to obtain the normalized coordinates corresponding to the feature points on the normalized plane; a second preset number of coordinate points are selected within a preset area with the normalized coordinates as the center, and the pixel centroid coordinates are determined based on the coordinate points; wherein the coordinate points are projected onto a pixel coordinate system to obtain pixel coordinates corresponding to the coordinate points, and the pixel centroid coordinates are obtained based on the pixel coordinates; an error angle determination module, configured to determine an error angle according to the normalized coordinates and the pixel centroid coordinates; An adjustment module, configured to adjust the initial sampling template according to the error angle to obtain a modified sampling template; a descriptor determination module, configured to determine a descriptor corresponding to the feature point according to the modified sampling template; The posture determination module matches the feature points corresponding to any other frame of camera image according to the descriptor to determine the posture of the camera.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the visual positioning method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium, characterized in that The device stores a computer program, which executes the visual positioning method according to any one of claims 1 to 6 when the computer program is run on a processor.

Images