Visual navigation and positioning methods, devices, terminals, and computer-readable storage media

Abstract

This invention provides a visual navigation and positioning method, apparatus, terminal, and computer-readable storage medium. The visual navigation and positioning method includes: acquiring a current environmental image of a running device; responding to historical positioning information of the running device associated with the current environmental image, performing feature matching between the current environmental image and a preset image corresponding to the nearest preset location in the historical positioning information; and responding to the matching between the current environmental image and the preset image, determining the preset location corresponding to the preset image as the location information of the running device when the current environmental image was acquired. This application, based on the historical positioning information of the running device associated with the current environmental image, selects a preset image at the nearest preset location in the historical positioning information for feature matching, and performs a diffusion search using the historical positioning information, thereby achieving faster and more accurate visual positioning of the running device.

Description

Technical Field

[0001] This invention relates to the field of visual navigation technology, and in particular to a visual navigation positioning method, device, terminal, and computer-readable storage medium. Background Technology

[0002] Indoor navigation for ground mobile robots is mainly divided into active navigation such as laser navigation and visual navigation, and passive navigation using base station networking types such as WIFI (Wireless Fidelity), UWB (Ultra Wideband), and Bluetooth.

[0003] Among these, laser navigation, having been developed earlier, is more widely used. However, single-line lasers provide limited information and are highly dependent on the environment, often requiring the use of reflectors or reflective stickers in complex environments for navigation and positioning. Multi-line lasers generate large amounts of data, necessitating the processing of massive point cloud data and placing high demands on the robot platform's performance. Furthermore, laser sensors are more expensive than visual sensors. Navigation methods such as Wi-Fi, UWB, and Bluetooth have emerged briefly, offering lower costs and greater scalability, but they also have numerous limitations. In indoor applications, these methods are susceptible to obstruction by debris and signal interference; moreover, they perform poorly in scenarios requiring high precision. Visual navigation has seen rapid development and widespread application recently, with QR code navigation being a typical example, widely used in intelligent warehousing and logistics. However, QR codes are easily contaminated and damaged, and their installation can negatively impact the aesthetics of the site. Summary of the Invention

[0004] The main technical problem solved by this invention is to provide a visual navigation and positioning method, device, terminal and computer-readable storage medium, thereby solving the problem of low accuracy in visual navigation and positioning in the prior art.

[0005] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide a visual navigation and positioning method, the visual navigation and positioning method including acquiring the current environmental image of the operating device; in response to the historical positioning information of the operating device associated with the current environmental image, performing feature matching between the current environmental image and the preset image corresponding to the preset position closest to the historical positioning information; in response to the matching between the current environmental image and the preset image, determining the preset position corresponding to the preset image as the position information of the operating device when the current environmental image was acquired.

[0006] Among these steps, in response to the historical location information of the operating device associated with the current environmental image, feature matching is performed between the current environmental image and the preset image corresponding to the preset location closest to the historical location information, which includes:

[0007] The current environment image is split into multiple regions based on a quadtree structure algorithm, and texture information is extracted from the regions; the texture information includes detected feature points.

[0008] This includes feature matching between the current environmental image and the preset image corresponding to the nearest preset location in historical positioning information, including:

[0009] Using historical location information as the center, arrange all preset images corresponding to preset locations in order of increasing distance from the preset location to the center.

[0010] Select the top-ranked preset images as candidate matching images for the current environment image;

[0011] Perform feature matching between the current environment image and the candidate matching images.

[0012] This includes feature matching between the current environment image and the candidate matching image, including:

[0013] The matching feature point pairs corresponding to the detection feature point pairs determined based on the detection feature points of the current environment image are statistically analyzed and compared with the preset feature point pairs determined based on the preset feature points of the candidate matching image, so as to obtain the initial matching feature point pairs corresponding to the candidate matching image and the current environment image.

[0014] The initial matching feature points corresponding to the candidate matching image and the current environment image are filtered to obtain the true matching feature point pairs corresponding to the candidate matching image and the current environment image.

[0015] Based on the true matching feature point pairs of the candidate matching images, it is determined whether the current environment image and the candidate matching image are successfully matched.

[0016] The visual navigation and positioning methods also include:

[0017] In response to the fact that the current environmental image is not associated with the historical location information of the running device, feature matching is performed between the current environmental image and the preset images corresponding to all preset locations.

[0018] This includes performing feature matching between the current environment image and preset images corresponding to all preset locations, including:

[0019] The matching feature point pairs between the detection feature point pairs determined based on the detection feature points of the current environment image and the preset feature point pairs determined based on the preset feature points of the preset image are statistically analyzed to obtain the initial matching feature point pairs between the preset image and the current environment image.

[0020] The initial matching feature points between the preset image and the current environment image are filtered to obtain the true matching feature point pairs between the preset image and the current environment image;

[0021] Based on the initial matching feature point pairs and the actual matching feature point pairs of the preset image, the matching probability value of the preset image is determined.

[0022] Select the preset image corresponding to the maximum matching probability value as the candidate matching image of the current environment image;

[0023] Based on the true matching feature point pairs of the candidate matching images, it is determined whether the current environment image and the candidate matching image are successfully matched.

[0024] Specifically, in response to a match between the current environmental image and a preset image, the preset position corresponding to the preset image is determined as the position information of the running device when acquiring the current environmental image, including:

[0025] If the number of true matching feature point pairs in a candidate matching image exceeds a preset value, then the preset position corresponding to the candidate matching image is the position information of the running device when the current environmental image is acquired;

[0026] If the number of true matching feature point pairs in the candidate matching image does not exceed a preset value, then the preset position corresponding to the candidate matching image is not the position information of the running device when the current environment image was acquired.

[0027] The method of determining the preset location corresponding to the preset image as the location information of the running device when acquiring the current environmental image, in response to a match between the current environmental image and a preset image, also includes:

[0028] Based on the positional error value of the detected feature point pairs mapped to the preset feature point pairs between the real matching feature point pairs corresponding to the current environment image and the candidate matching image;

[0029] Based on the positional error values ​​and / or the number of true matching feature point pairs between the current environment image and the candidate matching image, determine the matching confidence between the current environment image and the candidate matching image.

[0030] If the matching confidence between the current environment image and the candidate matching image exceeds the confidence threshold, the preset position of the candidate matching image is determined to be the position information of the running device when the current environment image was acquired.

[0031] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide a visual navigation and positioning device, the visual navigation and positioning device comprising: an acquisition module for acquiring a current environmental image of the operating device; a matching module for performing feature matching between the current environmental image and a preset image corresponding to the preset position closest to the historical positioning information in response to the association of the current environmental image with the historical positioning information; and a determination module for determining the preset position corresponding to the preset image as the position information of the operating device when the current environmental image is acquired in response to the matching of the current environmental image and the preset image.

[0032] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor is used to execute program data to implement the steps in the above-mentioned visual navigation and positioning method.

[0033] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the above-mentioned visual navigation and positioning method.

[0034] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention provides a visual navigation and positioning method, apparatus, terminal, and computer-readable storage medium. The visual navigation and positioning method includes: acquiring a current environmental image of a running device; responding to historical positioning information of the running device associated with the current environmental image, performing feature matching between the current environmental image and a preset image corresponding to the nearest preset location in the historical positioning information; and responding to the matching between the current environmental image and the preset image, determining the preset location corresponding to the preset image as the location information of the running device when the current environmental image was acquired. This application, based on the historical positioning information of the running device associated with the current environmental image, selects a preset image at the nearest preset location in the historical positioning information for feature matching, and performs a diffusion search using the historical positioning information, thereby achieving faster and more accurate visual positioning of the running device. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating an embodiment of the visual navigation and positioning method provided by the present invention;

[0037] Figure 2 This is a flowchart illustrating a specific embodiment of the visual navigation and positioning method provided by the present invention;

[0038] Figure 3 yes Figure 2 A flowchart illustrating a specific embodiment of step S202 in the provided visual navigation and positioning method;

[0039] Figure 4 This is a schematic diagram of each preset position within the running scene in one embodiment;

[0040] Figure 5 yes Figure 2 A flowchart illustrating a specific embodiment of step S205 in the provided visual navigation and positioning method;

[0041] Figure 6 yes Figure 2 A flowchart illustrating a specific embodiment of step S206 in the provided visual navigation and positioning method;

[0042] Figure 7 yes Figure 2 A flowchart illustrating a specific embodiment of step S207 in the provided visual navigation and positioning method;

[0043] Figure 8 This is a schematic diagram of the framework of an embodiment of the visual navigation and positioning device provided by the present invention;

[0044] Figure 9 This is a schematic diagram of the framework of an embodiment of the terminal provided in this application;

[0045] Figure 10 A schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0046] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0047] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0048] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0049] To enable those skilled in the art to better understand the technical solution of the present invention, a visual navigation and positioning method provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the visual navigation and positioning method provided by the present invention. This embodiment provides a visual navigation and positioning method, which includes the following steps.

[0051] S11: Obtain the current environment image of the running device.

[0052] Specifically, an image acquisition device is fixed to the bottom of the operating device. The operating device, which requires navigation, acquires images of the current environment to obtain an image of the current environment. In another embodiment, the device for acquiring images at a preset location can be the operating device itself, with a camera installed on its bottom to capture environmental information at the preset location. The current environmental image acquired by the operating device is stored and processed. Alternatively, the acquired image can be sent to a server for data processing. The process terminates when no image is collected by the image acquisition device. In one specific embodiment, the environmental image can be a ground image of the operating environment. No navigation markers are set in the operating environment of the operating device.

[0053] S12: In response to the historical positioning information of the device associated with the current environmental image, perform feature matching between the current environmental image and the preset image corresponding to the preset location closest to the historical positioning information.

[0054] Specifically, the current environment image is split into multiple regions based on a quadtree structure algorithm, and texture information in the regions is extracted; the texture information includes detected feature points.

[0055] In one embodiment, using historical location information as the center position, all preset images corresponding to preset positions are arranged in order of increasing distance from the preset position to the center position; the preset image at the top of the sorting is selected as the candidate matching image for the current environment image; and feature matching is performed between the current environment image and the candidate matching image.

[0056] In one embodiment, the matching feature point pairs corresponding to the detection feature point pairs determined based on the detection feature points of the current environment image and the preset feature point pairs determined based on the preset feature points of the candidate matching image are statistically analyzed to obtain the initial matching feature point pairs corresponding to the candidate matching image and the current environment image; the initial matching feature points corresponding to the candidate matching image and the current environment image are filtered to obtain the true matching feature point pairs corresponding to the candidate matching image and the current environment image; based on the true matching feature point pairs of the candidate matching image, it is determined whether the current environment image and the candidate matching image are successfully matched.

[0057] In another embodiment, in response to the fact that the current environmental image is not associated with the historical location information of the operating device, feature matching is performed between the current environmental image and the preset images corresponding to all preset locations.

[0058] In one specific embodiment, matching feature point pairs are obtained between the detection feature point pairs determined based on the detection feature points of the current environment image and the preset feature point pairs determined based on the preset feature points of the preset image, thus obtaining initial matching feature point pairs between the preset image and the current environment image. The initial matching feature points between the preset image and the current environment image are then filtered to obtain true matching feature point pairs between the preset image and the current environment image. Based on the initial matching feature point pairs and the true matching feature point pairs of the preset image, the matching probability value of the preset image is determined. The preset image corresponding to the maximum matching probability value is selected as a candidate matching image for the current environment image. Based on the true matching feature point pairs of the candidate matching images, it is determined whether the current environment image and the candidate matching image are successfully matched.

[0059] S13: In response to the matching of the current environment image and the preset image, the preset position corresponding to the preset image is determined as the position information of the running device when the current environment image is acquired.

[0060] Specifically, if the number of true matching feature point pairs in the candidate matching image exceeds a preset value, then the preset position corresponding to the candidate matching image is the position information of the running device when the current environmental image is acquired; if the number of true matching feature point pairs in the candidate matching image does not exceed the preset value, then the preset position corresponding to the candidate matching image is not the position information of the running device when the current environmental image is acquired.

[0061] In one embodiment, the positional error value of the detected feature point pair mapped to the preset feature point pair is determined based on the positional error value and / or the number of real matching feature point pairs corresponding to each real matching feature point pair between the current environment image and the candidate matching image; the matching confidence between the current environment image and the candidate matching image is determined based on the positional error value and / or the number of real matching feature point pairs corresponding to the current environment image and the candidate matching image; in response to the matching confidence between the current environment image and the candidate matching image exceeding the confidence threshold, the preset position of the candidate matching image is determined to be the position information of the running device when the current environment image was acquired.

[0062] This embodiment provides a visual navigation and positioning method. It acquires a current environmental image of the operating device; in response to the historical positioning information of the operating device associated with the current environmental image, it performs feature matching between the current environmental image and a preset image corresponding to the nearest preset location in the historical positioning information; in response to the match between the current environmental image and the preset image, it determines the preset location corresponding to the preset image as the location information of the operating device when the current environmental image was acquired. This application selects a preset image at the nearest preset location in the historical positioning information based on the historical positioning information associated with the current environmental image for feature matching, and performs a diffusion search using the historical positioning information, thus enabling faster and more accurate visual positioning of the operating device.

[0063] Please see Figure 2 , Figure 2 This is a flowchart illustrating a specific embodiment of the visual navigation and positioning method provided by the present invention. This embodiment provides a visual navigation and positioning method, which includes the following steps.

[0064] S201: Obtain the current environmental image of the running device.

[0065] In one specific embodiment, before generating the current environmental image of the running device, a planning map corresponding to the running site is first generated by planning the running scene of the running device. The planning map corresponding to the running scene has multiple preset locations, and preset images of each preset location are collected. Texture information is extracted from the preset images corresponding to each preset location. The world coordinates of the preset location are associated with the preset image and texture information collected at the corresponding preset location. In another optional embodiment, the preset location is associated only with the texture information at that preset location. The texture information collected at all preset locations is stored in a texture information database.

[0066] Specifically, an image acquisition device is fixed to the bottom of the operating device. The operating device, which requires navigation, acquires images of the environment at its current location to obtain an image of the current environment. In another embodiment, the device for acquiring images at a preset location can be the operating device itself. A camera is installed on the bottom of the operating device to acquire environmental information at the preset location. The current environmental image acquired by the operating device is stored in the operating device and processed. Alternatively, the acquired image can be sent to a server for data processing. The process terminates when no image is collected by the image acquisition device. In a specific embodiment, the environmental image can be a ground image of the operating environment. No navigation markers are provided in the operating environment of the operating device. In this embodiment, the operating device can be a mobile robot, etc.

[0067] S202: The current environment image is split into multiple regions based on the quadtree structure algorithm, and the texture information in the regions is extracted.

[0068] Specifically, texture information includes detected feature points. Detected feature points typically represent certain characteristics of an image. In this embodiment, the detected feature points are ORB (Oriented Fast and Rotated BRIEF) feature points. ORB feature points have local invariance and strong noise resistance, and can be used in visual SLAM (Simultaneous Localization and Mapping) systems of various scales. Based on the quadtree structure algorithm and the rBRIEF algorithm, the acquired current environment image is identified, and the corresponding texture information in the current environment image is extracted.

[0069] Please see Figure 3 , Figure 3 yes Figure 2 A flowchart illustrating a specific embodiment of step S202 in the provided visual navigation and positioning method. The specific steps are as follows.

[0070] S2021: The current environment image is split into multiple region images based on the quadtree structure algorithm.

[0071] Specifically, the number of initial root nodes is selected based on the aspect ratio of the acquired current environmental image. Different aspect ratios correspond to different numbers of initial root nodes. Typically, there are 1 or 2 initial root nodes. The determined initial root nodes are then "split" based on the aspect ratio of the current environmental image, primarily to select points of interest (POIs) within the acquired current environmental image. That is, after acquiring POIs in the current environmental image, it is determined whether the response value of the POI is greater than a response threshold. If the response value of the POI is greater than the response threshold, the POI is retained. Typically, there are 4 POIs with large response values ​​in a split image region. The "split" operation continues in the split image regions until the total number of POIs in the resulting image regions reaches the required number, at which point the splitting operation can be stopped. In another optional embodiment, the splitting operation stops when the number of POIs in the split region images does not meet the response threshold. Specifically, the acquired environmental image is split using a quadtree structure. That is, a frame of acquired environmental image is split into 4 region images, and then these 4 region images are further split to obtain 16 region images. As long as the splitting conditions are met, further splitting operations can be performed on this basis.

[0072] S2022: Feature points are extracted from the region image using the Fast feature point detection method to obtain the detected feature points.

[0073] Specifically, feature points are extracted from the region images obtained by the above-mentioned splitting. The pyramid principle can be used to perform downsampling processing on the acquired current environment image at different levels to obtain image pyramids of the acquired current environment image. Subsequently, FAST feature point detection is performed on each layer of the image pyramid to obtain multi-size features, making the obtained FAST feature points scale-invariant. ORB feature point extraction uses FAST corner features. This method determines whether the current point is a detection feature point by comparing the difference in grayscale values ​​between the current pixel and its surrounding points. In a specific embodiment, assume the grayscale value I at pixel P... p Given a set threshold T, in each layer of the image pyramid, taking any pixel P as the center, first judge the four pixels surrounding pixel P (top, bottom, left, and right). If three of these four pixels do not satisfy threshold T... p -T≤I p ≤I p If +T, then select 16 pixels on a circle with radius 3; if 12 consecutive pixels among these 16 pixels do not satisfy I... p -T≤I p ≤I p +T means determining if pixel P is a detection feature point; otherwise, discard pixel P and proceed to the next pixel. Here, T can be 20%.

[0074] The process involves splitting the acquired environmental image once, extracting feature points from the resulting regions, and then determining whether the response value of each feature point in the region image is greater than a response threshold to ascertain whether further splitting is necessary.

[0075] The feature points extracted by decomposing the environmental image using a quadtree structure are relatively evenly distributed in the image, which can make full use of the global information of the image and thus more accurately determine the specific location information of the running equipment when the current environmental image is collected.

[0076] S203: Determine whether the current environmental image is associated with the historical location information of the running device.

[0077] Specifically, to more quickly determine the exact location of the operating device when acquiring the current environmental image, it is necessary to determine whether the historical location information of the operating device was determined before acquiring the current environmental image. If the current environmental image is associated with the historical location information of the operating device before acquiring the current environmental image, a preset location around the historical location information can be delineated, and the preset image of the preset location can be compared with the current environmental image to quickly determine the location information of the operating device when acquiring the current environmental image. If the current environmental image is not associated with the historical location information of the operating device before acquiring the current environmental image, it is necessary to compare the current environmental image with the preset images corresponding to all preset locations to determine the location information of the operating device when acquiring the current environmental image.

[0078] In one specific embodiment, if the current environmental image is associated with the historical location information of the running device, the process jumps directly to step S204; if the current environmental image is not associated with the historical location information of the running device, the process jumps directly to step S207.

[0079] S204: Using historical positioning information as the center position, arrange all preset images corresponding to preset positions in order of increasing distance from the preset position to the center position.

[0080] Specifically, if the current environmental image is associated with historical location information of the operating equipment, the location of the historical location information is determined in the planning map, and the distance between all preset locations and the historical location information is calculated with the historical location information as the center. The historical location information associated with the current environmental image refers to the location information of the operating equipment when it was captured by the operating equipment or image acquisition device in the previous frame of the current environmental image.

[0081] In one embodiment, the current environmental image is associated only with the historical location information of the operating device when it acquired the previous environmental image frame adjacent to the current environmental image. In another embodiment, the current environmental image may also be associated with the historical location information of the operating device when it acquired the environmental images of a preset number of frames prior to the current environmental image. The historical location information corresponding to the environmental image closest in time to the current environmental image is used as the center position, and the distances between each preset position and the center position are calculated.

[0082] Based on the distance between each preset position and the center position, the preset images corresponding to the preset positions are arranged in ascending order.

[0083] S205: Select the top-ranked preset images as candidate matching images and perform feature matching with the current environment image.

[0084] Specifically, a predetermined number of pre-selected images with the highest ranking are selected as candidate matching images. The pre-selected texture information corresponding to each candidate matching image is compared with the texture information corresponding to the current environment image to determine the pre-selected image that matches the features of the current environment image. If none of the predetermined number of pre-selected images selected in the first round match the current environment image, the first selected pre-selected images are discarded, and another predetermined number of pre-selected images are selected as candidate matching images to perform feature matching with the current environment image until a pre-selected image that matches the current environment image is determined.

[0085] See Figure 4 , Figure 4 This is a schematic diagram of each preset position within the running scene in one embodiment.

[0086] In one specific embodiment, the current environmental image is associated with historical location information, specifically location 1. If it is necessary to determine the location information of the device running when the current environmental image was acquired, a tree-like diffusion path is needed, starting from location 1. Based on the distance between each preset location and location 1, the preset images at locations 2 and 5 are first selected as candidate matching images and matched with the current environmental image. If neither the preset images at locations 2 nor 5 successfully match the current environmental image, the preset images at locations 3 and 7 are then selected as candidate matching images and matched with the current environmental image. If neither the preset images at locations 3 nor 7 successfully match the current environmental image, the preset images at locations 4 and 6 are then selected as candidate matching images and matched with the current environmental image.

[0087] In one specific embodiment, the matching feature point pairs corresponding to the detection feature point pairs determined based on the detection feature points of the current environment image and the preset feature point pairs determined based on the preset feature points of the candidate matching image are used to obtain the initial matching feature point pairs corresponding to the candidate matching image and the current environment image.

[0088] See Figure 5 , Figure 5 yes Figure 2 A flowchart illustrating a specific embodiment of step S205 in the provided visual navigation and positioning method.

[0089] S2051: The rBRIEF algorithm is used to select N pixels within a set range centered on the detected feature point to form N feature point pairs with the detected feature point. By comparing the gray values, binary assignment is performed to generate a combination of 0 or 1.

[0090] Specifically, a 31*31 area can be selected centered on a detected feature point in step S202, and N pixels are selected within this area. The selection of N pixels is based on the training data; that is, N pixels are selected from N positions within the training area. N is typically 256, which determines the rBRIEF descriptor dimension to be 256. The selected N pixels are paired with the centered detected feature point to obtain N feature point pairs. In one specific embodiment, within the feature point pairs, by comparing the grayscale values ​​of the centered detected feature point with the selected 256 pixels, pixels with grayscale values ​​smaller than the detected feature point's grayscale value in the image are defaulted to 0, and pixels with grayscale values ​​larger than the detected feature point's grayscale value in the image are defaulted to 1, thus generating a 256-dimensional descriptor that is either 0 or 1.

[0091] S2052: Determine the centroid of N feature point pairs by weighted summation of the encoded combinations of 0 or 1.

[0092] Specifically, based on the above-obtained 0 or 1 encoding combination, the 0 or 1 at different pixel positions are weighted and summed to obtain the gray centroid of N feature point pairs.

[0093] S2053: Connect the detected feature point to the centroid and determine the orientation angle of the detected feature point.

[0094] Specifically, the detected feature points are connected to the centroid, giving the line connecting the feature points and the centroid a direction. In one specific embodiment, the direction angle θ of the line connecting the feature points and the centroid is determined by the position coordinates of N feature point pairs. The specific direction angle θ is obtained using Formula 1.

[0095]

[0096] In Formula 1: N is the number of feature point pairs; y Ni It is the ordinate of the pixel in the feature point pair, y A It is the ordinate of the feature point detected in the feature point pair, x Ni It is the x-coordinate of the pixel in the feature point pair. A It is the x-coordinate of the detected feature point in the feature point pair.

[0097] S2054: The pixel is rotated and sampled according to the direction angle to obtain the feature point pair in the rotated state, and it is determined whether the feature point pair matches the pre-stored texture information in the texture information library.

[0098] Specifically, the 256 pixels obtained above are sampled by rotating them 360° with the obtained direction angle θ as the angular step size to obtain a lookup table of rotation descriptors. That is, feature point pairs with multiple angles are obtained by rotation, and then the rotated feature point pairs are compared with the feature point pairs in the pre-stored texture information of each preset image, and the matching feature point pairs that match the pre-stored texture information are determined.

[0099] Through the above steps S2051 to S2054, matching feature point pairs corresponding to the current environmental image and each preset image are determined. In one embodiment, in order to improve the positioning accuracy of the operating device, it is necessary to filter the matching feature point pairs corresponding to the current environmental image and each preset image.

[0100] Specifically, the matching feature point pairs obtained in steps S2051 to S2054 are filtered twice. S206: The matching feature point pairs that match the pre-stored texture information are filtered.

[0101] See Figure 6 , Figure 6 yes Figure 2 A flowchart illustrating a specific embodiment of step S206 in the provided visual navigation and positioning method. The specific screening steps are as follows.

[0102] S2061: Obtain the Hamming distance during the matching process between each matching feature point pair and the pre-stored texture information.

[0103] Specifically, the Hamming distance is used to calculate the Hamming distance between the binary hash code of the current environment image and the binary hash code of the preset image associated with each texture information in the pre-stored texture information library. All the calculated Hamming distances are sorted in ascending order to obtain matching feature point pairs sorted in ascending order of Hamming distance.

[0104] S2062: Determine whether the Hamming distance corresponding to each matched feature point pair is less than a preset value.

[0105] Specifically, a preset value is set and preset images associated with texture information whose Hamming distance is less than the preset value are selected. The threshold can be preset to the Hamming distance value located at the nth position in ascending order, and then adjusted according to the test results to finally obtain the threshold with the highest accuracy. The threshold needs to be tested multiple times and the threshold needs to be continuously adjusted to find the optimal threshold, because the features extracted from the preset images are different, and their relativity and similarity are also different. The setting of the threshold is an existing technology and will not be elaborated here.

[0106] If the Hamming distance is less than the preset value, proceed directly to step S2063; if the Hamming distance is not less than the preset value, proceed directly to step S2064.

[0107] S2063: Preserve the matching feature point pairs corresponding to the Hamming distance.

[0108] Specifically, if the Hamming distance is less than a preset value, the matching feature point pair corresponding to the Hamming distance is retained.

[0109] S2064: Delete the matching feature point pairs corresponding to the Hamming distance.

[0110] Specifically, if the Hamming distance is greater than or equal to a preset value, the matching feature point pairs corresponding to the Hamming distance are removed.

[0111] S2065: Determine whether the distance between detected feature points in the current environment image is equal to the distance between detected feature points in the previous frame environment image.

[0112] Specifically, since the image acquisition device is fixed at the bottom of the running device, and the running environment of the running device is mostly a horizontal bottom surface, according to the structural information of the image acquisition device being fixedly connected to the bottom of the running device, the running device can move or rotate in a plane at a fixed height from the bottom surface. The size of the image acquisition device in the optical axis direction remains unchanged, and the size of the texture information in the image frame acquired by the running device is the same as the size of the texture information in the adjacent previous frame stored in the running device.

[0113] In one specific embodiment, it is determined whether the distance between each detected feature point in the current environmental image is equal to the distance between the same detected feature point pairs in the previous frame environmental image. If the distance between each detected feature point in the current environmental image is equal to the distance between the same detected feature point pairs in the previous frame environmental image, it indicates that the environmental image acquired by the image acquisition device has not moved in the optical axis direction, and the process directly jumps to step S2066; if the distance between each detected feature point in the current environmental image is not equal to the distance between the same detected feature point pairs in the previous frame environmental image, it indicates that the environmental image acquired by the image acquisition device has moved in the optical axis direction, and the process directly jumps to step S2067.

[0114] S2066: Retain the matching feature point pairs in the current environment image that are equidistant from the detected feature points in the previous frame environment image.

[0115] Specifically, if the distance between each detected feature point in the current environmental image is equal to the distance between the same detected feature point pairs in the previous frame environmental image, it indicates that the environmental image acquired by the image acquisition device has not moved in the optical axis direction. The image acquisition device satisfies the fixed connection structure between the image acquisition device and the running device when acquiring the image. Therefore, the data of the feature point pair is reliable, and the matching feature point pair corresponding to the detected feature points in the current environmental image with the same distance between the detected feature points in the previous frame environmental image is retained.

[0116] S2067: Delete the matching feature point pairs in the current environment image that are not equidistant from the detected feature points in the previous frame environment image.

[0117] Specifically, if the distance between each detected feature point in the current environmental image is not equal to the distance between the same detected feature point pairs in the previous frame environmental image, it indicates that the environmental image acquired by the image acquisition device has moved in the optical axis direction. The image acquisition device violated the fixed connection structure between the image acquisition device and the running device when acquiring the image. Therefore, the data of the feature point pair is unreliable. The matching feature point pair corresponding to the detected feature point in the current frame that has the same distance between the detected feature points in the previous frame environmental image is deleted.

[0118] In another optional embodiment, since the installation of the image acquisition device may have errors or the ground may be uneven or non-ideal, an error value can be set to determine whether the difference between the distance between detected feature points in the current environmental image and the distance between detected feature points in the previous frame environmental image exceeds the error value. The error value ranges from (0, 1], for example, the error value is greater than 0.9 and less than or equal to 1. If it does not exceed the error value, the matching feature point pair corresponding to the detected feature point in the current environmental image is retained. If it exceeds the error value, the matching feature point pair corresponding to the detected feature point in the current frame environmental image is retained.

[0119] Through the above steps S2051 to S2054, the initial matching feature point pairs m corresponding to the current environmental image and each preset image are determined. i_total Through steps S2061 to S2067 above, the initial matching feature point pairs are filtered out, and mismatched feature point pairs m are eliminated. i_mismatch The retained matching feature point pairs are all true matching feature point pairs m. i_true .

[0120] S207: Perform feature matching between the current environment image and the preset images corresponding to all preset locations.

[0121] Specifically, if the current environmental image is not associated with the historical positioning information of the operating device, that is, the historical positioning information before the current environmental image is lost, it is equivalent to the operating device performing the initialization settings of the positioning system, that is, the operating device takes the current location as the starting point. In this case, the current environmental image needs to be matched with the preset images of all preset locations to determine the location information of the operating device in the planning map and the position coordinates in the world coordinate system when the current environmental image was collected.

[0122] See Figure 7 , Figure 7 yes Figure 2 A flowchart illustrating a specific embodiment of step S207 in the provided visual navigation and positioning method.

[0123] S2071: Based on the detected feature points corresponding to the current environment image and the preset feature points of each preset image in the database, determine the matching probability value between the preset image and the current environment image.

[0124] Specifically, through the steps S2051 to S2054 described above, the feature point pairs of the current environment image are sequentially matched with the feature point pairs of each preset image to obtain the initial matching feature point pairs m between the current environment image and each preset image. i_total Through steps S2061 to S2067, the initial matching feature point pairs are filtered out, eliminating mismatched feature point pairs m between the current environment image and each preset image. i_mismatch The retained matching feature point pairs are all true matching feature point pairs m. i_true .

[0125] In one specific embodiment, the matching probability value p between each preset image and the current environment image is calculated using the following formula 2. i .

[0126] p i =m i_total -m i_mismatch / m i_total (Formula 2)

[0127] S2072: Use the preset image corresponding to the highest matching probability value as the candidate matching image for the current environment image.

[0128] Specifically, the detected feature points corresponding to the current environment image are matched with the preset feature points of the preset image corresponding to the maximum matching probability value to determine the matching feature point pairs between the current environment image and the preset image.

[0129] Specifically, to reduce computation and improve positioning accuracy, the preset image with the highest matching probability value is selected from all preset images and matched with the current environment image. In other words, the preset image with the highest matching probability value is used as a candidate matching image, and this candidate matching image is then matched with the current environment image.

[0130] S208: Determine whether the number of true matching feature point pairs between the current environment image and the candidate matching image exceeds a preset value.

[0131] Specifically, in order to determine whether the current environment image is the same as or similar to the candidate matching image, the number of true matching feature point pairs between the current environment image and the candidate matching image is used for evaluation.

[0132] If the number of true matching feature point pairs between the current environment image and the candidate matching image exceeds a preset value, proceed directly to step S209; if the number of true matching feature point pairs between the current environment image and the candidate matching image does not exceed a preset value, proceed directly to step S210.

[0133] S209: Preliminarily determine the preset position corresponding to the candidate matching image as the position information of the running device when acquiring the current environment image.

[0134] Specifically, if the number of real matching feature point pairs between the current environmental image and the preset image exceeds a preset value, then the preset position corresponding to the preset image is determined as the position information of the running device when the current environmental image is acquired.

[0135] S210: Match another preset image as a candidate matching image with the current environment image.

[0136] Specifically, if the number of true matching feature point pairs between the current environment image and the candidate matching image does not exceed a preset value, then another preset image is directly selected as the candidate matching image to match the current environment image.

[0137] To improve the reliability and accuracy of the location information of the running device when acquiring the current environment image, it is necessary to further confirm whether the candidate matching image matches the current environment image.

[0138] S211: Detect the position error value of the feature point pair mapped to the preset feature point pair based on the real matching feature point pairs corresponding to the current environment image and the candidate matching image.

[0139] Specifically, the positional error value of the detected feature point pair in the true matching point pair is calculated from the current environment image to the preset feature point pair in the corresponding preset image.

[0140] In one specific embodiment, the positional error value e of the detected feature point pairs in the current environment image mapped to the preset feature point pairs in the candidate matching image that initially matches the current environment image is obtained based on the following formula 3. repro .

[0141]

[0142] S212: Determine the matching confidence between the current environment image and the candidate matching image based on the corresponding positional error values ​​and / or the number of true matching feature point pairs between the current environment image and the candidate matching image.

[0143] In one specific embodiment, the matching confidence between the current environment image and the candidate matching image is determined based on the corresponding positional error value between the current environment image and the candidate matching image and the number of true matching feature point pairs.

[0144] For example, the matching confidence between the current environment image and the candidate matching image can be calculated using the following formula 4.

[0145] confidence = a·e repro / e thred +b·m match / m total (Formula 4)

[0146] In Formula 4: confident represents the matching confidence level; a and b are weight constants.

[0147] A lower matching confidence indicates that the matching result between the current environment image and the candidate matching image is unreliable; a higher matching confidence indicates that the matching result between the current environment image and the candidate matching image is more reliable.

[0148] S213: In response to the matching confidence between the current environment image and the candidate matching image exceeding the confidence threshold, the preset position of the candidate matching image is determined to be the position information of the running device when the current environment image was acquired.

[0149] Specifically, if the matching confidence between the current environmental image and the preset image exceeds a confidence threshold, then the preset location corresponding to the candidate matching image is determined as the location information of the device running when acquiring the current environmental image. If the matching confidence between the current environmental image and the preset image does not exceed the confidence threshold, then the preset location corresponding to the candidate matching image cannot be used as the location information of the device running when acquiring the current environmental image.

[0150] The visual navigation and positioning method provided in this embodiment includes: acquiring a current environmental image of the operating device; responding to the historical positioning information of the operating device associated with the current environmental image, performing feature matching between the current environmental image and a preset image corresponding to the preset location closest to the historical positioning information; and responding to the matching between the current environmental image and the preset image, determining the preset location corresponding to the preset image as the position information of the operating device when the current environmental image was acquired. This application selects a preset image at the preset location closest to the historical positioning information based on the historical positioning information of the operating device associated with the current environmental image for feature matching, and performs a diffusion search using the historical positioning information, thereby achieving faster and more accurate visual positioning of the operating device.

[0151] See Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of the visual navigation and positioning device provided by the present invention. This embodiment provides a visual navigation and positioning device 60, which includes an acquisition module 61, a matching module 62, and a determination module 63.

[0152] The acquisition module 61 is used to acquire the current environmental image of the running device.

[0153] The matching module 62 is used to perform feature matching between the current environmental image and the preset image corresponding to the preset location closest to the historical location information in response to the historical location information associated with the current environmental image.

[0154] The determination module 63 is used to determine the preset position corresponding to the preset image as the position information of the running device when the current environmental image is acquired if the current environmental image is matched with the preset image.

[0155] The visual navigation and positioning device provided in this embodiment selects a preset image of a preset location closest to the historical positioning information based on the historical positioning information of the operating equipment associated with the current environmental image, performs feature matching, and performs diffusion search through historical positioning information, which can complete the visual positioning of the operating equipment more quickly and accurately.

[0156] Please see Figure 9 , Figure 9 This is a schematic diagram of a terminal embodiment provided in this application. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described visual navigation and positioning method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0157] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described visual navigation and positioning method embodiments. Processor 82 may also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 82 may be implemented using integrated circuit chips.

[0158] The above scheme, the visual navigation and positioning method includes: acquiring the current environmental image of the operating device; in response to the historical positioning information of the operating device associated with the current environmental image, performing feature matching between the current environmental image and the preset image corresponding to the preset position closest to the historical positioning information; in response to the matching between the current environmental image and the preset image, determining the preset position corresponding to the preset image as the position information of the operating device when acquiring the current environmental image.

[0159] Please see Figure 10 , Figure 10 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described visual navigation and positioning method embodiments.

[0160] The above scheme, the visual navigation and positioning method includes acquiring the current environmental image of the operating device; responding to the historical positioning information of the operating device associated with the current environmental image, performing feature matching between the current environmental image and the preset image corresponding to the preset position closest to the historical positioning information; responding to the matching between the current environmental image and the preset image, determining the preset position corresponding to the preset image as the position information of the operating device when acquiring the current environmental image.

[0161] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0162] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0167] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of visual navigation positioning, characterized in that, The method comprises: acquiring a current environment image of a running device; in response to historical positioning information of the running device being associated with the current environment image, performing feature matching between the current environment image and a preset image corresponding to a preset position closest to the historical positioning information; in response to the current environment image matching the preset image, determining the preset position corresponding to the preset image as position information of the running device when the current environment image is acquired; the response to the current environment image matching the preset image includes: in response to the number of real matching feature point pairs between the preset image and the current environment image exceeding a preset value, determining the preset position corresponding to the preset image as the position information of the running device when the current environment image is acquired; the real matching feature point pairs are selected from matching feature point pairs between the preset image and the current environment image; the matching feature point pairs are obtained by performing feature matching between the preset image and the current environment image.

2. The visual navigation positioning method of claim 1, wherein: the response to the current environment image being associated with the historical positioning information of the running device includes: based on a quadtree structure algorithm, splitting the current environment image to obtain a plurality of regions, and extracting texture information in the regions; the texture information includes detected feature points.

3. The visual navigation positioning method of claim 2, wherein: the feature matching between the current environment image and the preset image corresponding to the preset position closest to the historical positioning information includes: arranging the preset images corresponding to all the preset positions in order from near to far according to the distance between the preset positions and a center position of the historical positioning information; selecting the preset images at the front of the order as candidate matching images of the current environment image; performing feature matching between the current environment image and the candidate matching images.

4. The visual navigation positioning method of claim 3, wherein: the feature matching between the current environment image and the candidate matching images includes: counting corresponding matching feature point pairs between detected feature point pairs determined based on the detected feature points of the current environment image and preset feature point pairs determined based on preset feature points of the candidate matching images, to obtain initial matching feature point pairs between the candidate matching images and the current environment image; screening the initial matching feature points between the candidate matching images and the current environment image to obtain real matching feature point pairs between the candidate matching images and the current environment image; based on the real matching feature point pairs of the candidate matching images, determining whether the current environment image matches the candidate matching images successfully. 5.The visual navigation positioning method of claim 2, wherein, the visual navigation positioning method further comprises: in response to the current environment image not associating the historical positioning information of the running device, performing feature matching between the current environment image and the preset image corresponding to each of the preset positions respectively. 6.The visual navigation positioning method of claim 5, wherein, the performing feature matching between the current environment image and the preset image corresponding to each of the preset positions respectively comprises: counting corresponding matching feature point pairs between detection feature point pairs determined based on the detection feature points of the current environment image and preset feature point pairs determined based on the preset feature points of the preset image to obtain initial matching feature point pairs between the preset image and the current environment image; screening the initial matching feature points between the preset image and the current environment image to obtain real matching feature point pairs between the preset image and the current environment image; determining a matching probability value of the preset image based on the initial matching feature point pairs and the real matching feature point pairs of the preset image; selecting the preset image corresponding to the maximum matching probability value as a candidate matching image of the current environment image; determining whether the current environment image and the candidate matching image match successfully based on the real matching feature point pairs of the candidate matching image. 7.The visual navigation positioning method of claim 4 or 6, wherein, in response to the current environment image matching the preset image, the preset position corresponding to the preset image is determined as the position information of the running device when the current environment image is collected, comprising: in response to the number of real matching feature point pairs of the candidate matching image exceeding a preset value, the preset position corresponding to the candidate matching image is determined as the position information of the running device when the current environment image is collected. 8.The visual navigation positioning method of claim 7, wherein, in response to the current environment image matching the preset image, the preset position corresponding to the preset image is determined as the position information of the running device when the current environment image is collected, further comprising: based on the position error value of the detection feature point pairs in each of the real matching feature point pairs between the current environment image and the candidate matching image mapping to the preset feature point pairs; based on the position error value and / or the number of real matching feature point pairs between the current environment image and the candidate matching image, determining a matching confidence between the current environment image and the candidate matching image; in response to the matching confidence between the current environment image and the candidate matching image exceeding a confidence threshold, determining the preset position of the candidate matching image as the position information of the running device when the current environment image is collected.

9. A vision navigated positioning device, characterized by comprising: an acquisition module, configured to acquire a current environment image of a running device; The matching module is configured to perform feature matching between the current environment image and a preset image corresponding to a preset position closest to the historical positioning information of the running device in response to the current environment image being associated with the historical positioning information of the running device. The determining module is configured to determine the preset position corresponding to the preset image as the position information of the running device when the current environment image is captured in response to the current environment image matching the preset image. The determining module is further configured to determine the preset position corresponding to the preset image as the position information of the running device when the current environment image is captured in response to the number of real matching feature point pairs between the preset image and the current environment image exceeding a preset value, wherein the real matching feature point pairs are selected from matching feature point pairs between the preset image and the current environment image, and the matching feature point pairs are obtained by performing feature matching between the preset image and the current environment image.

10. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is configured to execute program data to implement the steps in the visual navigation positioning method according to any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps in the visual navigation positioning method according to any one of claims 1-8.

Images