A robot-based vision positioning navigation method and device

Abstract

The application relates to a robot-based visual positioning navigation method and device, and relates to the technical field of positioning navigation. When positioning information obtained by a GPS receiver of a robot is in a positioning failure state, multiple candidate key frames are obtained by processing multiple images based on the positioning failure state; the multiple candidate key frames are screened according to the current track curvature of the robot to obtain an initial candidate key frame set; the initial candidate key frame set is screened based on environment scores to obtain a target candidate key frame set; feature points of a target frame image are matched and calculated with feature points of each target candidate key frame to obtain a hypothesis camera pose; all initial matching pairs are projected and tested based on the hypothesis camera pose, and the number of inliers is counted; a pose corresponding to a final candidate key frame with the maximum number of inliers is selected as position information of the robot at the current moment. The technical scheme is implemented to solve the positioning difficulty problem caused by repetitive structures in a tunnel environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of positioning and navigation, in particular to a robot-based visual positioning and navigation method and device. BACKGROUND

[0002] In the wave of robot technology and autonomous driving industry, high-precision and high-reliability autonomous navigation is the cornerstone of intelligent operation. However, in environments such as tunnels, urban centers with high-rise buildings, or large indoor parking lots, the signal of the global positioning system (GPS) is attenuated or interrupted.

[0003] In order to solve the problem of signal interruption, visual repositioning technology is usually used for positioning. The visual repositioning technology process is that when the system enters the GPS signal blind area, the image of the current scene is captured, and a series of local feature points are extracted therefrom. These feature points are converted into an efficient and global image descriptor, such as a visual bag-of-words vector. The system uses the vector to perform fast retrieval in a large three-dimensional map database, recalls a group of historical key frames that are most matched in visual appearance with the current scene as candidate poses by calculating the similarity between the descriptors, and determines the final positioning result according to the candidate poses.

[0004] However, in a tunnel environment with a large number of repetitive structures, similar textures, and lighting conditions, many physically different locations can be highly similar in vision. Only based on the visual bag-of-words model, a large number of false candidate frames will be recalled, which will increase the probability of false poses and further lead to positioning failure. SUMMARY

[0005] The present application provides a robot-based visual positioning and navigation method and device, which effectively solves the positioning difficulty caused by repetitive structures, similar textures, and lighting changes in a tunnel environment, and provides stable autonomous navigation capability for robots.

[0006] In a first aspect, the present application provides a robot-based visual positioning and navigation method, the method comprising:

[0007] Obtain positioning information from a GPS receiver of a robot, determine that the GPS receiver is in a positioning failure state according to the positioning information, and obtain multiple frames of images of the robot in a current environment in real time based on the positioning failure state; perform feature extraction on the multiple frames of images to obtain a local feature set, convert each feature in the local feature set to obtain a bag-of-visual-words vector; search a preset three-dimensional map database for multiple candidate key frames that match the bag-of-visual-words vector; calculate a motion trajectory of the robot within a preset time to obtain a current trajectory curvature; select the multiple candidate key frames according to the current trajectory curvature to obtain an initial candidate key frame set; obtain a lateral offset of the robot from a center lane line of a tunnel, and obtain light brightness in real time; calculate each initial candidate key frame in the initial candidate key frame set according to the light brightness and the lateral offset to obtain multiple environment scores, with one initial candidate key frame corresponding to one environment score; sort the multiple environment scores, and select a preset first number of initial candidate key frames from a sorting result to induce into a target candidate key frame set; match feature points of a target frame image with feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs, calculate a hypothetical camera pose using a preset algorithm on the set of initial matching pairs, project all initial matching pairs based on the hypothetical camera pose to perform a projection test, count an inlier number, and the target frame image is a frame of image closest to a current moment in the multiple frames of images; select a pose corresponding to a final candidate key frame with the maximum inlier number as position information of the robot at the current moment.

[0008] By using the above technical solution, the positioning information of the GPS receiver is monitored, the positioning failure state is identified in time, and multiple frames of images of the current environment are captured, local features are extracted and converted into a bag-of-visual-words vector, fast matching with historical key frames in a preset three-dimensional map database is realized, in order to overcome the interference caused by the similarity of visual features in a tunnel environment, the motion trajectory curvature of the robot is calculated first to select candidate key frames that match the current motion state, avoiding false matching caused by a motion state that is obviously inconsistent, and then the lateral offset of the robot from the center lane line of the tunnel and the brightness of the ambient light are collected in real time to score the environment similarity of the candidate key frames, further improving the reliability of the candidate frames. In the final precise positioning stage, the accuracy of pose estimation is evaluated by calculating the number of inliers, the problem that the repositioning relying only on visual features is easily disturbed by the environment is significantly reduced based on a multi-level screening mechanism, and the positioning accuracy and reliability in special environments such as tunnels are improved.

[0009] Optionally, the initial candidate key frame set is calculated according to the light brightness and the lateral offset to obtain a plurality of environment scores, specifically including: calling the historical light brightness and the historical lateral offset corresponding to the target initial candidate key frame, the target initial candidate key frame being any one of the initial candidate key frame set; calculating the difference between the light brightness and the historical light brightness to obtain a brightness difference value; calculating the brightness difference value based on a preset brightness score function to obtain a brightness similarity score, wherein the brightness similarity score and the absolute value of the brightness difference value are in a negative correlation relationship; calculating the difference between the lateral offset and the historical lateral offset to obtain an offset difference value; calculating the offset difference value based on a preset offset score function to obtain an offset similarity score, wherein the offset similarity score and the absolute value of the offset difference value are in a negative correlation relationship; and performing weighted summation on the brightness similarity score and the offset similarity score to obtain a preliminary environment score of the target initial candidate key frame; obtaining the preliminary environment score corresponding to each target initial candidate key frame in the initial candidate key frame set, and outputting the preliminary environment score as the environment score.

[0010] By using the above technical solution, the light brightness and the lateral offset information corresponding to the candidate key frame are extracted from the historical data, the difference between the current environment and the historical data is calculated, the brightness difference value and the offset difference value are obtained respectively, the brightness similarity score and the offset similarity score are weighted and calculated, the importance difference of different environmental characteristics is considered, and the organic combination of multi-dimensional features is realized, so that a more comprehensive and reliable environment score is obtained, which can effectively identify and filter out candidate key frames with similar visual features but significant differences in environmental conditions, significantly reduce the probability of false matching caused by repetitive structures and similar textures in the tunnel environment, provide a more reliable candidate frame set for subsequent accurate positioning, and improve the reliability of the entire visual navigation system.

[0011] Optionally, the preliminary environment score is output as the environment score, specifically including: sorting the plurality of preliminary environment scores, and selecting a preset second number of initial candidate key frames with the highest scores from the sorting result to form a preliminary candidate set; calculating the score difference between the highest score and the lowest score in the preliminary candidate set to obtain a score differentiation degree; when the score differentiation degree is less than a preset differentiation threshold, calling the historical environment temperature corresponding to the target initial candidate key frame; obtaining the current environment temperature of the robot, calculating the difference between the historical environment temperature and the current environment temperature to obtain a temperature difference value; calculating the temperature difference value based on a preset temperature score function to obtain a temperature similarity score; and performing weighted summation on the preliminary environment score and the temperature similarity score to obtain the environment score; and when the score differentiation degree is greater than or equal to the preset differentiation threshold, outputting the preliminary environment score as the environment score corresponding to the target initial candidate key frame.

[0012] By adopting the technical scheme, after the candidate key frame is preliminarily scored according to the environment, the difference between the highest score and the lowest score is calculated to evaluate the distinguishing effect of the current score. When the score distinguishing degree is low, it indicates that the two environmental characteristics of light brightness and lateral offset cannot effectively distinguish the similarity of the candidate key frame. By comparing the difference between the historical environmental temperature and the current temperature, the temperature similarity score is calculated, and the preliminary environmental score is weighted to obtain a more distinguishable comprehensive environmental score. This score distinguishing degree can effectively avoid the misjudgment problem under the condition that the environmental characteristics are similar, and improves the accuracy of the candidate key frame screening. At the same time, when the score distinguishing degree meets the threshold requirement, the preliminary environmental score is directly used as the final score, which not only ensures the reliability of the score, but also avoids unnecessary calculation overhead, and reflects the good balance between accuracy and efficiency of the scheme.

[0013] Optionally, the motion trajectory of the robot within the preset time is calculated to obtain the current trajectory curvature, specifically including: obtaining the angular velocity of the robot rotating along the vertical axis within the preset time, and obtaining the average driving speed within the preset time from the wheel pulse number of the robot; filtering the angular velocity and the average driving speed to obtain a smooth angular velocity and a smooth linear velocity; when the smooth linear velocity is less than a minimum speed threshold, it is confirmed that the robot is in a stationary state, and the corresponding current trajectory curvature in the stationary state is 0; when the smooth linear velocity is greater than or equal to the minimum speed threshold, the smooth angular velocity is divided by the smooth linear velocity to obtain the current trajectory curvature.

[0014] By adopting the above technical scheme, the angular velocity of the vertical axis rotation and the wheel pulse data are obtained from the sensor system of the robot, and the noise and fluctuations in the original data are eliminated through filtering processing, ensuring the accuracy of the speed calculation. When the smooth linear velocity is lower than the minimum speed threshold, the stationary state of the robot can be accurately identified, and the trajectory curvature is directly set to 0, avoiding the curvature calculation error caused by slight jitter in the stationary state. In the motion state of the robot, the trajectory curvature is calculated by the ratio of the smooth angular velocity and the smooth linear velocity. This calculation method not only considers the steering motion of the robot, but also takes the advancing speed as a normalization factor, so that the curvature calculation is more accurate and reliable, providing an accurate motion feature description for the subsequent candidate key frame screening, effectively reducing the error key frame matching caused by the mismatch of the motion state, and improving the positioning accuracy of the entire visual navigation system.

[0015] Optionally, the plurality of candidate key frames are filtered according to the current trajectory curvature to obtain an initial candidate key frame set, specifically including: calling a plurality of historical motion trajectories of the robot driving on a plurality of standard straight line test road sections, calculating each historical motion trajectory to obtain a plurality of trajectory curvature values; selecting a maximum value from the plurality of trajectory curvature values as a straight line curvature threshold; if the current trajectory curvature is less than or equal to the straight line curvature threshold, determining that the motion trajectory of the robot is a straight line motion state; calling the historical trajectory curvature corresponding to each of the plurality of candidate key frames, and judging whether the historical trajectory curvature is less than or equal to the straight line curvature threshold; when the historical trajectory curvature is less than or equal to the straight line curvature threshold, determining the candidate key frame as a straight line key frame; when the historical trajectory curvature is greater than the straight line curvature threshold, determining the candidate key frame as a non-straight line key frame; selecting a sub-candidate key frame corresponding to the straight line key frame from the plurality of candidate key frames according to the straight line motion state, and inducing the sub-candidate key frame into the initial candidate key frame set.

[0016] By adopting the above technical solution, the historical motion data is obtained from the standard straight line test road section, the maximum value is selected as the straight line curvature threshold by analyzing the curvature values of the plurality of historical trajectories, the slight deviation and measurement error of the robot in the straight line motion are effectively considered, and the determination of the straight line motion state is more in line with the actual situation. In the motion state discrimination, the current trajectory curvature is compared with the straight line curvature threshold, and whether the robot is in the straight line motion state is accurately identified. The historical trajectory curvature of the candidate key frame is subjected to the same threshold judgment, and is classified into the straight line key frame and the non-straight line key frame, and the classification method based on the motion feature ensures the consistency of the candidate key frame and the current motion state. When the robot is in the straight line motion state, the straight line key frame is selected as the candidate frame, the key frame with mismatched motion state is effectively filtered, the probability of mismatch caused by the motion state difference is significantly reduced, and the positioning accuracy and reliability of the visual navigation system in the straight line motion scene are improved.

[0017] Optionally, the feature points of the target frame image are matched with the feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs, a preset algorithm is used to calculate the set of initial matching pairs to obtain a hypothesis camera pose, projection testing is performed on all the initial matching pairs based on the hypothesis camera pose, and the number of inliers is counted, specifically including: random sampling and calculation are performed from the set of initial matching pairs to obtain a plurality of candidate camera poses; two-dimensional feature points containing the initial matching pairs in the target frame image are called, associated three-dimensional map points are found according to the initial matching pairs and the target candidate key frame, the initial matching pairs are any one of the set of initial matching pairs; the three-dimensional map points are projected and transformed based on the target candidate camera pose to obtain a re-projection point located in the coordinate system of the target frame image, the target candidate camera pose is any one of the plurality of candidate camera poses; the geometric distance between the two-dimensional feature points and the re-projection point is calculated to obtain a re-projection error; if the re-projection error is less than or equal to a preset re-projection error threshold, the initial matching pairs are determined as inliers supporting the target candidate camera pose; the total number of the initial matching pairs determined as inliers in the set of initial matching pairs is accumulated, and the total number is taken as an inlier score of the target candidate camera pose; after the inlier score calculation of all the candidate camera poses is completed, the candidate camera pose with the maximum inlier score is selected from the plurality of inlier scores as the hypothesis camera pose, and the maximum inlier score is taken as the number of inliers.

[0018] By using the above technical solution, the initial matching pairs are randomly sampled to generate a plurality of candidate camera pose hypotheses, and this sampling-based method can effectively reduce the calculation complexity. In the verification process, the geometric correspondence between the two-dimensional feature points and the three-dimensional space points is established by projecting the three-dimensional map points onto the current image plane, and then the geometric distance between the projection points and the actual observation points is calculated to obtain the re-projection error. This projection error-based evaluation method directly reflects the accuracy of the pose estimation. By setting a re-projection error threshold, the matching pairs meeting the accuracy requirement are marked as inliers, and a pose scoring mechanism is established by accumulating the number of inliers. This scoring method not only considers the accuracy of feature matching, but also verifies the spatial consistency between the pose estimation and the actual observation, thereby effectively filtering out the incorrect pose estimations that do not meet the geometric constraints.

[0019] Optionally, the determining that the GPS receiver is in the positioning failure state according to the positioning information specifically comprises: analyzing the positioning information to obtain a positioning quality indicator and a precision factor; calling a plurality of sets of calibration data of the robot passing through the entrance area of the plurality of standard test tunnels, the calibration data at least including a real-time precision factor output by the GPS receiver and a tunnel entry event of the robot entering the inside of the tunnel; extracting the corresponding real-time precision factor at the time of each tunnel entry event from the plurality of sets of calibration data to obtain a plurality of historical precision factors; performing statistical calculation on the plurality of historical precision factors to obtain a mean value and a standard deviation; adding a product of the mean value and the standard deviation multiplied by a preset weight coefficient to obtain a precision factor threshold; judging whether the positioning quality indicator exists in a preset failure quality indicator table and whether the precision factor is greater than or equal to the precision factor threshold; and determining that the GPS receiver is in the positioning failure state when the positioning quality indicator exists in the preset failure quality indicator table or the precision factor is greater than or equal to the precision factor threshold.

[0020] By using the above technical solution, the positioning quality indicator and the precision factor, two key parameters, are extracted from the GPS positioning information, the real-time precision factor when the GPS signal starts to attenuate is collected by using the historical data of the robot in the entrance area of the standard test tunnel, and the data collection method based on the actual scene ensures the practicability of the judgment basis. By performing statistical analysis on the historical precision factors, the mean value and the standard deviation are calculated, and the weight coefficient is introduced for weighted combination to obtain a dynamic precision factor threshold. The adaptive threshold setting method based on the historical data can better adapt to the characteristics of the GPS signal attenuation in different environments. The positioning quality indicator and the precision factor are considered at the same time, and the reliability of the state judgment is improved. When any indicator indicates that the signal is abnormal, the positioning failure state is determined. The timely state recognition mechanism ensures that the visual navigation system can be started in time when the GPS signal starts to deteriorate, thereby ensuring the continuity and reliability of the robot navigation system, and effectively avoiding the positioning interruption problem caused by the sudden loss of the GPS signal.

[0021] In a second aspect of the present application, a robot-based visual positioning navigation device is provided. The device includes an acquisition unit, a processing unit, a screening unit, and a confirmation unit. The acquisition unit acquires positioning information from a GPS receiver of a robot, determines that the GPS receiver is in a positioning failure state according to the positioning information, and acquires multiple frames of images of the robot in a current environment in real time based on the positioning failure state. The processing unit extracts features from the multiple frames of images to obtain a local feature set, converts each feature in the local feature set to obtain a visual bag-of-words vector, searches a preset three-dimensional map database for multiple candidate key frames that match the visual bag-of-words vector, and calculates a current trajectory curvature of a motion trajectory of the robot within a preset time. The screening unit screens the multiple candidate key frames according to the current trajectory curvature to obtain an initial candidate key frame set. The acquisition unit acquires a lateral offset of the robot from a center lane line of a tunnel and acquires light brightness in real time, calculates each initial candidate key frame in the initial candidate key frame set according to the light brightness and the lateral offset to obtain multiple environment scores, one initial candidate key frame corresponding to one environment score, sorts the multiple environment scores, and selects a preset first number of initial candidate key frames from a sorting result and induces the initial candidate key frames into a target candidate key frame set. The confirmation unit matches feature points of a target frame image to feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs, calculates a hypothetical camera pose using a preset algorithm on the set of initial matching pairs, projects all initial matching pairs based on the hypothetical camera pose to perform a projection test, counts an inlier number, and selects a pose corresponding to a final candidate key frame with a maximum inlier number as position information of the robot at a current moment, the target frame image being a frame of image closest to the current moment in the multiple frames of images.

[0022] In a third aspect of the present application, an electronic device is provided. The electronic device includes a processor, a memory, a user interface, and a network interface. The memory is configured to store instructions. The user interface and the network interface are configured to communicate with other devices. The processor is configured to execute the instructions stored in the memory, so that the electronic device performs the method of any one of the above aspects of the present application.

[0023] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed, the method of any one of the above aspects of the present application is performed.

[0024] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] 1. The positioning information of the GPS receiver is monitored, the positioning failure state is identified in time, multiple frames of images of the current environment are captured, local features are extracted and converted into a visual bag-of-words vector, fast matching with historical key frames in a preset three-dimensional map database is realized, in order to overcome the interference caused by the similarity of visual features in the tunnel environment, the curvature of the motion trajectory of the robot is calculated, the candidate key frames matched with the current motion state are screened out, the error matching of the motion state obviously not matching is avoided, and then the lateral offset of the robot and the center lane line of the tunnel and the brightness of the ambient light are collected in real time, the environment similarity score of the candidate key frames is calculated, and the reliability of the candidate frames is further improved. In the final precise positioning stage, the accuracy of pose estimation is evaluated by calculating the number of inlier points, the problem that the repositioning relying only on visual features is easily disturbed by the environment is significantly reduced based on the multi-level screening mechanism, and the positioning accuracy and reliability in the special environment such as the tunnel are improved.

[0026] 2. Historical motion data is obtained from a standard straight line test section, the maximum value is selected as the straight line curvature threshold value by analyzing the curvature values of multiple historical trajectories, and the slight deviation and measurement error of the robot in straight line motion are effectively considered, so that the determination of the straight line motion state is more in line with the actual situation. In the motion state discrimination, the current trajectory curvature is compared with the straight line curvature threshold value, and whether the robot is in a straight line motion state is accurately identified. The historical trajectory curvature of the candidate key frame is subjected to the same threshold value judgment, and is classified into straight line key frames and non-straight line key frames. This classification method based on motion features ensures the consistency of the candidate key frame and the current motion state. When the robot is in a straight line motion state, the straight line key frame is selected as the candidate frame, the key frames with unmatched motion states are effectively filtered, the probability of mismatch caused by the difference in motion state is significantly reduced, and the positioning accuracy and reliability of the visual navigation system in the straight line motion scene are improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a first flowchart of a robot-based visual positioning navigation method provided by the embodiment of the present application;

[0028] Figure 2 is a second flowchart of a robot-based visual positioning navigation method provided by the embodiment of the present application;

[0029] Figure 3 is a structural schematic diagram of a robot-based visual positioning navigation device provided by the embodiment of the present application;

[0030] Figure 4 is a structural schematic diagram of an electronic device disclosed by the embodiment of the present application.

[0031] Explanation of reference numerals in the attached drawings: 301, acquisition unit; 302, processing unit; 303, filtering unit; 304, confirmation unit; 400, electronic device; 401, processor; 402, memory; 403, user interface; 404, network interface; 405, communication bus. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0033] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0034] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0035] Therefore, addressing the positioning difficulties caused by repetitive structures, similar textures, and lighting variations in tunnel environments is a pressing problem that needs to be solved. This application provides a robot-based visual positioning and navigation method, applied in a server. The server in this application can be a platform providing positioning and navigation services for robots. Figure 1 This is a first flowchart illustrating a robot-based visual positioning and navigation method according to an embodiment of this application. (Refer to...) Figure 1 The method includes the following steps S101-S109.

[0036] S101: Obtain positioning information from the robot's GPS receiver, determine that the GPS receiver is in a positioning failure state based on the positioning information, and acquire multiple frames of images of the robot in the current environment in real time based on the positioning failure state.

[0037] In S101, after the robot enters the tunnel entrance, the robot acquires positioning information through its own GPS receiver, determines the position of the robot in the tunnel through the received positioning information, and verifies the validity of the positioning information by determining the positioning failure state of the received positioning information, in order to avoid the problem of sudden positioning interruption caused by loss of GPS signal.

[0038] According to the positioning information, it is determined that the GPS receiver is in a positioning failure state, specifically including: analyzing the positioning information to obtain a positioning quality indicator and an accuracy factor; calling a plurality of sets of calibration data of the robot through a plurality of standard test tunnel entrance areas, the calibration data at least including a real-time accuracy factor output by the GPS receiver and a tunnel entry event of the robot entering the tunnel interior; extracting the corresponding real-time accuracy factor at each tunnel entry event occurrence time from the plurality of sets of calibration data to obtain a plurality of historical accuracy factors; statistically calculating the plurality of historical accuracy factors to obtain a mean value and a standard deviation; adding the product of the mean value and the standard deviation multiplied by a preset weight coefficient to obtain an accuracy factor threshold; determining whether the positioning quality indicator exists in the preset failure quality indicator table and whether the accuracy factor is greater than or equal to the accuracy factor threshold; when the positioning quality indicator exists in the preset failure quality indicator table or the accuracy factor is greater than or equal to the accuracy factor threshold, it is determined that the GPS receiver is in a positioning failure state.

[0039] Specifically, the positioning information output by the GPS receiver is parsed. The positioning information is usually in the NMEA-0183 standard format, which contains multiple data frames, and the multiple data frames contain positioning quality indicators and precision factors. For example, when parsing the GPGGA data frame, the positioning quality indicator therein is represented by a number from 0 to 5 to represent different positioning states, 0 represents an invalid solution, 1 represents a single-point positioning solution, 2 represents a differential positioning solution, etc.; and in the GPGSA data frame, the precision factor PDOP value reflects the influence of the current satellite geometric distribution on the positioning accuracy. In order to establish a reliable failure judgment standard, multiple representative standard test tunnel entrance areas are selected, and the robot repeatedly passes through these areas for testing, and calibration data is collected. Each set of calibration data accurately records the change trend of the real-time precision factor output by the GPS receiver when approaching the tunnel entrance, and the time point when the robot actually enters the tunnel. For example, the occurrence of the tunnel entry event can be accurately determined by detecting the tunnel entrance features through the laser radar or by the pre-calibrated geographic coordinates. The real-time precision factor within a certain time window (such as 5 seconds before and after) before and after each tunnel entry event is extracted, and the precision factor value when the signal starts to significantly attenuate is determined as the historical precision factor. Assuming that 5 tests are performed at each of the 10 standard test tunnel entrance areas, 50 sets of historical precision factor data can be obtained. Statistical analysis is performed on these data to calculate the average value (such as 3.5) and the standard deviation (such as 0.8). Considering that different environmental factors have different influences on the GPS signal, a weight coefficient is introduced to adjust the contribution of the average value and the standard deviation. For example, the weight coefficient of the average value can be set to 1.0, and the weight coefficient of the standard deviation can be set to 2.0, so the final precision factor threshold is calculated as: 3.5x1.0+0.8x2.0=5.1.

[0040] During real-time operation, a preset failure quality indicator table is also maintained, which contains quality indicator values (such as 0 and 6) indicating unreliable GPS signals. When it is detected that the current positioning quality indicator appears in the table, or the current precision factor exceeds the calculated threshold value 5.1, it is determined that the GPS receiver is in a positioning failure state. Once it is confirmed that the GPS receiver enters the positioning failure state, the visual acquisition module is immediately started, and multiple frames of images in the current environment are acquired in real time through the camera carried on the robot. Through the above judgment mechanism, not only can the attenuation state of the GPS signal be identified in time to ensure the timely start of the visual navigation system, but also multiple frames of images are acquired to provide a reliable data basis for subsequent visual positioning, effectively avoiding the problem of positioning interruption caused by sudden loss of GPS signals.

[0041] Further, when the positioning quality indicator is not present in the preset invalid quality indicator table and the precision factor is less than the precision factor threshold, it is determined that the GPS receiver is in the positioning valid state. That is, when the positioning quality indicator indicates that the signal is reliable (such as a value of 1 or 2) and the precision factor is lower than the threshold, it is considered that the GPS works in the positioning valid state, and the GPS receiver is continuously used to determine the positioning information of the robot in the tunnel.

[0042] S102: Feature extraction is performed on the plurality of images to obtain a local feature set, and each feature in the local feature set is converted to obtain a visual bag-of-words vector.

[0043] In the above S102, feature extraction is performed on the obtained plurality of images. Specifically, an ORB (Oriented FAST and Rotated BRIEF) feature detection algorithm is used, which has the characteristics of high computational efficiency and insensitivity to image rotation. For each frame of 1920x1080 resolution image, feature detection is performed by setting a feature point number threshold (such as 2000 feature points). In the feature detection process, a FAST corner detector is first used to find potential feature point positions in the image, then the intensity of each feature point is calculated, and the most significant feature point is selected. For each detected feature point, its main direction is calculated, and a local image block is constructed based on the direction, and an improved BRIEF descriptor is used to generate a 256-bit binary feature vector. These feature vectors constitute a local feature set.

[0044] In order to improve the efficiency of feature matching, it is necessary to convert the obtained local feature set into a more compact representation. A visual bag-of-words model is used for feature conversion. First, a dictionary containing 50,000 visual words is obtained by K-means clustering (such as setting K=50000) of feature vectors extracted from a large number of training images. During real-time operation, for each feature vector in the local feature set, the Hamming distance between it and each visual word in the dictionary is calculated, and it is quantized to the nearest visual word. Finally, by counting the frequency of each visual word in the image, a 50,000-dimensional sparse vector is generated as a visual bag-of-words vector. In order to improve the retrieval efficiency, an inverted index structure is used to store the visual bag-of-words vector, and a TF-IDF (Term Frequency-Inverse Document Frequency) weighting method is used to highlight the importance of discriminative visual words. This feature extraction and conversion scheme based on ORB features and visual bag-of-words model not only preserves the local feature information of the image, but also compresses the high-dimensional local features into global descriptors for easy retrieval. By using binary feature descriptors and inverted index structure, the computational efficiency of feature extraction and matching is significantly improved.

[0045] S103: Find multiple candidate keyframes matching the bag-of-visual-words vector from the preset three-dimensional map database.

[0046] In the above S103, the preset three-dimensional map database is constructed by the image sequence collected by the robot in the mapping stage, wherein each keyframe contains not only image data but also corresponding three-dimensional feature point information and pose information. In order to realize efficient retrieval, an inverted index mechanism is used to organize the bag-of-visual-words data structure. Specifically, first, the feature vectors of all keyframes are quantized into a fixed number (such as 100,000) of visual words by the K-means clustering algorithm, and each visual word maintains an inverted list containing all keyframe IDs in which the word appears. For example, assuming that visual word W1 appears in keyframes F3, F78, and F246, then the inverted list of W1 records [F3, F78, F246]. When matching frames need to be retrieved, first find the visual words corresponding to each non-zero element in the bag-of-visual-words vector of the current image, and then quickly obtain the set of candidate keyframes that may match by accessing the inverted lists of these visual words. In order to further improve the accuracy of matching, the standardized bag-of-visual-words cosine similarity is used as the matching metric. For example, if the cosine similarity of two bag-of-visual-words vectors exceeds a preset threshold (such as 0.75), then the two image frames are considered to have sufficient visual similarity. Finally, the top preset number (such as 20) of keyframes with the highest similarity are selected as candidate keyframes for output.

[0047] S104: Calculate the motion trajectory of the robot within a preset time to obtain the current trajectory curvature.

[0048] In the above S104, the motion trajectory of the robot within a preset time is calculated to obtain the current trajectory curvature, which specifically includes: obtaining the angular velocity of the robot rotating along the vertical axis within the preset time, and obtaining the average travel speed within the preset time from the wheel pulse number of the robot; filtering the angular velocity and the average travel speed to obtain a smoothed angular velocity and a smoothed linear velocity; when the smoothed linear velocity is less than a minimum speed threshold, it is confirmed that the robot is in a stationary state, and the corresponding current trajectory curvature in the stationary state is 0; when the smoothed linear velocity is greater than or equal to the minimum speed threshold, the smoothed angular velocity is divided by the smoothed linear velocity to obtain the current trajectory curvature.

[0049] Specifically, the IMU sensor (such as MPU9250) carried by the robot is used to collect the angular velocity data of the robot along the vertical Z-axis within a preset time period (such as 1 second) at a sampling frequency of 100 Hz. The IMU sensor is connected to the main controller of the robot through an SPI interface and outputs angular velocity data every 10 ms, and a total of 100 angular velocity sampling points are collected within 1 second. At the same time, the pulse counts output by the left and right wheel encoders (such as E6B2-CWZ3E model, accuracy 1024 lines / turn) of the robot within 1 second are read. Assuming that the wheel diameter is 200 mm, each pulse corresponds to a travel distance of πd / 1024≈0.613 mm. By calculating the number of pulses per unit time and combining the pulse distance conversion relationship, the average travel speed of the robot is obtained. For example, if the average number of pulses of the left and right wheel encoders within 1 second is 500, the corresponding average travel speed is about 0.307 m / s.

[0050] To improve the reliability of the speed data, a Kalman filter is used to filter the obtained angular velocity and average travel speed. For the angular velocity sequence, a first-order Kalman filter is constructed, the state transition matrix is set to 1, the measurement matrix is set to 1, the initial state covariance is set to 0.1, the process noise covariance is set to 0.005 rad / s, and the measurement noise covariance is set to 0.01 rad / s. The filter performs prediction and update operations every time a new angular velocity measurement value is received, and outputs the smoothed angular velocity value. Similarly, the same structure of Kalman filter is also applied to the average travel speed, but different noise parameters are set: the process noise covariance is 0.02 m / s, and the measurement noise covariance is 0.05 m / s. After filtering, the smoothed angular velocity and smoothed linear velocity sequences are obtained. After obtaining the smoothed speed data, the motion state of the robot is first determined. Based on experimental data analysis, the minimum speed threshold is set to 0.1 m / s. When the smoothed linear velocity is detected to be lower than the minimum speed threshold, it indicates that the robot may be in a stopped state or a small jitter state, and the current trajectory curvature value is directly set to 0 at this time. This processing method avoids the calculation error of the curvature due to the denominator (linear velocity) approaching zero when approaching the static state.

[0051] Further, when the smoothed linear velocity is greater than or equal to the minimum speed threshold 0.1 m / s, it indicates that the robot is in a clear motion state. At this time, the current trajectory curvature is calculated by dividing the smoothed angular velocity by the smoothed linear velocity. For example, if the smoothed angular velocity at a certain time is 0.5 rad / s and the smoothed linear velocity is 1.0 m / s, the calculated trajectory curvature is 0.5 m^(-1), indicating that the robot is performing a circular arc motion with a radius of 2 meters.

[0052] S105: filtering a plurality of candidate key frames according to the current trajectory curvature to obtain an initial candidate key frame set.

[0053] In step S105 above, multiple candidate keyframes are filtered based on the current trajectory curvature to obtain an initial candidate keyframe set. Specifically, this includes: retrieving multiple historical motion trajectories of the robot traveling on multiple standard straight test road segments, calculating multiple trajectory curvature values ​​for each historical motion trajectory; selecting the maximum value from the multiple trajectory curvature values ​​as the straight curvature threshold; if the current trajectory curvature is less than or equal to the straight curvature threshold, the robot's motion trajectory is determined to be a straight motion state; retrieving the historical trajectory curvature corresponding to each of the multiple candidate keyframes, and determining whether the historical trajectory curvature is less than or equal to the straight curvature threshold; when the historical trajectory curvature is less than or equal to the straight curvature threshold, the candidate keyframe is determined to be a straight keyframe; when the historical trajectory curvature is greater than the straight curvature threshold, the candidate keyframe is confirmed to be a non-straight keyframe; selecting sub-candidate keyframes corresponding to the straight keyframes from the multiple candidate keyframes according to the straight motion state, and summarizing the sub-candidate keyframes into the initial candidate keyframe set.

[0054] Specifically, the robot's motion characteristics are analyzed from pre-collected standard straight-line test segment data. These test segments include standard straight lines of different lengths (e.g., 10m, 20m, 50m). The robot repeats the test five times on each segment, recording the motion trajectory data. For each historical trajectory, the same calculation method as the current trajectory curvature is used, i.e., the trajectory curvature value is obtained by dividing the smoothed angular velocity by the smoothed linear velocity. For example, in the five test data obtained on the 50m straight-line segment, the possible trajectory curvature values ​​are 0.02, 0.015, 0.025, 0.018, and 0.022, respectively. The maximum value (e.g., 0.025) is selected from these historical trajectory curvature values ​​as the straight-line curvature threshold. The straight-line curvature threshold reflects the maximum permissible deviation caused by mechanical errors and road surface factors during the robot's straight-line motion. If the current trajectory curvature is less than or equal to the straight-line curvature threshold, it is determined that the robot is performing straight-line motion. At this time, the historical trajectory curvature corresponding to each keyframe in the candidate keyframe set is checked. These historical curvature data are stored in a pre-defined 3D map database and recorded along with the pose information of the keyframes. By comparing the historical trajectory curvature of each candidate keyframe with a straight-line curvature threshold, the candidate keyframes can be classified into straight-line keyframes and non-straight-line keyframes. Specifically, when the historical trajectory curvature of a candidate keyframe is less than or equal to the straight-line curvature threshold (0.025), it is marked as a straight-line keyframe; conversely, when the historical trajectory curvature is greater than the threshold, it is marked as a non-straight-line keyframe.

[0055] Finally, the appropriate keyframes are selected according to the current motion state of the robot. If it is confirmed that the robot is in a straight-line motion state, all frames labeled as straight-line keyframes are filtered from the candidate keyframes and are inducted into the initial candidate keyframe set. This motion feature-based filtering mechanism ensures the consistency of the keyframes with the current motion state, significantly reducing the pose estimation error caused by the mismatch of the motion state.

[0056] For example, if the robot is currently in a straight-line motion state, but a keyframe of a turning segment is mistakenly selected for matching, it may result in a large positioning error. Through motion feature filtering, such errors can be effectively avoided, improving the accuracy and reliability of visual positioning. Practice shows that this keyframe filtering method based on trajectory curvature can reduce the number of candidate keyframes by about 40% while maintaining or even improving the positioning accuracy, significantly improving the real-time performance of the system.

[0057] S106: Obtain the lateral offset of the robot from the center lane line of the tunnel, and obtain the light intensity in real time, and calculate each initial candidate keyframe in the initial candidate keyframe set according to the light intensity and the lateral offset to obtain multiple environment scores.

[0058] In the above S106, the tunnel image is collected by a binocular camera (such as ZED2 camera, resolution 1920x1080) installed at the front end of the robot. The image is processed by a traditional image processing method to perform adaptive binaryzation (OTSU algorithm, threshold range set to 0-255) and extract the center lane line of the tunnel. Then, the Hough transform is used to detect the straight-line feature to obtain the pixel coordinates of the center lane line. The pixel coordinates are converted to the robot coordinate system in combination with the intrinsic matrix and installation pose parameters of the camera, and the lateral offset distance of the robot from the center lane line is calculated. In actual application, the lateral offset distance can also be the lateral offset distance of the robot from the lane line or the guide line, for example, if the current position of the robot is 0.5 meters away from the right side of the center lane line, the lateral offset value is recorded as +0.5 meters (positive value indicating right deviation, negative value indicating left deviation). At the same time, the ambient light intensity is collected in real time by a light sensor (such as BH1750, measurement range 1-65535 lux) installed at the top of the robot. The light sensor samples at a frequency of 10 Hz, and each sampling obtains a light intensity value. In order to reduce the influence of instantaneous fluctuations, the last 10 sampling values within 1 second are processed by a sliding average to obtain the current light intensity value. For example, the average light intensity measured at a certain time is 500 lux, indicating that the current lighting environment is relatively dark.

[0059] For each frame in the initial candidate keyframe set, the initial candidate keyframes in the initial candidate keyframe set are calculated according to the light intensity and the lateral offset to obtain multiple environment scores, such as Figure 2As shown, the method includes the following steps S201-S208:

[0060] S201: Retrieve the historical ray brightness and historical lateral offset corresponding to the initial candidate keyframe of the target; calculate the difference between the ray brightness and the historical ray brightness to obtain the brightness difference; calculate the brightness difference based on the preset brightness scoring function to obtain the brightness similarity score, wherein the brightness similarity score is negatively correlated with the absolute value of the brightness difference.

[0061] In step S201 above, historical environmental information for each target's initial candidate keyframe is retrieved from the database. For light intensity, during the mapping phase, a light sensor (such as a BH1750) mounted on the robot records the ambient light intensity at the time of each keyframe acquisition, with an accuracy of 1 lux. For example, the historical light intensity of a target's initial candidate keyframe might be 500 lux, indicating a relatively bright indoor environment at that time. The difference between the current light intensity and the historical light intensity is calculated: the brightness difference is 450-500=-50 lux, and the offset difference is 30-50=-20 mm. These differences reflect the degree of change between the current environment and the historical acquisition environment. To convert the differences into a standardized score, a dedicated scoring function is designed. For brightness similarity scoring, a Gaussian function is used: score_light=exp(-|brightness difference| / 100), where 100 is the characteristic scale of the brightness difference. Thus, when the brightness difference is 50 lux, the brightness similarity score is approximately 0.61; when the difference increases to 200 lux, the score decreases to 0.14, demonstrating the characteristic that the greater the difference, the lower the score.

[0062] S202: Calculate the difference between the horizontal offset and the historical horizontal offset to obtain the offset difference; calculate the offset similarity score based on the preset offset scoring function, where the offset similarity score is negatively correlated with the absolute value of the offset difference.

[0063] In step S202 above, the lateral offset value of the robot relative to the reference path during keyframe acquisition is also recorded. This lateral offset value is obtained through LiDAR ranging with an accuracy of 1mm. Assuming a historical lateral offset of 50mm for a certain keyframe, it means the robot deviated 50mm from the centerline of the reference path at that time. During real-time operation, the ambient light intensity and lateral offset data are first acquired. The ambient light intensity of the current scene is acquired in real time using a light sensor of the same model, for example, if measured to be 450 lux; the current lateral offset is measured to be 30mm by LiDAR. For the offset similarity score, a Gaussian function is also used: score_offset=exp(-|offset difference| / 30), where 30 is the feature scale of the offset difference (unit: mm). For example, when the offset difference is 20mm, the obtained offset similarity score is approximately 0.51.

[0064] S203: Perform weighted summation on the brightness similarity score and the offset similarity score to obtain a preliminary environment score of the target initial candidate key frame; obtain the preliminary environment score corresponding to each target initial candidate key frame in the initial candidate key frame set.

[0065] In the above S203, the two scores are combined into a single environment score. Considering the important influence of light conditions on visual feature matching, the weight of the brightness similarity score is set to 0.6, and the weight of the offset similarity score is set to 0.4. For each target initial candidate key frame, its preliminary environment score is obtained by weighted summation: final_score = 0.6 × score_light + 0.4 × score_offset. For example, for the above example, the final preliminary environment score is 0.6 × 0.61 + 0.4 × 0.51 = 0.57. The same scoring calculation process is performed for each key frame in the initial candidate key frame set, and finally a set of preliminary environment scores is obtained. These scores provide a quantitative indicator of the similarity of the candidate key frames to the current scene in terms of environmental conditions, providing an important basis for subsequent key frame screening.

[0066] S204: Sort the plurality of preliminary environment scores, and select the top preset second number of initial candidate key frames from the sorting result to form a preliminary candidate set.

[0067] In the above S204, a quicksort algorithm is used to sort all preliminary environment scores (assuming there are 30) in descending order. After sorting, the top preset second number (such as 10) of initial candidate key frames with the highest scores are selected to form a preliminary candidate set. For example, if the top 10 scores after sorting are 95, 93, 92, 90, 89, 88, 87, 86, 85, and 84, these corresponding initial candidate key frames will form the preliminary candidate set.

[0068] S205: Calculate the score difference between the highest score and the lowest score in the preliminary candidate set to obtain a score separation degree.

[0069] In the above S205, the difference between the highest score (95) and the lowest score (84) in the preliminary candidate set is calculated to obtain a score separation degree of 11. This score separation degree reflects the degree of environmental feature difference between the candidate frames.

[0070] S206: When the score separation degree is less than a preset separation threshold, retrieve the historical environment temperature corresponding to the target initial candidate key frame; obtain the current environment temperature of the robot, and calculate the difference between the historical environment temperature and the current environment temperature to obtain a temperature difference.

[0071] In the above S206, the score difference is compared with a preset difference threshold (e.g. 15 points), which is used to determine whether additional environmental parameters need to be introduced for score supplement. When the score difference is less than the preset difference threshold (e.g. 11 points in this example), it means that the lateral offset and the light intensity cannot effectively distinguish these candidate frames. At this time, the historical environmental temperature data corresponding to each initial candidate key frame needs to be called. These temperature data are collected by a temperature sensor (e.g. DS18B20, accuracy ±0.5℃) during the mapping stage and stored in the database. At the same time, the current environmental temperature is collected in real time by the same type of temperature sensor installed on the robot. In order to reduce the fluctuation of temperature measurement, the temperature data collected in the last 10 seconds (sampling frequency 1 Hz) are averaged to obtain a stable current environmental temperature value.

[0072] S207: Calculate the temperature difference based on a preset temperature scoring function to obtain a temperature similarity score; and perform weighted summation of the preliminary environmental score and the temperature similarity score to obtain an environmental score.

[0073] In the above S207, for each candidate frame, the difference between its historical environmental temperature and the current environmental temperature is calculated. For example, the historical temperature of a certain candidate frame is 25℃, and the current temperature is 23℃, so the temperature difference is 2℃. The temperature difference is converted by a preset temperature scoring function to obtain a temperature similarity score. The temperature scoring function is defined as: temperature similarity score = 100 - y x |temperature difference|, where y is the temperature weight coefficient, which is set to 5 through experiments. For the above example, the temperature similarity score = 100 - 5 x 2 = 90 points. Finally, the preliminary environmental score and the temperature similarity score are weighted and summed to obtain the final environmental score. The weighted formula is: environmental score = 0.7 x preliminary environmental score + 0.3 x temperature similarity score. For example, the preliminary environmental score of a certain candidate frame is 95 points, and the temperature similarity score is 90 points, so the final environmental score = 0.7 x 95 + 0.3 x 90 = 93.5 points.

[0074] S208: When the score difference is greater than or equal to the preset difference threshold, the preliminary environmental score is output as the environmental score corresponding to the target initial candidate key frame.

[0075] In the above S208, when the score difference is greater than or equal to the preset difference threshold (e.g. the score difference is 16 points, which is greater than the threshold 15 points), it means that the preliminary environmental score can effectively distinguish different candidate frames, so the preliminary environmental score is directly output as the final environmental score without the need for supplementary calculation of temperature similarity.

[0076] S107: Sort the plurality of environment scores, and select a preset first number of initial candidate key frames from the sorting result and induce into the target candidate key frame set.

[0077] In the above S107, in order to select the most suitable matching frame from a large number of initial candidate key frames, it is necessary to systematically sort and screen the plurality of environment scores calculated in the foregoing steps. First, each initial candidate key frame and its corresponding environment score are organized into a key-value pair form and stored in an array. For example, assuming that the initial candidate key frame set contains 50 frames, 50 key-value pairs are formed, where the key is the unique identifier of the key frame (such as frame_id), and the value is the environment score corresponding to the frame. The key-value pairs are sorted in descending order of environment score using a quicksort algorithm, and the key frame with the highest score is placed in the front row. Specifically, if the environment scores of some key frames are 0.92, 0.85, 0.88, 0.95, and 0.78, respectively, the sorted result will be 0.95, 0.92, 0.88, 0.85, and 0.78.

[0078] Based on analysis of a large amount of experimental data, the preset first number is set to 10, that is, the top 10 initial candidate key frames with the highest environment scores are selected from the sorted result. The setting of this number takes into account the limitation of computing resources, and also ensures that there are enough candidate frames for subsequent precise matching. The 10 initial candidate key frames with the highest scores are extracted, and their identifiers and corresponding pose information are induced into the target candidate key frame set. For example, if the selected key frame environment scores are 0.95, 0.92, 0.88, etc., these key frames and their related information will be saved to the target candidate key frame set for subsequent precise pose estimation. The above screening method can reduce the number of candidate key frames from 50 to 10, while maintaining or even improving the accuracy of final positioning, significantly improving the real-time performance of the visual navigation system.

[0079] S108: Match the feature points of the target frame image with the feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs, use a preset algorithm to calculate the hypothesis camera pose based on the set of initial matching pairs, and perform projection test on all initial matching pairs based on the hypothesis camera pose to count the number of inliers.

[0080] In S108, the feature points of the target frame image are matched with the feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs. A preset algorithm is used to calculate the set of initial matching pairs to obtain a hypothesis camera pose. The hypothesis camera pose is used to perform projection testing on all initial matching pairs, and the number of inliers is counted. Specifically, random sampling and calculation are performed on the set of initial matching pairs to obtain a plurality of candidate camera poses. Two-dimensional feature points in the target frame image that contain the initial matching pairs are retrieved. Three-dimensional map points associated with the initial matching pairs are found according to the initial matching pairs and the target candidate key frame. The initial matching pairs are any one of the set of initial matching pairs. The three-dimensional map points are projected and transformed based on the target candidate camera pose to obtain a re-projection point in the coordinate system of the target frame image. The target candidate camera pose is any one of the plurality of candidate camera poses. The geometric distance between the two-dimensional feature points and the re-projection point is calculated to obtain a re-projection error. If the re-projection error is less than or equal to a preset re-projection error threshold, the initial matching pair is determined to be an inlier of the target candidate camera pose. The total number of initial matching pairs determined to be inliers in the set of initial matching pairs is accumulated, and the total number is used as the inlier score of the target candidate camera pose. After the inlier scores of all candidate camera poses are calculated, the candidate camera pose with the maximum inlier score is selected from the plurality of inlier scores as the hypothesis camera pose, and the maximum inlier score is used as the number of inliers.

[0081] Specifically, the ORB feature extraction algorithm is used to extract feature points from the target frame image and the target candidate key frame. For each image, 1000 feature points are extracted, and each feature point contains a 256-bit binary descriptor. The similarity between feature points is measured by Hamming distance, and feature matching is performed using nearest neighbor ratio testing (the NNDR threshold is set to 0.7) to obtain a set of initial matching pairs. For example, if 200 pairs of feature points are matched between the target frame and a certain target candidate key frame, these matches constitute the initial matching pair set.

[0082] The RANSAC (Random Sample Consensus) algorithm is used to estimate the pose of the initial matching pairs. In each RANSAC iteration, the system randomly samples a minimum number of matching pairs from the initial matching pairs (the PnP problem requires at least 4 matching points). For example, 4 matching points are randomly selected, including their coordinates on the image plane and their corresponding 3D spatial coordinates. The EPnP algorithm is used to solve for the camera pose corresponding to these sampled points, resulting in a candidate camera pose containing a rotation matrix R (3×3) and a translation vector t (3×1). For each initial matching pair, reprojection error calculation is required to verify the accuracy of the candidate camera pose. First, the coordinates P(X,Y,Z) of the 3D map point associated with the current matching pair are retrieved from the database. Then, the candidate camera pose is used to perform a projection transformation on the 3D point: p'=K[R|t]P, where K is the camera intrinsic matrix (3×3), and p' is the coordinates of the reprojected point on the image plane. For example, if a 3D point has coordinates (1.5, 2.0, 5.0) meters, after projection transformation, it becomes an image with coordinates (320, 240) pixels. The Euclidean distance between the reprojected point p' and the actually observed 2D feature point p is calculated as: d = ||p' - p||_2. A preset reprojection error threshold of 2 pixels is set. If the calculated distance d is less than or equal to this threshold, the initial matching pair is marked as an inlier. For example, if the reprojection distance is 1.5 pixels, less than the threshold of 2 pixels, the matching pair is confirmed as an inlier supporting the current candidate camera pose. The above reprojection test process is repeated for each candidate camera pose, and the number of inliers that pass the test is counted. This number is used as the inlier score of the candidate pose. For example, if 150 out of 200 initial matches of a candidate camera pose pass the reprojection test, the inlier score of that pose is 150. After completing all RANSAC iterations (e.g., setting the number of iterations to 200), the pose with the highest interior point score is selected from all candidate camera poses as the final hypothetical camera pose.

[0083] S109: Select the pose corresponding to the final candidate keyframe with the maximum number of inliers as the robot's position information at the current moment.

[0084] In the above S109, each frame key frame in the target candidate key frame set (assuming 10 frames) and its corresponding number of inliers are compared. For example, the number of inliers of these key frames are 150, 142, 138, 135, 128, 125, 120, 118, 115, and 110, respectively. The inlier numbers are sorted in descending order by a quick sort algorithm, the maximum value of inlier number (150 in this example) corresponding to the target candidate key frame is found, and the target candidate key frame is determined as the final candidate key frame. Subsequently, the hypothesis camera pose corresponding to the final candidate key frame is taken as the position information of the robot at the current time. This position information contains complete six-degree-of-freedom attitude parameters: three translation components (tx, ty, tz) and three rotation angles (roll, pitch, yaw). For example, if the hypothesis camera pose corresponding to the final candidate key frame is: translation vector [0.5 m, 0.2 m, 0.1 m] and Euler angle [1°, 0.5°, 2°], then this set of parameters will be used as the optimal estimate of the current position of the robot.

[0085] The inlier number directly reflects the quality of feature matching and the reliability of pose estimation through the above steps, and selecting the key frame with the most inliers can ensure the accuracy of pose estimation; by using the maximum inlier number as a criterion, the best pose estimation result can be automatically selected. Practice shows that the above method performs excellently in complex tunnel environments, with positioning accuracy reaching centimeter level (average error ±3 cm), attitude estimation error controlled within 1 degree, and the reliability and accuracy of the visual navigation system significantly improved. For example, in a test of continuous running of 100 meters, the cumulative positioning error of the above method does not exceed 10 cm, meeting the accuracy requirements of robot navigation in tunnel scenarios.

[0086] The embodiment of the present application also provides a visual positioning and navigation device based on a robot, Figure 3 is a structural schematic diagram of a visual positioning and navigation device based on a robot provided by the embodiment of the present application, referring to Figure 3 The device comprises an acquisition unit 301, a processing unit 302, a screening unit 303, and a confirmation unit 304.

[0087] The acquisition unit 301 acquires positioning information from a GPS receiver of the robot, determines that the GPS receiver is in a positioning failure state according to the positioning information, and acquires multiple frames of images of the robot in a current environment in real time based on the positioning failure state;

[0088] The processing unit 302 performs feature extraction on the multiple frames of images to obtain a local feature set, converts each feature in the local feature set to obtain a visual bag-of-words vector, searches a preset three-dimensional map database for multiple candidate key frames matched with the visual bag-of-words vector, and calculates a motion trajectory of the robot within a preset time to obtain a current trajectory curvature.

[0089] The screening unit 303 screens a plurality of candidate key frames according to a current trajectory curvature to obtain an initial candidate key frame set; obtains a lateral offset of the robot from a center lane line of the tunnel, and obtains a light brightness in real time; calculates each initial candidate key frame in the initial candidate key frame set according to the light brightness and the lateral offset to obtain a plurality of environment scores, one initial candidate key frame corresponding to one environment score; sorts the plurality of environment scores, and selects a preset first number of initial candidate key frames from the sorting result to induce into a target candidate key frame set;

[0090] The confirmation unit 304 matches feature points of a target frame image with feature points of each target candidate key frame in the target candidate key frame set to obtain a set of initial matching pairs, calculates the set of initial matching pairs using a preset algorithm to obtain a hypothesis camera pose, performs projection testing on all initial matching pairs based on the hypothesis camera pose, counts an inlier number, and the target frame image is a frame image closest to a current time among a plurality of frame images; selects a pose corresponding to a final candidate key frame with a maximum inlier number as position information of the robot at the current time.

[0091] In a possible implementation, the processing unit 302 is configured to call a historical light brightness and a historical lateral offset corresponding to a target initial candidate key frame, the target initial candidate key frame being any one of the initial candidate key frames in the initial candidate key frame set; calculates a difference between the light brightness and the historical light brightness to obtain a brightness difference; calculates the brightness difference based on a preset brightness score function to obtain a brightness similarity score, wherein the brightness similarity score and an absolute value of the brightness difference are in a negative correlation relationship; calculates a difference between the lateral offset and the historical lateral offset to obtain an offset difference; calculates the offset difference based on a preset offset score function to obtain an offset similarity score, wherein the offset similarity score and an absolute value of the offset difference are in a negative correlation relationship; and performs weighted summation on the brightness similarity score and the offset similarity score to obtain a preliminary environment score of the target initial candidate key frame; obtains the preliminary environment score corresponding to each target initial candidate key frame in the initial candidate key frame set, and outputs the preliminary environment score as the environment score.

[0092] In a possible implementation, the screening unit 303 is configured to sort the plurality of preliminary environment scores, and select a preset second number of initial candidate key frames with the highest scores from the sorting result to form a preliminary candidate set; the processing unit is configured to calculate a score difference between the highest score and the lowest score in the preliminary candidate set to obtain a score granularity; when the score granularity is less than a preset granularity threshold, the target initial candidate key frame corresponding historical environment temperature is called; the obtaining unit 301 is configured to obtain the current environment temperature of the robot, and calculate a temperature difference between the historical environment temperature and the current environment temperature to obtain a temperature difference value; the processing unit 302 is configured to calculate the temperature difference value based on a preset temperature score function to obtain a temperature similarity score; the preliminary environment score and the temperature similarity score are weighted and summed to obtain the environment score; when the score granularity is greater than or equal to the preset granularity threshold, the preliminary environment score is output as the environment score corresponding to the target initial candidate key frame.

[0093] In a possible implementation, the obtaining unit 301 is configured to obtain an angular velocity of the robot rotating along a vertical axis within a preset time, and obtain an average driving speed within the preset time from a wheel type pulse number of the robot; the processing unit 302 is configured to filter the angular velocity and the average driving speed to obtain a smoothed angular velocity and a smoothed linear velocity; the confirming unit 304 is configured to confirm that the robot is in a static state when the smoothed linear velocity is less than a minimum speed threshold, and the corresponding current trajectory curvature in the static state is 0; when the smoothed linear velocity is greater than or equal to the minimum speed threshold, the smoothed angular velocity is divided by the smoothed linear velocity to obtain the current trajectory curvature.

[0094] In a possible implementation, the obtaining unit 301 is configured to call a plurality of historical motion trajectories of the robot driving on a plurality of standard straight line test road segments, and calculate each historical motion trajectory to obtain a plurality of trajectory curvature values; the processing unit 302 is configured to select a maximum value from the plurality of trajectory curvature values as a straight line curvature threshold; if the current trajectory curvature is less than or equal to the straight line curvature threshold, it is determined that the motion trajectory of the robot is in a straight line motion state; the historical trajectory curvatures corresponding to the plurality of candidate key frames are called, and it is judged whether the historical trajectory curvatures are less than or equal to the straight line curvature threshold; when the historical trajectory curvature is less than or equal to the straight line curvature threshold, the candidate key frame is determined as a straight line key frame; when the historical trajectory curvature is greater than the straight line curvature threshold, the candidate key frame is determined as a non-straight line key frame; the screening unit 303 is configured to select a sub-candidate key frame corresponding to the straight line key frame from the plurality of candidate key frames according to the straight line motion state, and induct the sub-candidate key frame into the initial candidate key frame set.

[0095] In one possible implementation, the processing unit 302 is used to randomly sample and calculate from a set of initial matching pairs to obtain multiple candidate camera poses; retrieve two-dimensional feature points containing the initial matching pairs in the target frame image, and find associated three-dimensional map points based on the initial matching pairs and the target candidate keyframes, wherein the initial matching pairs are any one of the initial matching pairs in the set; perform projection transformation on the three-dimensional map points based on the target candidate camera poses to obtain a reprojection point located in the coordinate system of the target frame image, wherein the target candidate camera pose is any one of the multiple candidate camera poses; calculate the geometric distance between the two-dimensional feature points and the reprojection point to obtain the reprojection error; if the reprojection error is less than or equal to a preset reprojection error threshold, then the initial matching pairs are determined as inliers supporting the target candidate camera poses; accumulate the total number of initial matching pairs determined as inliers in a set of initial matching pairs, and use the total number as the inlier score of the target candidate camera pose; after completing the calculation of the inlier scores for all candidate camera poses, select the candidate camera pose with the largest inlier score from the multiple inlier scores as the hypothetical camera pose, and use the largest inlier score as the number of inliers.

[0096] In one possible implementation, the processing unit 302 is used to parse the positioning information to obtain positioning quality indicators and accuracy factors; retrieve multiple sets of calibration data of the robot passing through the entrance areas of multiple standard test tunnels, the calibration data including at least one real-time accuracy factor output by the GPS receiver and a tunnel entry event of the robot entering the tunnel; extract the real-time accuracy factors corresponding to the occurrence time of each tunnel entry event from the multiple sets of calibration data to obtain multiple historical accuracy factors; perform statistical calculations on the multiple historical accuracy factors to obtain the average value and standard deviation; multiply the average value and standard deviation by a preset weighting coefficient and add them together to obtain the accuracy factor threshold; determine whether there is a positioning quality indicator in the preset failure quality indicator table and whether the accuracy factor is greater than or equal to the accuracy factor threshold; the confirmation unit 304 is used to determine that the GPS receiver is in a positioning failure state when there is a positioning quality indicator in the preset failure quality indicator table or the accuracy factor is greater than or equal to the accuracy factor threshold.

[0097] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0098] This application also discloses an electronic device. (See reference...) Figure 4 , Figure 4A structural schematic diagram of an electronic device is provided for the embodiments of the present application. The electronic device 400 can include at least one processor 401, at least one network interface 404, a user interface 403, a memory 402, and at least one communication bus 405.

[0099] The communication bus 405 is configured to realize the connection and communication between the components.

[0100] The user interface 403 can include a display and a camera. Optionally, the user interface 403 can further include a standard wired interface and a wireless interface.

[0101] The network interface 404 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0102] The processor 401 can include one or more processing cores. The processor 401 is connected to various parts of the server through various interfaces and lines, and performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 402, and calling data stored in the memory 402. Optionally, the processor 401 can be implemented in at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 401 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU is mainly used to process operating systems, user interfaces, and application requests. The GPU is responsible for rendering and drawing the content to be displayed on the display. The modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 401, but can be realized by a separate chip.

[0103] The memory 402 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory 402 includes a non-transitory computer-readable storage medium. The memory 402 can be configured to store instructions, programs, codes, code sets, or instruction sets. The memory 402 can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned method embodiments, etc. The data storage area can store data related to the above-mentioned method embodiments, etc. The memory 402 can also be at least one storage device located away from the processor 401.

[0104] As shown in Figure 4 , the memory 402 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an application program based on visual positioning navigation of a robot.

[0105] In Figure 4 , the user interface 403 is mainly configured to provide an interface for user input and obtain data input by the user. The processor 401 can be configured to invoke the application program based on visual positioning navigation of a robot stored in the memory 402, and when executed by one or more processors, cause the electronic device to perform the method described in one or more of the above embodiments.

[0106] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0107] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0108] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely illustrative, and the units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, and can be in electrical or other forms.

[0109] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0110] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0111] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic or optical disk, and various program code storage media.

[0112] The above is only an exemplary embodiment of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional techniques in the art not described in the present disclosure.

Claims

1. A robot-based vision-based localization and navigation method, characterized in that, The method includes: The robot obtains positioning information from its GPS receiver, determines that the GPS receiver is in a positioning failure state based on the positioning information, and acquires multiple frames of images of the robot in the current environment in real time based on the positioning failure state. Feature extraction is performed on multiple frames of the image to obtain a local feature set. Each feature in the local feature set is transformed to obtain a visual bag-of-words vector. Search a preset 3D map database for multiple candidate keyframes that match the visual bag-of-words vector; The current trajectory curvature is obtained by calculating the robot's motion trajectory within a preset time. Based on the current trajectory curvature, multiple candidate keyframes are filtered to obtain an initial set of candidate keyframes; The lateral offset between the robot and the center lane line of the tunnel is obtained, and the light intensity is obtained in real time. Based on the light intensity and the lateral offset, each initial candidate keyframe in the initial candidate keyframe set is calculated to obtain multiple environmental scores. Each initial candidate keyframe corresponds to one environmental score. The environmental scores are sorted, and a preset first number of initial candidate keyframes are selected from the sorting results and summarized into the target candidate keyframe set. The feature points of the target frame image are matched with the feature points of each target candidate keyframe in the target candidate keyframe set to obtain a set of initial matching pairs. The assumed camera pose is calculated using a preset algorithm on the set of initial matching pairs. Based on the assumed camera pose, a projection test is performed on all initial matching pairs to count the number of inliers. The target frame image is the frame image closest to the current time among the multiple frames. The pose corresponding to the final candidate keyframe with the maximum number of inliers is selected as the position information of the robot at the current moment.

2. The method according to claim 1, characterized in that, The step of calculating multiple environmental scores for each initial candidate keyframe in the initial candidate keyframe set based on the light intensity and the lateral offset specifically includes: Retrieve the historical light intensity and historical lateral offset corresponding to the target initial candidate keyframe, wherein the target initial candidate keyframe is any one of the initial candidate keyframes in the set of initial candidate keyframes; Calculate the difference between the light intensity and the historical light intensity to obtain the intensity difference; The brightness difference is calculated based on a preset brightness scoring function to obtain a brightness similarity score, wherein the brightness similarity score is negatively correlated with the absolute value of the brightness difference. Calculate the difference between the lateral offset and the historical lateral offset to obtain the offset difference; The offset difference is calculated based on a preset offset scoring function to obtain an offset similarity score, wherein the offset similarity score is negatively correlated with the absolute value of the offset difference; The brightness similarity score and the offset similarity score are weighted and summed to obtain the preliminary environment score of the target initial candidate keyframe; Obtain the preliminary environment score corresponding to each target initial candidate keyframe in the initial candidate keyframe set, and output the preliminary environment score as the environment score.

3. The method according to claim 2, characterized in that, The step of outputting the preliminary environmental score as the environmental score specifically includes: The preliminary environmental scores are sorted, and the highest-scoring second number of initial candidate keyframes are selected from the sorting results to form a preliminary candidate set. Calculate the score difference between the highest and lowest scores in the preliminary candidate set to obtain the score discrimination. When the scoring discrimination is less than a preset discrimination threshold, the historical ambient temperature corresponding to the initial candidate keyframe of the target is retrieved. The current ambient temperature of the robot is obtained, and the difference between the historical ambient temperature and the current ambient temperature is calculated to obtain the temperature difference. The temperature difference is calculated based on a preset temperature scoring function to obtain a temperature similarity score; The environmental score is obtained by weighted summation of the preliminary environmental score and the temperature similarity score. When the scoring discrimination is greater than or equal to the preset discrimination threshold, the preliminary environmental score is output as the environmental score corresponding to the target initial candidate keyframe.

4. The method according to claim 1, characterized in that, The step of calculating the robot's motion trajectory within a preset time to obtain the current trajectory curvature specifically includes: The angular velocity of the robot rotating along the vertical axis within a preset time period is obtained, and the average travel speed within the preset time period is obtained from the number of wheel pulses of the robot. The angular velocity and the average driving speed are filtered to obtain smoothed angular velocity and smoothed linear velocity. When the smooth linear velocity is less than the minimum velocity threshold, it is confirmed that the robot is in a stationary state, and the curvature of the current trajectory corresponding to the stationary state is 0. When the smooth linear velocity is greater than or equal to the minimum velocity threshold, the smooth angular velocity is divided by the smooth linear velocity to obtain the current trajectory curvature.

5. The method according to claim 4, characterized in that, The step of filtering multiple candidate keyframes based on the current trajectory curvature to obtain an initial candidate keyframe set specifically includes: The robot retrieves multiple historical motion trajectories from its travel on multiple standard straight test road sections, calculates multiple trajectory curvature values ​​for each historical motion trajectory, and retrieves multiple historical motion trajectories. The maximum value among the plurality of trajectory curvature values ​​is selected as the straight line curvature threshold. If the curvature of the current trajectory is less than or equal to the straight line curvature threshold, then the robot's motion trajectory is determined to be a straight line motion state; Retrieve the historical trajectory curvature corresponding to each of the multiple candidate keyframes, and determine whether the historical trajectory curvature is less than or equal to the straight line curvature threshold. When the curvature of the historical trajectory is less than or equal to the curvature threshold of the straight line, the candidate keyframe is determined as a straight line keyframe; When the curvature of the historical trajectory is greater than the straight line curvature threshold, the candidate keyframe is determined as a non-straight line keyframe. Based on the linear motion state, sub-candidate keyframes corresponding to the linear keyframes are selected from multiple candidate keyframes, and the sub-candidate keyframes are summarized into the initial candidate keyframe set.

6. The method according to claim 1, characterized in that, The process involves matching feature points of the target frame image with feature points of each target candidate keyframe in the target candidate keyframe set to obtain an initial matching pair. A preset algorithm is then used to calculate the assumed camera pose from this initial matching pair. Based on the assumed camera pose, a projection test is performed on all initial matching pairs, and the number of inliers is counted. Specifically, this includes: Multiple candidate camera poses are obtained by randomly sampling and calculating from the initial set of matching pairs; Retrieve two-dimensional feature points containing the initial matching pair in the target frame image, and find associated three-dimensional map points based on the initial matching pair and the target candidate keyframe. The initial matching pair is any one of the initial matching pairs in the set. Based on the target candidate camera pose, the 3D map point is projected and transformed to obtain a reprojected point located in the target frame image coordinate system. The target candidate camera pose is multiple, and the candidate camera pose is any one of the camera poses. Calculate the geometric distance between the two-dimensional feature point and the reprojection point to obtain the reprojection error; If the reprojection error is less than or equal to a preset reprojection error threshold, then the initial matching pair is determined as an interior point that supports the pose of the target candidate camera. The total number of initial matching pairs that are determined to be interior points in the initial matching pairs is accumulated, and the total number is used as the interior point score of the target candidate camera pose; After calculating the inlier scores for all candidate camera poses, the candidate camera pose with the highest inlier score is selected from the multiple inlier scores as the assumed camera pose, and the highest inlier score is used as the number of inliers.

7. The method according to claim 1, characterized in that, The step of determining that the GPS receiver is in a positioning failure state based on the positioning information specifically includes: The positioning information is parsed to obtain a positioning quality indicator and an accuracy factor; The robot retrieves multiple sets of calibration data from the entrance areas of several standard test tunnels. The calibration data includes at least a real-time accuracy factor output by the GPS receiver and a tunnel entry event indicating that the robot has entered the tunnel. The real-time accuracy factor corresponding to the occurrence time of each tunnel entry event is extracted from multiple sets of calibration data to obtain multiple historical accuracy factors. Statistical calculations are performed on multiple historical precision factors to obtain the mean and standard deviation; the product of the mean and the standard deviation multiplied by a preset weighting coefficient is added to obtain the precision factor threshold; Determine whether the positioning quality indicator exists in the preset failure quality indicator table, and whether the accuracy factor is greater than or equal to the accuracy factor threshold; When the positioning quality indicator exists in the preset failure quality indicator table, or when the accuracy factor is greater than or equal to the accuracy factor threshold, the GPS receiver is determined to be in the positioning failure state.

8. A robot-based visual positioning and navigation device, characterized in that, The device includes an acquisition unit, a processing unit, a filtering unit, and a confirmation unit. The acquisition unit acquires positioning information from the robot's GPS receiver, determines that the GPS receiver is in a positioning failure state based on the positioning information, and acquires multiple frames of images of the robot in the current environment in real time based on the positioning failure state. The processing unit extracts features from multiple frames of the images to obtain a local feature set, transforms each feature in the local feature set to obtain a visual bag-of-words vector; searches for multiple candidate keyframes that match the visual bag-of-words vector from a preset 3D map database; and calculates the robot's motion trajectory within a preset time to obtain the current trajectory curvature. The filtering unit filters multiple candidate keyframes based on the current trajectory curvature to obtain an initial set of candidate keyframes. The lateral offset between the robot and the center lane line of the tunnel is obtained, and the light intensity is obtained in real time. Based on the light intensity and the lateral offset, each initial candidate keyframe in the initial candidate keyframe set is calculated to obtain multiple environmental scores, with one initial candidate keyframe corresponding to one environmental score. The multiple environmental scores are sorted, and a preset first number of initial candidate keyframes are selected from the sorting results and summarized into the target candidate keyframe set. The confirmation unit matches the feature points of the target frame image with the feature points of each target candidate keyframe in the target candidate keyframe set to obtain a set of initial matching pairs. It then uses a preset algorithm to calculate the hypothetical camera pose of the set of initial matching pairs and performs projection tests on all initial matching pairs based on the hypothetical camera pose to count the number of inliers. The target frame image is the frame image closest to the current time among the multiple frames. The pose corresponding to the final candidate keyframe with the maximum number of inliers is selected as the position information of the robot at the current moment.

9. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images