Robot repositioning methods, devices and robots

Abstract

This invention discloses a robot relocalization method, apparatus, and robot. In the method, laser point cloud data and image data acquired by the robot at its current position are received; keyframe information corresponding to the image data is determined according to a visual dictionary; physical location information corresponding to the keyframe information is obtained from a visual map; feature point information of the image data is matched with feature point information corresponding to the keyframe information; and calculation is performed by combining the physical location information of the feature points in the visual map and the pixel location information of the feature points in the image data. After obtaining the robot's first pose data, a predetermined area is set based on the first pose data, and iterative matching calculations are performed within the predetermined area using the laser map and laser point cloud data to obtain the robot's second pose data. By using the above scheme, the iteration range of laser matching can be reduced by utilizing the initial pose data, thus accelerating the matching speed and quickly achieving robot relocalization.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a robot relocation method, apparatus, and robot. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is one of the most widely used robot localization technologies.

[0003] During use, robots may be picked up, kicked, or slip, which could cause them to lose their localization capabilities. In such cases, relocalization is necessary. Relocalization is a crucial foundation for intelligent navigation and environmental exploration in robots, and one of the key technologies for achieving true full autonomy in mobile robots. In SLAM technology, robot relocalization is a vital step, playing a key role in map reuse.

[0004] In related technologies, laser relocation is the primary method, which calculates the robot's position and orientation by matching the current laser beam with an existing map. However, in practical applications, especially in complex, large-scale environments where a full map search is required, laser relocation is insufficient for rapid relocation. Summary of the Invention

[0005] The main objective of this invention is to disclose a robot relocation method, apparatus, and robot, so as to at least solve the problems in related technologies such as the difficulty in quickly relocating robots in complex large-scale environments where a full map search is required and laser relocation schemes are used.

[0006] According to one aspect of the present invention, a robot relocation method is provided.

[0007] The robot relocalization method according to the present invention includes: receiving laser point cloud data and image data acquired by the robot at its current position; determining image keyframe information corresponding to the image data according to a visual dictionary; acquiring physical location information corresponding to the image keyframe information in a visual map; matching the feature point information of the image data with the feature point information corresponding to the image keyframe information; performing calculation by combining the physical location information of the feature points in the matched visual map and the pixel location information of the feature points in the image data; and, in the case of obtaining the robot's first pose data through calculation, setting a predetermined area range according to the first pose data; performing iterative matching calculation within the predetermined area range using the laser map and the laser point cloud data to obtain the robot's second pose data.

[0008] According to another aspect of the present invention, a robot repositioning device is provided.

[0009] The robot relocalization device according to the present invention includes: a receiving module for receiving laser point cloud data and image data acquired by the robot at its current position; a determining module for determining image keyframe information corresponding to the image data according to a visual dictionary; a first solving module for acquiring physical location information corresponding to the image keyframe information in a visual map, matching the feature point information of the image data with the feature point information corresponding to the image keyframe information, and performing a solving operation by combining the physical location information of the feature points in the matched visual map and the pixel location information of the feature points in the image data; and a second solving module for, when the first solving module has obtained the robot's first pose data, setting a predetermined area range based on the first pose data, and performing iterative matching calculations within the predetermined area range using the laser map and the laser point cloud data to obtain the robot's second pose data.

[0010] According to another aspect of the present invention, a robot is provided.

[0011] The robot according to the present invention includes: a memory and a processor, wherein the memory is used to store computer execution instructions; and the processor is used to execute the computer execution instructions stored in the memory, causing the robot to perform the method described above.

[0012] According to the present invention, keyframe information corresponding to image data acquired by the robot at its current position is determined by combining a visual dictionary, and physical location information corresponding to the keyframe information is obtained by combining a visual map. Using the physical location information of feature points in the visual map obtained by matching the feature point information of the image data with the feature point information corresponding to the keyframe information, as well as the pixel location information of the feature points in the image data, the initial pose data of the robot is calculated. A predetermined area is set based on the initial pose data, and the robot is traversed within this predetermined area. Iterative matching calculations are performed using a pre-established laser map and the laser point cloud data to obtain the pose data for robot relocalization. Especially for complex, large-scale environments, it eliminates the need for searching the entire map, shortening the search time for laser sensors in large scenes. Utilizing the initial pose data reduces the iteration range of laser matching, accelerates the matching speed, and quickly achieves robot relocalization. Attached Figure Description

[0013] Figure 1 This is a flowchart of a robot relocalization method according to an embodiment of the present invention;

[0014] Figure 2 This is a flowchart of a robot relocalization method according to a preferred embodiment of the present invention;

[0015] Figure 3 This is a structural block diagram of a robot relocation device according to an embodiment of the present invention;

[0016] Figure 4 This is a structural block diagram of a robot repositioning device according to a preferred embodiment of the present invention;

[0017] Figure 5 This is a structural block diagram of a robot according to an embodiment of the present invention. Detailed Implementation

[0018] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] According to an embodiment of the present invention, a robot relocation method is provided.

[0020] Figure 1 This is a flowchart of a robot relocalization method according to an embodiment of the present invention. Figure 1 As shown, the robot relocation method includes:

[0021] Step S101: Receive laser point cloud data and image data acquired by the robot at the current location;

[0022] Step S102: Determine the image keyframe information corresponding to the above image data based on the visual dictionary;

[0023] Step S103: Obtain the physical location information corresponding to the above image keyframe information in the visual map, match the feature point information of the above image data with the feature point information corresponding to the above image keyframe information, and perform calculation by combining the physical location information of the feature points in the matched visual map and the pixel location information of the feature points in the above image data.

[0024] Step S104: After obtaining the first pose data of the robot, a predetermined area is set according to the first pose data. Within the predetermined area, the laser map and the laser point cloud data are used to perform iterative matching calculations to obtain the second pose data of the robot.

[0025] use Figure 1The method described herein combines a visual dictionary to determine the keyframe information corresponding to the image data acquired by the robot at its current position, and combines a visual map to obtain the physical location information corresponding to the keyframe information. Using the physical location information of feature points in the visual map obtained by matching the feature point information of the image data with the feature point information corresponding to the keyframe information, as well as the pixel location information of the feature points in the image data, the initial pose data of the robot is calculated. Based on the initial pose data, a predetermined area is set, and the robot is traversed within this predetermined area. Iterative matching calculations are performed using a pre-established laser map and the laser point cloud data to obtain the pose data for robot relocalization. Especially for complex, large-scale environments, it eliminates the need for a full map search, shortening the search time for laser sensors in large scenes. Utilizing the initial pose data reduces the iteration range of laser matching, accelerates the matching speed, and quickly achieves robot relocalization.

[0026] In step S103, if the first pose data of the robot is not obtained through calculation, the laser map and the laser point cloud data are used to match within the entire map range to obtain the pose data of the robot.

[0027] Preferably, before receiving the laser point cloud data and image data acquired by the robot at the current location, the following processing may be included: establishing the aforementioned visual dictionary, the aforementioned visual map, and the aforementioned laser map, wherein the aforementioned visual dictionary includes: feature vectors of image keyframes, the aforementioned visual map includes: physical location information (e.g., three-dimensional position coordinates) and feature point information of image keyframes, and the aforementioned laser map includes: an occupied grid map.

[0028] Preferably, in step S102 above, determining the image keyframe information corresponding to the image data according to the visual dictionary may further include: extracting a first feature vector from the image data, matching the first feature vector with the second feature vector of the image keyframe in the visual dictionary, and obtaining the image keyframe information (e.g., keyframe index information) with the highest matching degree.

[0029] Visual dictionaries are image modeling methods used in fields such as image classification and retrieval. A dictionary represents a document as a vector describing the frequency of keywords appearing in the dictionary. For example, the SURF (SpeededUp Robust Features) algorithm is used to extract natural local visual feature vectors from an image. Similar SURF natural visual feature vectors are grouped into the same natural visual words (using the K-means algorithm to cluster the local visual feature sets, with each cluster center representing a visual word). Each natural visual word in the natural visual dictionary is modeled using the GMM (Gaussian Mixture Model) method to create a probabilistic model of the natural visual word. This probabilistic model establishes a more accurate match between local natural visual features and natural visual words.

[0030] In the preferred implementation, a first feature vector is extracted from the image data and matched with the second feature vector of the image keyframe in the visual dictionary. The distance between these two feature vectors can be calculated using the Euclidean distance formula. When the distance is minimized, the image keyframe information with the highest matching degree is obtained. For example, when the Euclidean distance is minimized, the image keyframe with the highest matching degree to the robot's current position is frame 15. Then, the three-dimensional position coordinates corresponding to frame 15 are obtained from the visual map. The feature point information of the above image data is matched with the feature point information corresponding to frame 15. Combining the three-dimensional position coordinates of the feature points in the matched visual map and the pixel position information of the feature points in the above image data, the initial pose data of the robot is calculated. For example, the PnP algorithm is used to calculate the projection relationship between n feature points and n pixels in the image imaging, thereby calculating the robot's pose data.

[0031] The PnP algorithm estimates the robot's pose data when n 3D spatial points and their projected positions are known.

[0032] Assume the robot is located at point Oc, and P1, P2, P3... are feature points.

[0033] Scenario 1: When n = 1;

[0034] When there is only one feature point P1, assuming it is in the exact center of the image, the vector OcP1 is the Z-axis in the robot's coordinate system. In this case, the robot is always facing P1. Therefore, the robot's possible position is on a sphere with P1 as the center. Furthermore, the radius of the sphere cannot be determined, resulting in an infinite number of solutions.

[0035] Scenario 2: When n = 2;

[0036] With an additional constraint, OcP1P2 forms a triangle. Since the positions of points P1 and P2 are determined, the sides P1P2 of the triangle are also determined. Furthermore, with vectors OcP1 and OcP2, the direction angle of the beam from point Oc to the feature point can also be determined. Therefore, the length of OcP1 = r1, and the length of OcP2 = r2 can be calculated. In this case, we obtain two spheres: sphere A with center P1 and radius r1; and sphere B with center P2 and radius r2. Clearly, the camera is located at the intersection of spheres A and B, and there are still countless solutions.

[0037] Scenario 3: When n = 3;

[0038] This time, there is an additional sphere C with P3 as its center. The camera is located at the intersection of the three spheres ABC. There are four solutions, one of which is the robot's pose.

[0039] Scenario 4: When n>3;

[0040] When n>3, the correct solution can be obtained. To solve the problem faster and with less computer resource consumption, four sets of solutions can be calculated using three points to obtain four rotation matrices and translation matrices. According to the formula:

[0041]

[0042] Substitute the world coordinates of the 4th point into the formula to obtain its four projections in the image (one solution corresponds to one projection). Take the solution with the smallest projection error, which is the correct solution we need.

[0043] Preferably, in step S104, the process of using the laser map and the laser point cloud data to perform iterative matching calculations within the predetermined area to obtain the robot's second pose data can be further divided into the following steps: selecting multiple location points within the predetermined area; selecting multiple angles corresponding to each location point; projecting the laser point cloud data corresponding to each angle onto the occupancy grid map; calculating the occupancy probability value corresponding to the angle based on the projection result; selecting the largest occupancy probability value among all obtained occupancy probability values; and determining the robot pose data corresponding to the largest occupancy probability value as the second pose data.

[0044] Preferably, within the aforementioned preset area, multiple location points are selected, and for each location point, multiple angles corresponding to that location point are selected. For each angle, the laser point cloud data corresponding to that angle is projected onto the occupancy grid map. Calculating the occupancy probability value corresponding to that angle based on the projection results may further include the following processing:

[0045] S1: Take the position point in the first pose data above as the initial point, select multiple angles corresponding to the position point, and for each angle, project the laser point cloud data corresponding to the angle onto the occupancy grid map, and calculate the occupancy probability value corresponding to the angle based on the projection result.

[0046] S2: Determine multiple next-level location points according to the predetermined step size and direction, select multiple angles corresponding to each location point in the next-level location points, project the laser point cloud data corresponding to each angle onto the occupied grid map, calculate the occupancy probability value corresponding to each angle based on the projection result, and repeat S2 until the preset area range is traversed.

[0047] In the preferred implementation, the robot's initial pose data is initially calculated using the acquired image data and a pre-established visual dictionary and visual map. Then, using the acquired laser point cloud data and the pre-established laser map, a precise pose calculation is performed within a predetermined area based on the initial pose. This shortens the time required for laser search on a large map, and using the initial visual values ​​reduces the iteration range of laser matching, thus accelerating the matching speed.

[0048] Preferably, step S2 may further include:

[0049] 1. Starting from the initial point, expand in multiple directions (e.g., east, south, west, north, northeast, northwest, southwest, and southeast) around the initial point according to the predetermined step size (e.g., a step size of 5 cm) and predetermined direction;

[0050] 2. Determine the angle direction corresponding to the maximum occupancy probability value among the occupancy probability values ​​corresponding to the initial point, and determine a first angle range (e.g., an angle range of 3°) with this angle direction as the center. Preferably, the first angle range is usually smaller than the angle range formed by multiple angles corresponding to the initial point (e.g., an angle range of 5°); of course, the first angle range can also be equal to or greater than the angle range formed by multiple angles corresponding to the initial point.

[0051] For each position point in the expanded next-level position points, multiple angles are determined within the first angle range (e.g., each angle is 0.5°), and the occupancy probability value corresponding to each angle is obtained. The maximum occupancy probability value among the occupancy probability values ​​corresponding to the position point is compared with the maximum occupancy probability value among the occupancy probability values ​​corresponding to the initial point. If the comparison result is less than, the position point is discarded. If the comparison result is greater than or equal to, the position point is taken as an expandable point, and the maximum occupancy probability value among all occupancy probability values ​​corresponding to the expandable point is set as the current optimal solution.

[0052] Starting from the expandable point, the system expands in multiple directions around the initial point according to the predetermined step size and direction to determine the next-level new location point that can be expanded. The angle direction corresponding to the current optimal solution is determined, and a second angle range (e.g., a 2° angle range) is determined with this angle direction as the center. Preferably, the second angle range is usually smaller than the first angle range; however, the second angle range can also be greater than or equal to the first angle range. For each location point in the next-level new location point, multiple angles (e.g., each angle is 0.5°) are determined within the second angle range. The occupancy probability value corresponding to each angle is obtained. The maximum occupancy probability value among the occupancy probability values ​​corresponding to the location point is compared with the current optimal solution. If the comparison result is less than, the location point is discarded; if the comparison result is greater than or equal to, the location point is set as the expandable point, and the maximum occupancy probability value among the occupancy probability values ​​corresponding to the expandable point is set as the current optimal solution. This step is repeated until the preset area range is traversed.

[0053] Based on the robot's initial pose data, during the process of accurate pose calculation within a predetermined area, the search time is further shortened because some position points are discarded during the search calculation. Each time, the search is performed within an angle range determined by the angle direction corresponding to the current optimal solution, and the search angle range is continuously narrowed. This greatly improves the pose calculation efficiency and effectively speeds up the matching process.

[0054] Preferably, the occupancy probability value P corresponding to each angle at each location point can be calculated in the following way:

[0055] P = (P(x1,y1) + ... + P(x...) i ,y i )+...+P(x n ,y n )) / n

[0056] Among them, (x i ,y i P(x) represents the coordinates of the i-th point cloud data point in the raster. i ,y i ) represents the raster occupancy probability of the i-th point cloud data point, and n is the number of scanned point cloud data points.

[0057] Preferably, after acquiring the second pose data of the robot, the following processing may be further included: determining the visual 3D point corresponding to the first pose data and the pixel coordinates corresponding to the visual 3D point; determining the laser data and laser map corresponding to the second pose data; constructing a nonlinear graph of vision and laser coupling based on the visual 3D point, the pixel coordinates, the laser data, the laser map, and the visual sensor pose data; optimizing the nonlinear graph to obtain third pose data close to the most probable value, which is used as the final pose data of the robot.

[0058] In the preferred implementation process, after step S104, to obtain more accurate robot pose data, a vision-laser coupling method can be adopted. This involves joint optimization based on the first and second pose data. Specifically, using the first pose data, the corresponding visual 3D points and their corresponding pixel coordinates are obtained. Using the second pose data, the corresponding laser data and laser map are obtained, constructing a vision-laser coupled nonlinear graph. The visual 3D points, pixel coordinates, laser data and map, and vision sensor (e.g., camera) pose data are used as vertices of the nonlinear graph. Based on vision and laser physical rules (e.g., projection relationships), connecting edges between vertices are established. Finally, a graph optimization method (e.g., Ceres optimization) is used to jointly optimize the vision and laser data, obtaining a third pose data close to the most probable value, which serves as the robot's final repositioning pose data. Of course, other optimization methods, such as least squares, can also be used.

[0059] The following combination Figure 2 The preferred embodiments described above are further described below.

[0060] Figure 2 This is a flowchart of a robot relocalization method according to a preferred embodiment of the present invention. Figure 2 As shown, the robot relocation method includes:

[0061] Step S201: During mapping, a visual dictionary, a visual map, and a laser map are pre-built. The visual dictionary stores the feature vectors of keyframes in the image, the visual map stores the index identification information, physical location information, and feature point information of keyframes in the image, and the laser map stores the occupancy grid map built using a laser sensor.

[0062] Step S202: Using the map established in S201, receive the laser data and image data acquired by the robot at the current location.

[0063] Step S203: Using the image data corresponding to the current location and the visual dictionary and visual map established in step S201, extract feature vectors from the image data, match them with the feature vectors in the visual dictionary, obtain the image keyframe with the highest matching degree, and obtain the physical location information (e.g., three-dimensional location coordinates) of the visual map keyframe with the highest similarity to the current location through index identification information (e.g., the 10th frame) in the visual map.

[0064] Step S204: Based on the above image data and feature points of the visual map, matching is performed by calculating feature similarity. The pose data for robot visual relocalization is obtained by using the PnP method to solve the pose data of the feature points in the visual map and the pixel position of the feature points in the current image.

[0065] Step S205: Based on the pose data obtained in step S204, iteratively match the laser point cloud data with the laser map established in step S201 to obtain the robot's relocalization pose data. Specifically:

[0066] Based on the first pose data, a predetermined area is set. For example, a circular area is constructed with the position of the first pose data as the center and a predetermined length (e.g., 50 cm) as the radius, or a rectangular area is constructed with the position of the first pose data as the center point. Within the above area, the pose is changed with a predetermined step size (e.g., 5 cm) and a predetermined angle (e.g., 0.5°) to calculate and obtain multiple occupancy probability values. The largest occupancy probability value is selected from all the obtained occupancy probability values, and the robot pose data corresponding to the largest occupancy probability value is determined as the second pose data.

[0067] Specifically, the occupancy probability value P corresponding to each angle at each location point can be calculated in the following way:

[0068] P = (P(x1,y1) + ... + P(x...) i ,y i )+...+P(x n ,y n )) / n

[0069] Among them, (x i ,y i P(x) represents the coordinates of the i-th point cloud data point in the raster. i ,y i ) represents the raster occupancy probability of the i-th point cloud data point, and n is the number of scanned point cloud data points.

[0070] In the preferred implementation process, the position point in the first pose data is taken as the initial point. Multiple angles corresponding to this position point are selected (for example, 11 angular directions, one angular direction directly in front, 5 angles to the left and 5 angles to the right based on the direct front, with a 0.5° interval between each angle). For each angle, the laser point cloud data corresponding to the angle is projected onto the occupancy grid map. The occupancy probability value corresponding to each angle is calculated based on the projection result. The largest occupancy probability value is taken as the current optimal solution. The occupancy probability value is expanded in 8 predetermined directions around the initial point at a step size of 5 cm to determine the angular direction corresponding to the largest occupancy probability value. An angle range of 3° is determined with this angular direction as the center, resulting in 7 angular directions, one of which is the angular direction corresponding to the largest occupancy probability value. The angular direction corresponding to the largest occupancy probability value is taken as the reference and deviated 3 angles to the left and 3 angles to the right, with a 0.5° interval between each angle.

[0071] For each position point in the expanded next-level position points, obtain the occupancy probability value corresponding to each of the above angles. Compare the maximum occupancy probability value of the position point with the maximum occupancy probability value of the initial point. If the comparison result is less than, discard the position point. If the comparison result is greater than or equal to, set the position point as the above expandable point, and set the maximum occupancy probability value of the above expandable point as the above current optimal solution.

[0072] Starting from the expandable point, expand in eight directions around the initial point in 5 cm increments to determine the next level of new locations that can be expanded. Determine the angle direction corresponding to the current optimal solution. Using this angle direction as the center, define a 2° angle range. For each location in the next level of new locations, determine multiple angles within this 2° angle range, resulting in five angle directions. One of these is the angle direction corresponding to the maximum occupancy probability value. Using this angle direction as the reference, deviate 2 degrees to the left and 2 degrees to the right, with each angle spaced 0.5° apart. Obtain the occupancy probability value corresponding to each angle. Compare the maximum occupancy probability value of this location with the current optimal solution. If the comparison result is less than, discard the location. If the comparison result is greater than or equal to, set this location as the expandable point, and set the maximum occupancy probability value among the expandable points as the current optimal solution. Repeat this step until the entire preset area has been traversed. After traversing the above-mentioned preset area range, the largest occupancy probability value is selected from all the obtained occupancy probability values, and the robot pose data corresponding to the largest occupancy probability value is determined as the pose data for robot relocalization.

[0073] According to an embodiment of the present invention, a robot repositioning device is also provided.

[0074] Figure 3 This is a structural block diagram of a robot relocation device according to an embodiment of the present invention. Figure 3 As shown, the robot relocalization device includes: a receiving module 30, used to receive laser point cloud data and image data acquired by the robot at its current position; a determining module 32, used to determine the image keyframe information corresponding to the image data according to a pre-established visual dictionary; a first solving module 34, used to acquire the physical location information corresponding to the image keyframe information in a pre-established visual map, match the feature point information of the image data with the feature point information corresponding to the image keyframe information, and perform solving by combining the physical location information of the feature points in the matched visual map and the pixel location information of the feature points in the image data; and a second solving module 36, used to, when the first solving module has obtained the robot's first pose data, set a predetermined area range based on the first pose data, and perform iterative matching calculations within the predetermined area using a pre-established laser map and the laser point cloud data to obtain the robot's second pose data.

[0075] exist Figure 3 In the device shown, the determining module 32, combined with a visual dictionary, determines the keyframe information of the image data acquired by the robot at its current position. The first solving module 34, combined with a visual map, obtains the physical position information corresponding to the keyframe information. Using the physical position information of the feature points in the visual map after matching the feature point information of the image data with the feature point information corresponding to the keyframe information, and the pixel position information of the feature points in the image data, the initial pose data of the robot is calculated. The second solving module 36 sets a predetermined area range based on the initial pose data, traverses within the predetermined area, and uses a pre-established laser map and the laser point cloud data to perform iterative matching calculations to obtain the pose data for robot relocalization. Using this device, especially in complex large-scene environments, eliminates the need for searching the entire map, shortening the search time for laser sensors in large scenes. Utilizing the initial pose data reduces the iteration range of laser matching, accelerates the matching speed, and quickly achieves robot relocalization.

[0076] Preferably, such as Figure 4As shown, the second calculation module 36 may further include: a calculation submodule 360, used to select multiple location points within the preset area, select multiple angles corresponding to each location point, project the laser point cloud data corresponding to each angle onto the occupancy grid map, and calculate the occupancy probability value corresponding to the angle based on the projection result; and a determination submodule 362, used to select the largest occupancy probability value among all the obtained occupancy probability values, and determine the robot pose data corresponding to the largest occupancy probability value as the second pose data.

[0077] It should be noted that the preferred implementation method for the combination of the modules in the above-mentioned robot repositioning device can be found in [reference needed]. Figures 1 to 2 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0078] According to an embodiment of the present invention, a robot is provided.

[0079] Figure 5 This is a structural block diagram of a robot according to an embodiment of the present invention. Figure 5 As shown, the robot according to the present invention includes: a memory 50 and a processor 52. The memory 50 is used to store computer execution instructions; the processor 52 is used to execute the computer execution instructions stored in the memory, causing the robot to perform the robot relocation method provided in the above embodiments.

[0080] Processor 52 can be a central processing unit (CPU). Processor 52 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0081] The memory 50, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the robot relocation method in this embodiment of the invention. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory.

[0082] Memory 40 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 50 may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0083] One or more of the above modules are stored in the above memory 50, and when executed by the above processor 52, they perform the following: Figure 1 and Figure 2 The robot relocation method in the illustrated embodiment.

[0084] For specific details about the aforementioned robots, please refer to the relevant documentation. Figure 1 and Figure 2 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0085] In summary, by utilizing the embodiments provided by this invention, an initial pose is initially calculated using a visual dictionary and visual map along with image data acquired at the current location. Based on this initial pose, precise localization is performed within a preset area using laser point cloud data and a laser map acquired at the current location. This shortens the time required for laser searching across a large map area, and the use of initial visual values ​​reduces the iteration range of laser matching, improving relocalization efficiency. Furthermore, during the precise pose calculation within the predetermined area based on the robot's initial pose data, the search time is further shortened because some position points are discarded during the search calculation, significantly improving pose calculation efficiency and accelerating the matching speed more effectively.

[0086] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A robot repositioning method, characterized by, include: Receive laser point cloud data and image data acquired by the robot at its current location; Determine the image keyframe information corresponding to the image data based on the visual dictionary; Obtain the physical location information corresponding to the keyframe information of the image in the visual map, match the feature point information of the image data with the feature point information corresponding to the keyframe information of the image, and perform calculation by combining the physical location information of the feature points in the matched visual map and the pixel location information of the feature points in the image data. When the first pose data of the robot is obtained by solving the calculation, a predetermined area is set according to the first pose data. Within the predetermined area, the laser map and the laser point cloud data are used to perform iterative matching calculation to obtain the second pose data of the robot. The process of obtaining the second pose data of the robot within the predetermined area by using a pre-established laser map and the laser point cloud data to perform iterative matching calculation includes: selecting multiple position points within the predetermined area; selecting multiple angles corresponding to each position point; projecting the laser point cloud data corresponding to each angle onto an occupancy grid map; calculating the occupancy probability value corresponding to the angle based on the projection result; selecting the largest occupancy probability value among all obtained occupancy probability values; and determining the robot pose data corresponding to the largest occupancy probability value as the second pose data.

2. The method of claim 1, wherein, Before receiving the laser point cloud data and image data acquired by the robot at its current location, the process also includes: The visual dictionary, the visual map, and the laser map are established, wherein the visual dictionary includes feature vectors of image keyframes, the visual map includes physical location information and feature point information of image keyframes, and the laser map includes an occupied grid map.

3. The method according to claim 1, characterized in that, The keyframe information corresponding to the image data determined according to the visual dictionary includes: A first feature vector is extracted from the image data, and the first feature vector is matched with the second feature vector of the image keyframe in the visual dictionary to obtain the image keyframe information with the highest matching degree.

4. The method according to claim 1, characterized in that, Multiple location points are selected within the predetermined area. For each location point, multiple angles corresponding to that location point are selected. For each angle, the laser point cloud data corresponding to that angle is projected onto the occupancy grid map. The occupancy probability value corresponding to that angle is calculated based on the projection results, including: S1: Take the position point in the first pose data as the initial point, select multiple angles corresponding to the initial point, and for each angle, project the laser point cloud data corresponding to the angle onto the occupancy grid map, and calculate the occupancy probability value corresponding to the angle based on the projection result. S2: Determine multiple next-level location points according to the predetermined step size and direction, select multiple angles corresponding to each location point in the next-level location points, project the laser point cloud data corresponding to each angle onto the occupied grid map, calculate the occupancy probability value corresponding to each angle based on the projection result, and repeat S2 until the predetermined area range is traversed.

5. The method according to claim 4, characterized in that, S2 further includes: Starting from the initial point, the process expands in multiple predetermined directions around the initial point according to the predetermined step size. Determine the angle direction corresponding to the maximum occupancy probability value among the occupancy probability values ​​corresponding to the initial point, and determine the first angle range with this angle direction as the center; For each position point in the expanded next-level position points, multiple angles are determined within the first angle range, and the occupancy probability value corresponding to each angle is obtained. The maximum occupancy probability value among the occupancy probability values ​​corresponding to the position point is compared with the maximum occupancy probability value among the occupancy probability values ​​corresponding to the initial point. If the comparison result is less than, the position point is discarded. If the comparison result is greater than or equal to, the position point is regarded as an expandable point, and the maximum occupancy probability value among all occupancy probability values ​​corresponding to the expandable point is set as the current optimal solution. Starting from the expandable point, expand in multiple directions around the initial point according to the predetermined step size to determine the next level of new position points that can be expanded. Determine the angle direction corresponding to the current optimal solution, and determine a second angle range centered on this angle direction. For each position point in the next level of new position points, determine multiple angles within the second angle range, obtain the occupancy probability value corresponding to each angle, and compare the maximum occupancy probability value among the occupancy probability values ​​corresponding to the position point with the current optimal solution. If the comparison result is less than, discard the position point; if the comparison result is greater than or equal to, set the position point as the expandable point, and set the maximum occupancy probability value among the occupancy probability values ​​corresponding to the expandable point as the current optimal solution. Repeat this step until the predetermined area range has been traversed.

6. The method according to claim 1, 4, or 5, characterized in that, The occupancy probability value P corresponding to each angle at each location point is calculated in the following way: P = (P(x1, y1 ) +... + P(x i ,y i )+... +P(x n ,y n )) / n wherein (x i ,y i ) represents the coordinates of the i-th point cloud data point in the grid, P(x i ,y i ) represents the grid occupancy probability of the i-th point cloud data point, and n is the number of scanned point cloud data points.

7. The method according to claim 1, characterized in that, After obtaining the robot's second pose data, the method further includes: Determine the visual 3D point corresponding to the first pose data and the pixel coordinates corresponding to the visual 3D point; Determine the laser data and laser map corresponding to the second pose data; Based on the visual 3D points, the pixel coordinates, the laser data, the laser map, and the visual sensor pose data, a nonlinear graph of visual and laser coupling is constructed. The nonlinear graph is optimized to obtain third pose data that is close to the most probable value, which is used as the final pose data of the robot.

8. A robot repositioning device, characterized in that, include: The receiving module is used to receive laser point cloud data and image data acquired by the robot at its current location; The determination module is used to determine the image keyframe information corresponding to the image data based on the visual dictionary; The first solution module is used to obtain the physical location information corresponding to the key frame information of the image in the visual map, match the feature point information of the image data with the feature point information corresponding to the key frame information of the image, and perform solution by combining the physical location information of the feature points in the visual map after matching and the pixel location information of the feature points in the image data. The second calculation module is used to, when the first calculation module has obtained the first pose data of the robot, set a predetermined area range based on the first pose data, and perform iterative matching calculations using a laser map and the laser point cloud data within the predetermined area range to obtain the second pose data of the robot. The second calculation module is further used to select multiple position points within the predetermined area range, select multiple angles corresponding to each position point, project the laser point cloud data corresponding to each angle onto an occupancy grid map, calculate the occupancy probability value corresponding to the angle based on the projection result, select the largest occupancy probability value among all obtained occupancy probability values, and determine the robot pose data corresponding to the largest occupancy probability value as the second pose data.

9. A robot, comprising: Memory and processor, characterized in that, The memory is used to store computer-executed instructions; The processor is configured to execute computer execution instructions stored in the memory, causing the robot to perform the method as described in any one of claims 1 to 6.

Images