Mobile robot visual relocalization method and mobile robot

Abstract

The embodiment of the present application provides a mobile robot vision repositioning method and a mobile robot. When repositioning the mobile robot, the embodiment first controls the mobile robot to rotate with the current position as the center to collect data such as images for repositioning, and when repositioning cannot be performed based on the collected data, further determines a motion route which is easier to reposition successfully according to the data such as images collected in the rotating process, controls the robot to move along the motion route, so as to determine a position which is easier to reposition successfully, and finally realizes the repositioning of the mobile robot in the motion process.

Description

Technical Field

[0001] This application relates to mobile robots, and more particularly to a visual relocalization method for mobile robots and mobile robots. Background Technology

[0002] Currently, visual Figure 1 Visual maps are generally sparse, and when performing scene relocalization based on visual maps, there are many areas where relocalization is difficult to achieve. If relocalization is only performed at the initial position, the success rate will be very low, making it unsuitable for practical use by intelligent mobile robots. If the robot simply moves around randomly, it is easy to stray out of the map's boundaries or into blind spots, making scene relocalization even more difficult.

[0003] Therefore, a method to ensure the repositioning of mobile robots during movement is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides a mobile robot visual relocation method and a mobile robot to achieve successful relocation of the mobile robot during movement.

[0005] This application provides a visual relocalization method for a mobile robot, the method comprising:

[0006] When relocalizing the mobile robot based on the constructed visual map, the mobile robot rotates around its current position and acquires images during the rotation process through an image acquisition component deployed on the mobile robot.

[0007] If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data that matches the image in the visual map meets the relocation condition, then the relocation is determined to be successful; otherwise, the mobile robot starts moving from the current position along the optimal direction; the optimal direction is determined based on the matching degree between each image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches each image in the visual map.

[0008] During the process of the mobile robot moving along the optimal direction, if the matching degree between the image acquired by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocalization condition, then the relocalization is determined to be successful. Otherwise, after moving to the target position that meets the condition, the process returns to the step of relocalizing the mobile robot based on the constructed visual map.

[0009] This application embodiment also provides a mobile robot, which includes: a controller and an image acquisition component;

[0010] A controller, configured to, during relocalization of a mobile robot based on a constructed visual map, allow the mobile robot to rotate around its current position and acquire images during this rotation via image acquisition components deployed on the mobile robot; and,

[0011] If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data matching that image in the visual map meets the relocalization condition, then the relocalization is determined to be successful; otherwise, the mobile robot starts moving from the current position along the optimal direction; the optimal direction is determined based on the matching degree between each image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching each image in the visual map; and,

[0012] During the process of the mobile robot moving along the optimal direction, if the matching degree between the image captured by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocalization condition, then the relocalization is determined to be successful; otherwise, after moving to the target position that meets the condition, the process returns to the step of relocalizing the mobile robot based on the constructed visual map.

[0013] Image acquisition component, used to acquire images.

[0014] This application embodiment also provides a machine-readable storage medium, characterized in that the machine-readable storage medium stores executable machine-executable instructions;

[0015] The machine-executable instructions are executed to perform the steps of the method described above.

[0016] As can be seen from the above technical solutions, when repositioning a mobile robot, the embodiments of this application first control the mobile robot to rotate around the current position to collect data such as images for repositioning. If repositioning is not possible based on the collected data, the robot further determines a movement route that is easier to reposition successfully based on the data collected during the rotation process, such as images. The robot is then controlled to move along the movement route to determine a position that is easier to reposition successfully, thus ultimately achieving repositioning of the mobile robot during the movement process. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] Figure 1 A flowchart illustrating the method provided in this application embodiment;

[0019] Figure 2This is a schematic diagram of motion relocation provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of motion relocation in an indoor scene provided in an embodiment of this application;

[0021] Figure 4 A flowchart for determining the optimal orientation provided in this application embodiment;

[0022] Figure 5 A schematic diagram illustrating the optimal direction of the matching variance provided in the embodiments of this application;

[0023] Figure 6 Another schematic diagram illustrating the optimal direction of matching variance provided in the embodiments of this application;

[0024] Figure 7 A schematic diagram of a grid map provided in an embodiment of this application;

[0025] Figure 8 A schematic diagram of a grid map facing an open area provided in an embodiment of this application;

[0026] Figure 9 This is a structural diagram of the robot provided in an embodiment of this application. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0029] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0030] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. As one embodiment, this method can be applied to a mobile robot. Figure 1 As shown, the process may include the following steps:

[0031] Step 101: When relocalizing the mobile robot based on the constructed visual map, the mobile robot rotates around its current position and acquires images during the rotation using the image acquisition components deployed on the mobile robot.

[0032] As one embodiment, in step 101, the mobile robot can rotate clockwise around its current position (also called the origin). Of course, the mobile robot can also rotate counter-clockwise around its current position (also called the origin). This embodiment is not specifically limited. As one embodiment, to ensure continuous rotation of the robot, the maximum rotation angle of the mobile robot can be limited, for example, less than or equal to 360 degrees.

[0033] Optionally, in this embodiment, during the rotation of the mobile robot, the image acquisition component deployed on the mobile robot will also record the current rotation angle (specifically, the rotation angle up to the current position) in a timely manner each time an image is acquired. This rotation angle will be discussed later when determining the optimal direction, and will not be elaborated here.

[0034] Step 102: If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data matching the image in the visual map meets the relocation condition, then the relocation is determined to be successful. If the matching degree between any image acquired by the image acquisition component when the mobile robot rotates to the maximum rotation angle and the visual map data matching the image in the visual map does not meet the relocation condition, the mobile robot starts moving along the optimal direction from the current position.

[0035] In this embodiment, the degree of matching between an image (or at least one image) acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data that matches the image in the aforementioned visual map can be determined in many ways. For example, it can be obtained by weighting at least the following parameters that reflect the matching quality:

[0036] 1) The matching score between the image and the visual map data that matches the image in the visual map. As an example, the matching score between the image and the visual map data that matches the image in the visual map is a score calculated based on the feature point descriptors of the image and the visual map data that matches the image. Here, the feature point descriptor of the image is used to describe the image; similarly, the feature point descriptor of the visual map data that matches the image is used to describe the visual map data that matches the image. In a specific implementation, the feature point descriptor can be ORB (Oriented Fast and Rotated BRIEF) feature points.

[0037] 2) The number of identical feature point descriptors between the image and the visual map data that matches the image in the visual map.

[0038] 3) The number of geometric verification points that satisfy the geometric constraints obtained after projecting (e.g., projecting onto a plane) the image and the visual map data that matches the image in the visual map. Here, the geometric constraints can be set according to actual needs, and this embodiment is not specifically limited.

[0039] Based on the above description, the matching degree can be obtained by the following formula: Score = K1*S1 + K2S2 + K K *S3; where Score represents the matching degree, S1, S2, and S3 represent the three parameters mentioned above, and K1, K2, and K3 represent the weights of the three parameters mentioned above.

[0040] Based on the above matching degree determination method, as an example, if it is found that the matching degree between the image captured by the mobile robot when it rotates to a certain angle and the visual map data that matches the image in the above visual map meets the relocation condition, for example, the matching degree between the image and the visual map data that matches the image in the visual map is greater than or equal to the set matching degree threshold (which can be set according to actual needs), then as described in step 102, the relocation is directly determined to be successful. At this time, even if the mobile robot has not yet rotated to the maximum rotation angle, the rotating robot can be controlled to stop rotating.

[0041] As an example, when repositioning is determined to be successful, the robot can be moved to its current position and the rotation angle (collectively referred to as pose) of the image on which the successful repositioning was determined to be successful can be used to start positioning and tracking.

[0042] Of course, if the matching degree between any image acquired by the image acquisition component when the mobile robot rotates to the maximum rotation angle and the visual map data that matches the image in the visual map does not meet the relocation condition, then as described in step 102, the optimal direction is determined first, and then the mobile robot starts to move along the optimal direction from the current position.

[0043] The purpose of determining the optimal direction in this embodiment is to identify a movement path that is easier to successfully reposition based on data collected during the rotation process, such as images and the rotation angle of the mobile robot when acquiring the images. This allows the robot to be controlled to move along the movement path, thereby determining a position where successful repositioning is more likely. Here, the optimal direction is determined based on the matching degree between each image acquired by the image acquisition component during the mobile robot's rotation and the visual map data that matches each image in the aforementioned visual map. Examples will be described below, and will not be elaborated upon here.

[0044] As one embodiment, after determining the optimal direction, the mobile robot moving along the optimal direction from its current position may include: the mobile robot moving along the optimal direction according to a specified action while activating obstacle avoidance. As one embodiment, to avoid the mobile robot moving endlessly, this embodiment may limit the movement distance of the mobile robot. Optionally, the limited movement distance is used to ensure that the image data in at least one frame of the image acquisition component is different when the mobile robot rotates between its position before and after the movement. As one embodiment, the limited movement distance is, for example, 0.4m. Using this limited movement distance, another position at a distance from the current position can be designated as the target position that meets the conditions.

[0045] Optionally, the specified action can be an S-curve motion, or other actions such as walking in a straight line. The S-curve motion simply enriches the perspective of the image acquisition component. Figure 2 An example is shown in the schematic diagram of S-shaped curvature motion. Figure 3 An example is shown for motion relocation in an indoor scene.

[0046] Step 103: During the process of the mobile robot moving along the optimal direction, if the matching degree between the image acquired by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocation condition, then the relocation is determined to be successful; otherwise, after moving to the target position that meets the condition, return to the step of relocating the mobile robot based on the constructed visual map in step 101.

[0047] As an example, during the process of the mobile robot moving along the optimal direction from its current position, timely relocalization is required. That is, every time it moves to a position that meets certain conditions (e.g., any position, or a position 10cm away from the previous position), the image acquisition component will capture an image of the mobile robot at that position. If the matching degree between the captured image and the visual map data that matches the image in the visual map meets the relocalization conditions (see the description above), then the relocalization is considered successful. At this time, localization and tracking can begin according to the current pose. As an example, if the current position is not the target position, after confirming successful relocalization, the mobile robot can be controlled to stop moving along the optimal direction.

[0048] Of course, if the mobile robot moves from its current position to the target position along the optimal direction, and the matching degree between any image acquired by the image acquisition component during the movement of the mobile robot and the visual map data that matches that image in the visual map does not meet the relocalization condition, then the process returns to step 101 above, where the mobile robot is relocalized based on the constructed visual map. This continues until the relocalization is finally successful.

[0049] This concludes the process. Figure 1 The process is shown below.

[0050] pass Figure 1 The process shown demonstrates that when relocalizing a mobile robot, it first controls the robot to rotate around its current position to collect data such as images for relocalization. If relocalization is not possible based on the collected data, the robot then uses the data collected during the rotation process, such as images, to determine a more successful relocalization path. The robot is then controlled to move along this path to determine a more successful relocalization position, ultimately achieving relocalization of the mobile robot during its movement.

[0051] The determination of the optimal direction is described below:

[0052] See Figure 4 , Figure 4 A flowchart illustrating the optimal orientation determination process provided for embodiments of this application. Figure 4 As shown, the process may include the following steps:

[0053] Step 401: Determine whether the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches the image in the visual map meets the set positioning requirements. If yes, proceed to step 402; otherwise, proceed to step 403.

[0054] As an example, a matching variance is introduced. The matching variance is obtained by calculating the variance of each reference matching degree. Each reference matching degree is determined based on the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches that image in the visual map. Examples will be provided below, and will not be elaborated upon here. Optionally, in this embodiment, the smaller the matching variance, the more average the data, and the absence of a clear optimal direction, for example… Figure 5 As shown, the larger the variance of the matching degree, the more obvious the optimal direction is, for example... Figure 6 As shown. Based on this, the judgment in step 401 may include: calculating the matching degree variance; if the matching degree variance is greater than a set matching degree variance threshold, then it is determined that the set positioning requirements are met; otherwise, it is determined that the set positioning requirements are not met. Here, the matching degree variance threshold can be set according to actual needs.

[0055] Step 402: Determine the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires an image that meets the set conditions.

[0056] Step 402 is performed under the premise that the above-mentioned positioning requirements are met as determined in step 401. If the above-mentioned positioning requirements are met, it means that there is an accurate optimal direction. At this time, as described in step 402, the optimal direction can be determined based on the rotation angle of the mobile robot when the image acquisition component acquires an image that meets the set conditions.

[0057] As an example, there are many ways to determine the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires an image that meets the set conditions. For example, one approach is to select the angle corresponding to the reference matching degree with the largest value from the current linked list, and determine the optimal direction based on the direction corresponding to that angle. The current linked list contains the correspondence between the reference matching degree and the angle. The following example describes the current linked list:

[0058] As an example, the reference matching degree in the current linked list refers to the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image. The angle refers to the rotation angle of the image acquisition component when acquiring the image during the rotation of the mobile robot. For example, if the reference matching degree in the current linked list is 'a', where 'a' represents the matching degree between the image I1 acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image I1, then the angle 'b' corresponding to the reference matching degree 'a' represents the rotation angle of the image acquisition component when acquiring the image I1 during the rotation of the mobile robot. Optionally, in this embodiment, the above-mentioned optimal direction can be the angle corresponding to the reference matching degree with the largest value in the current linked list.

[0059] In another embodiment, the angles in the current linked list are represented by angle windows, which are obtained by grouping the rotation angles of the mobile robot when the image acquisition component acquires images. For example, an angle window may contain at least two adjacent rotation angles. Correspondingly, the reference matching degree in the current linked list is obtained by performing a mathematical operation (e.g., calculating the average value) on the matching degree between the image acquired by the image acquisition component when the mobile robot is at each rotation angle in the corresponding angle window and the visual map data in the visual map that matches that image. For example, if an angle window k in the current linked list contains two rotation angles k1 and k2, the reference matching degree corresponding to angle window k is obtained by performing a mathematical operation (e.g., calculating the average value) on the following matching degrees: the matching degree between the image acquired by the image acquisition component when the mobile robot rotates to k1 and the visual map data in the visual map that matches that image, and the matching degree between the image acquired by the image acquisition component when the mobile robot rotates to k2 and the visual map data in the visual map that matches that image. Optionally, in this embodiment, the optimal direction can be determined based on the rotation angles in the angle window corresponding to the reference matching degree with the largest value in the current linked list. For example, the average rotation angles in the angle window can be taken, and the direction corresponding to the average angle can be determined as the optimal direction.

[0060] The above provides an example of how to determine the optimal direction in step 402.

[0061] Step 403: Based on the obtained raster map, identify open areas that meet the open area conditions, and determine the direction facing the open area as the optimal direction.

[0062] Step 403 is performed under the premise that the above-mentioned positioning requirements are not met as determined in step 401. If the above-mentioned positioning requirements are not met, it means that there is no accurate optimal direction. At this time, as described in step 403, an open area that meets the open conditions can be determined based on the obtained grid map, and the direction towards the open area can be determined as the optimal direction.

[0063] As an example, the above-described grid map is a grid map constructed based on the rotation of the mobile robot. Figure 7 An example is given to illustrate a grid map. Figure 8 An example is shown pointing towards an open area (i.e., an open space).

[0064] This concludes the process. Figure 4 The process is shown below.

[0065] pass Figure 4 The process shown demonstrates how to determine the optimal direction.

[0066] It should be noted that, in this embodiment, before relocalizing the mobile robot based on the constructed visual map, the mobile robot's relocalization mode can be enabled first, so that execution can proceed. Figure 1 The process is shown below. After confirming successful relocation, the mobile robot's relocation mode can be further disabled.

[0067] Furthermore, in this embodiment, to avoid repeatedly relocalizing the mobile robot based on the constructed visual map, before returning from step 103 to step 101 to relocalize the mobile robot based on the constructed visual map, it can be checked whether the recorded number of relocalizations is the set number. If yes, the relocalization ends; otherwise, it continues to return to step 101 to relocalize the mobile robot based on the constructed visual map. Therefore, in step 101, when relocalizing the mobile robot based on the constructed visual map, the currently recorded number of relocalizations can be increased by a preset value, such as 1.

[0068] This concludes the description of the method provided in this embodiment. The robot provided in this application embodiment will now be described:

[0069] See Figure 9 , Figure 9 This is a structural diagram of a mobile robot provided in an embodiment of this application. Figure 9 As shown, the mobile robot includes: a controller and an image acquisition component;

[0070] The controller is used to, when relocalizing the mobile robot based on a constructed visual map, rotate the mobile robot around its current position and acquire images during the rotation process via image acquisition components deployed on the mobile robot; and,

[0071] If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data matching that image in the visual map meets the relocalization condition, then the relocalization is determined to be successful; otherwise, the mobile robot starts moving from the current position along the optimal direction; the optimal direction is determined based on the matching degree between each image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching each image in the visual map; and,

[0072] During the process of the mobile robot moving along the optimal direction, if the matching degree between the image captured by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocalization condition, then the relocalization is determined to be successful; otherwise, after moving to the target position that meets the condition, the process returns to the step of relocalizing the mobile robot based on the constructed visual map.

[0073] Image acquisition component, used to acquire images.

[0074] As one embodiment, acquiring images during the rotation of the mobile robot via an image acquisition component deployed on the mobile robot further includes: obtaining the current rotation angle of the mobile robot when the image acquisition component acquires each image; and constructing a corresponding grid map based on the rotation of the mobile robot.

[0075] The optimal direction is determined by the following steps:

[0076] Determine whether the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches the image in the visual map meets the set positioning requirements. If yes, determine the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires the image that meets the set conditions. Otherwise, determine an open area that meets the open conditions based on the grid map, and determine the direction toward the open area as the optimal direction.

[0077] As one embodiment, determining whether the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches the image in the visual map meets the set positioning requirements includes:

[0078] Calculate the matching degree variance; the matching degree variance is obtained by performing variance calculation on each reference matching degree. Each reference matching degree is determined based on the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image.

[0079] If the variance of the matching degree is greater than the set variance threshold of the matching degree, then the set positioning requirements are met; otherwise, the set positioning requirements are not met.

[0080] As one embodiment, determining the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires an image that meets the set conditions includes: selecting the angle corresponding to the reference matching degree with the largest value in the current linked list, and determining the optimal direction based on the angle; wherein, the current linked list contains the correspondence between the reference matching degree and the angle;

[0081] The reference matching degree in the current linked list refers to the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image. The angle refers to the rotation angle of the image acquisition component when acquiring the image during the rotation of the mobile robot, or...

[0082] The angles in the current linked list are represented by angle windows. The angle windows in the current linked list are obtained by grouping the rotation angles of the mobile robot when the image acquisition component acquires the image. The reference matching degree in the current linked list is obtained by setting a mathematical operation between the matching degree between the image acquired by the image acquisition component and the visual map data that matches the image when the mobile robot is at each rotation angle in the corresponding angle window.

[0083] As one embodiment, the mobile robot moving along the optimal direction from its current first position includes: the mobile robot moving along the optimal direction according to a specified action while activating obstacle avoidance;

[0084] The distance between the target position and the current position in the optimal direction is a set distance. The set distance is used to ensure that the image data in at least one frame of the image acquisition component is different when the mobile robot rotates between its position before and after the movement.

[0085] As an example, the matching degree between the image acquired by the image acquisition component and the visual map data that matches the image in the visual map is determined based on a weighted average of the following parameters:

[0086] The matching score between the image and the visual map data that matches the image in the visual map, the number of identical feature point descriptors between the image and the visual map data that matches the image in the visual map, and the number of geometric verification points that satisfy the geometric constraints after the image and the visual map data that matches the image in the visual map are projected.

[0087] As an example, before relocalizing the mobile robot based on the constructed visual map, the controller further controls the activation of the mobile robot's relocalization mode.

[0088] After confirming successful relocation, the controller further controls the disabling of the relocation mode of the mobile robot.

[0089] As an example, before returning to relocalize the mobile robot based on the constructed visual map, it is further checked whether the recorded number of relocalizations is the set number. If yes, the relocalization ends; if no, the relocalization of the mobile robot based on the constructed visual map continues.

[0090] When relocalizing the mobile robot based on the constructed visual map, the currently recorded number of relocalizations is further increased by a preset value.

[0091] This concludes the process. Figure 9 The structure is described as shown.

[0092] Correspondingly, in this embodiment, the present application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed, can implement the methods disclosed in the above examples of the present application.

[0093] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0094] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0095] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A visual relocalization method for a mobile robot, characterized in that, The method includes: When relocalizing the mobile robot based on the constructed visual map, the mobile robot rotates around its current position and acquires images during the rotation process through an image acquisition component deployed on the mobile robot. If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data matching the image in the visual map meets the repositioning condition, then the repositioning is determined to be successful. Otherwise, the mobile robot starts moving from the current position along the optimal direction. The optimal direction is used to determine a movement route that is easy to reposition successfully based on the images acquired by the image acquisition component during the rotation of the mobile robot and the rotation angle of the mobile robot when acquiring the images, so as to control the robot to determine the position that is easy to reposition successfully when moving along the movement route. The optimal direction is determined based on whether the matching degree between each image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching each image in the visual map meets the set positioning requirements. When the matching degree variance is greater than the set matching degree variance threshold, it is determined that the set positioning requirements are met; otherwise, it is determined that the set positioning requirements are not met. The matching degree variance is obtained by performing variance calculation on each reference matching degree, and each reference matching degree is determined based on the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching the image in the visual map. During the process of the mobile robot moving along the optimal direction, if the matching degree between the image acquired by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocalization condition, then the relocalization is determined to be successful. Otherwise, after moving to the target position that meets the condition, the process returns to the step of relocalizing the mobile robot based on the constructed visual map.

2. The method according to claim 1, characterized in that, The step of acquiring images during the rotation of the mobile robot using the image acquisition component deployed on the mobile robot further includes: obtaining the current rotation angle of the mobile robot when the image acquisition component acquires each image; and constructing a corresponding grid map based on the rotation of the mobile robot. The optimal direction is determined by the following steps: Determine whether the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches the image in the visual map meets the set positioning requirements. If yes, determine the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires the image that meets the set conditions. Otherwise, determine an open area that meets the open conditions based on the grid map, and determine the direction toward the open area as the optimal direction.

3. The method according to claim 2, characterized in that, The step of determining whether the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data that matches the image in the visual map meets the set positioning requirements includes: Calculate the matching degree variance; the matching degree variance is obtained by performing variance calculation on each reference matching degree. Each reference matching degree is determined based on the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image. If the variance of the matching degree is greater than the set variance threshold of the matching degree, then the set positioning requirements are met; otherwise, the set positioning requirements are not met.

4. The method according to claim 2, characterized in that, The step of determining the optimal direction based on the rotation angle of the mobile robot when the image acquisition component acquires an image that meets the set conditions includes: selecting the angle corresponding to the reference matching degree with the largest value in the current linked list, and determining the optimal direction based on the angle; wherein, the current linked list contains the correspondence between the reference matching degree and the angle; The reference matching degree in the current linked list refers to the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data in the visual map that matches the image. The angle refers to the rotation angle of the image acquisition component when acquiring the image during the rotation of the mobile robot, or... The angles in the current linked list are represented by angle windows. The angle windows in the current linked list are obtained by grouping the rotation angles of the mobile robot when the image acquisition component acquires the image. The reference matching degree in the current linked list is obtained by setting a mathematical operation between the matching degree between the image acquired by the image acquisition component and the visual map data that matches the image in the visual map when the mobile robot is at each rotation angle in the corresponding angle window.

5. The method according to claim 1, characterized in that, The movement of the mobile robot from its current first position along the optimal direction includes: the mobile robot moving along the optimal direction according to a specified action while activating obstacle avoidance; The distance between the target position and the current position in the optimal direction is a set distance. The set distance is used to ensure that the image data in at least one frame of the image acquisition component is different when the mobile robot rotates between its position before and after the movement.

6. The method according to claim 1, characterized in that, The matching degree between the image acquired by the image acquisition component and the visual map data that matches the image in the visual map is determined based on a weighted average of the following parameters: The matching score between the image and the visual map data that matches the image in the visual map, the number of identical feature point descriptors between the image and the visual map data that matches the image in the visual map, and the number of geometric verification points that satisfy the geometric constraints after the image and the visual map data that matches the image in the visual map are projected.

7. The method according to claim 1, characterized in that, Before relocalizing the mobile robot based on the constructed visual map, the method further includes: activating the relocalization mode of the mobile robot; After confirming successful relocation, the method further includes: disabling the relocation mode of the mobile robot.

8. The method according to claim 1, characterized in that, Before returning to relocalize the mobile robot based on the constructed visual map, the method further includes: checking whether the recorded number of relocalizations is a set number; if yes, ending the relocalization; if no, continuing to relocalize the mobile robot based on the constructed visual map. When relocalizing a mobile robot based on a constructed visual map, the method further includes: increasing the currently recorded number of relocalizations by a preset value.

9. A mobile robot, characterized in that, The mobile robot includes: a controller and an image acquisition component; A controller, configured to, during relocalization of a mobile robot based on a constructed visual map, allow the mobile robot to rotate around its current position and acquire images during this rotation via image acquisition components deployed on the mobile robot; and, If the matching degree between the image acquired by the image acquisition component at one of the rotation angles of the mobile robot and the visual map data matching that image in the visual map meets the repositioning condition, then the repositioning is determined to be successful; otherwise, the mobile robot starts moving from the current position along the optimal direction. The optimal direction is used to determine a movement route that is easy to reposition successfully based on the images acquired by the image acquisition component during the rotation of the mobile robot and the rotation angle of the mobile robot when acquiring the images, so as to control the robot to determine a position that is easy to reposition successfully when moving along the movement route. The optimal direction is determined based on whether the matching degree between each image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching that image in the visual map meets the set positioning requirements. When the matching degree variance is greater than the set matching degree variance threshold, it is determined that the set positioning requirements are met; otherwise, it is determined that the set positioning requirements are not met. The matching degree variance is obtained by performing variance calculations on each reference matching degree, and each reference matching degree is determined based on the matching degree between the image acquired by the image acquisition component during the rotation of the mobile robot and the visual map data matching that image in the visual map. During the process of the mobile robot moving along the optimal direction, if the matching degree between the image captured by the image acquisition component when the mobile robot moves to a certain position and the visual map data that matches the image in the visual map meets the relocalization condition, then the relocalization is determined to be successful; otherwise, after moving to the target position that meets the condition, the process returns to the step of relocalizing the mobile robot based on the constructed visual map. Image acquisition component, used to acquire images.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that can be executed; The machine-executable instructions are executed to implement the method steps of any one of claims 1-8.

Images