An intelligent mobile robot vision homing method based on multi-level road sign feature registration

Abstract

The application discloses a kind of intelligent mobile robot autonomous homing methods based on multi-scale landmark registration, comprising the following steps: obtaining and storing the panoramic image at the target position of intelligent mobile robot, and capturing the panoramic image at the current position of robot;Using VGG19 pre-training model respectively to the panoramic image at target position and current position carries out feature extraction to obtain 5 different scale feature map matching pairs;Using SURF feature registration algorithm respectively to n feature map matching pairs carries out image registration, to obtain multi-scale registration road sign;Using multi-scale registration road sign calculates the autonomous homing sub-direction of intelligent mobile robot under different scales;Using weight redistribution algorithm adjusts the weight of different scale homing sub-direction, and aggregates homing sub-direction to obtain the autonomous homing direction of mobile robot;The autonomous homing direction of intelligent mobile robot is updated regularly until mobile robot autonomous homing to target position.The application effectively improves the accuracy of robot autonomous homing.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot visual navigation technology, and relates to an intelligent mobile robot visual navigation method based on multi-level landmark feature registration. Background Technology

[0002] Autonomous navigation technology for intelligent mobile robots is a crucial technology in the field of intelligent perception and control for mobile robots. Visual homing, with its simple model, low computational cost, and ease of implementation, has become an important means of achieving autonomous navigation for intelligent mobile robots. Unlike simultaneous localization and mapping (SLAM) for mobile robots, visual homing does not require deep modeling of the robot's position and current environment information. Instead, it directly obtains landmarks by comparing the differences between two panoramic images of the robot's current and target locations. It then uses these landmarks to infer the robot's homing direction and continuously updates this direction to autonomously control the robot to reach the target location.

[0003] Current problems with visual homing technology include:

[0004] 1. Most home navigation algorithms struggle to extract abundant landmarks, especially landmarks in panoramic images at different scales, making it difficult for mobile robots to autonomously infer a home navigation direction that can accurately point to the target location, resulting in a decrease in the accuracy of autonomous home navigation of mobile robots;

[0005] 2. While increasing the number of landmarks acquired through algorithms can improve the accuracy of visual homing algorithms, it also increases the computational cost burden, making it difficult to achieve autonomous and rapid homing for mobile robots. Summary of the Invention

[0006] To address the following issues: 1. Most homing algorithms struggle to extract abundant landmarks, especially those at different scales in panoramic images, making it difficult for mobile robots to autonomously infer accurate homing directions and thus reducing the accuracy of autonomous homing; 2. While increasing the number of landmarks acquired through algorithms can improve the accuracy of visual homing algorithms, it also increases computational costs, hindering the achievement of rapid autonomous homing for mobile robots. Therefore, this invention provides the following technical solution:

[0007] An autonomous homing method for intelligent mobile robots based on multi-scale land reference registration, characterized by the following steps:

[0008] Acquire and store panoramic images of the target location of the intelligent mobile robot, and capture panoramic images of the robot's current location;

[0009] The VGG19 pre-training model is used for feature extraction of panoramic images at the target position and the current position to obtain five feature map matching pairs of different scales;

[0010] The SURF feature matching algorithm is used for image registration of the n feature map matching pairs to obtain multi-scale registration road signs;

[0011] The multi-scale registration road signs are used to calculate intelligent mobile robot autonomous homing sub-directions at different scales; the weight redistribution algorithm is used to adjust the weights of the homing sub-directions at different scales, and the homing sub-directions are aggregated to obtain the autonomous homing direction of the mobile robot;

[0012] The autonomous homing direction of the intelligent mobile robot is updated in real time until the mobile robot autonomously homing to the target position.

[0013] Further, the VGG19 pre-training model uses the feature extraction part of the VGG19 network.

[0014] Further, the VGG19 pre-training model is used for feature extraction of panoramic images at the target position and the current position to obtain five feature map matching pairs of different scales as follows:

[0015] The panoramic image I T at the target position of the intelligent mobile robot and the panoramic image I C at the current position are input into the VGG19 pre-training model respectively;

[0016] The output feature maps of the panoramic image I T at the target position in the VGG19 pre-training model Stage0, Stage1, Stage2, Stage3 and Stage4 are extracted, and are denoted as f T0 , f T1 , f T2 , f T3 and f T4 respectively;

[0017] The output feature maps of the panoramic image I C at the current position in the VGG19 pre-training model Stage0, Stage1, Stage2, Stage3 and Stage4 are extracted, and are denoted as f C0 , f C1 , f C2 , f C3 and f C4 respectively;

[0018] The feature maps with the same scale from the panoramic image at the target position and the current position are matched as a pair, thereby obtaining five feature map matching pairs of different scales {f T0 , fC0}、{f T1 ,f C1}、{f T2 ,f C2}、{f T3 ,f C3}、{f T4 ,f C4}, abbreviated as {f Ti ,f Ci}, i = 0, 1, 2, 3, 4.

[0019] Furthermore, the steps for using the SURF feature registration algorithm to perform image registration on n feature map matching pairs to obtain multi-scale registration landmarks are as follows:

[0020] S31: For feature matching pairs {f} at each scale Ti ,f Ci}, i=0,1,2,3,4, using the SURF feature matching algorithm, extract the features of the two feature maps in the feature map matching pair respectively, and register the above features to generate a feature matching pair, denoted as the landmark pair {L} of the intelligent mobile robot at this scale. Ti,1 ,L Ci,1}, {L Ti,2 ,L Ci,2}…,{L Ti,n ,L Ci,n}, where each landmark is essentially its pixel location coordinates in the feature map, and n is the total number of landmark pairs extracted at this scale;

[0021] S32: Following step S31, obtain the land reference pairs for the intelligent mobile robot across all five scales.

[0022] Furthermore, the calculation of the autonomous homing sub-directions of the intelligent mobile robot at different scales using multi-scale registration landmarks specifically includes:

[0023] S41: For all landmark pairs at the i-th scale, calculate the direction vector for each landmark pair, and for the j-th landmark pair at the i-th scale {L Ti,j ,L Ci,j}, where i = 0, 1, 2, 3, 4, j = 1, 2, ..., n; then the direction vector of this landmark pair is:

[0024]

[0025] Among them, {x Ti,j ,y Ti,j} is the land standard L Ti,j The pixel position coordinates, {x Ci,j ,y Ci,j} is the land standard L Ci,jpixel position coordinates of the i-th landmark;

[0026] S42: calculate the average direction vector of all landmark pairs at the i-th scale, that is, the intelligent mobile robot autonomous homing sub-direction at the current scale:

[0027]

[0028] Similarly, the intelligent mobile robot autonomous homing sub-directions at the 0th, 1st, 2nd, 3rd and 4th scales can be calculated respectively, and are denoted as h0, h1, h2, h3 and h4 respectively.

[0029] Further, the weight redistribution algorithm is used to adjust the weights of the homing sub-directions at different scales, and the homing sub-directions are aggregated to obtain the mobile robot autonomous homing direction, which specifically includes:

[0030] S51: assuming that the sizes of the feature maps at the 0th, 1st, 2nd, 3rd and 4th scales are a0×b0, a1×b1, a2×b2, a3×b3 and a4×b4 respectively, the weight ω0 at the 0th scale is set to 1, and the weight redistribution at the 1st-4th scales is:

[0031]

[0032] where k = 1, 2, 3, 4, and ω1, ω2, ω3 and ω4 are the weights at the 1st-4th scales respectively;

[0033] S52: calculate the mobile robot autonomous homing direction as h = ω0h0+ ω1h1+ ω2h2+ ω3h3+ ω4h4.

[0034]

[0035] Further, the mobile robot autonomous homing direction is updated in time until the mobile robot autonomously homes to the target position, which specifically includes the following steps:

[0036] S61: control the mobile robot to move to the target position using the calculated mobile robot autonomous homing direction h;

[0037] S62: stop moving after the mobile robot moves in the direction h for a period of time t;

[0038] S63: after the robot stops moving, determine whether the robot has reached the target position, if so, stop all subsequent movement of the robot, and the autonomous homing is completed; if not, control the robot to autonomously capture a panoramic image at the current position, overlay the previous panoramic image at the current position, and start S62 again until the robot reaches the target position.

[0039] An intelligent mobile robot autonomous homing device based on multi-scale landmark registration, comprising:

[0040] panoramic image capturing module: for acquiring and storing panoramic images at the target position of the intelligent mobile robot, and capturing panoramic images at the current position of the robot;

[0041] multi-scale feature extraction module: for extracting features from the panoramic images at the target position and the current position using a VGG19 pre-trained model to obtain n pairs of feature maps at different scales;

[0042] image registration module: for performing image registration on the n pairs of feature maps using a SURF feature registration algorithm to obtain multi-scale registration landmarks;

[0043] homing vector calculation module: for calculating intelligent mobile robot autonomous homing sub-directions at different scales using multi-scale registration landmarks; adjusting the weights of the homing sub-directions at different scales using a weight redistribution algorithm, and aggregating the homing sub-directions to obtain a mobile robot autonomous homing direction;

[0044] updating module: for updating the intelligent mobile robot autonomous homing direction at regular intervals until the mobile robot autonomously homing to the target position.

[0045] A computer readable storage medium having stored therein a computer program comprising a computer program, wherein the computer program, when executed, performs the method of autonomous homing of an intelligent mobile robot based on multi-scale landmark registration.

[0046] An autonomous homing device for an intelligent mobile robot based on multi-scale landmark registration, comprising:

[0047] panoramic image capturing module: for acquiring and storing panoramic images at the target position of the intelligent mobile robot, and capturing panoramic images at the current position of the robot;

[0048] multi-scale feature extraction module: for extracting features from the panoramic images at the target position and the current position using a VGG19 pre-trained model to obtain 5 pairs of feature maps at different scales;

[0049] image registration module: for performing image registration on the n pairs of feature maps using a SURF feature registration algorithm to obtain multi-scale registration landmarks;

[0050] homing vector calculation module: for calculating intelligent mobile robot autonomous homing sub-directions at different scales using multi-scale registration landmarks; adjusting the weights of the homing sub-directions at different scales using a weight redistribution algorithm, and aggregating the homing sub-directions to obtain a mobile robot autonomous homing direction;

[0051] An updating module is configured to update the autonomous homing direction of the intelligent mobile robot in a timely manner until the mobile robot autonomously homes to the target position.

[0052] A computer readable storage medium, the computer readable storage medium stores a computer program, wherein the computer program runs to execute the method.

[0053] The application provides an intelligent mobile robot visual homing method based on multi-level landmark feature registration, which has the following advantages:

[0054] 1. By using the VGG19 pre-training model, the multi-scale feature maps of the target position and the current position panoramic image are obtained, and the SURF feature registration algorithm is used to match the two feature maps under different scales, so that more rich and accurate landmark information is provided for the mobile robot.

[0055] 2. The weight redistribution algorithm is used to adjust the weights of the homing sub-directions under different scales, so as to optimize the contribution of the homing sub-directions obtained by the landmarks under different scales to the final result (i.e. the autonomous homing direction of the mobile robot), and further improve the accuracy of the autonomous homing direction of the mobile robot.

[0056] 3. Since the VGG19 pre-training model and the SURF feature registration algorithm can be regarded as lightweight models (the former has been pre-trained, and the model will not occupy too much operation cost during running, and the latter is a lightweight feature registration algorithm), the process of calculating the autonomous homing direction of the mobile robot is fast.

[0057] 4. As described above, the application can fully obtain the multi-scale landmarks of the mobile robot in the current environment, effectively improve the accuracy of the autonomous homing of the robot, greatly reduce the calculation cost in the autonomous homing process of the mobile robot, and effectively realize the autonomous and accurate homing of the mobile robot. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0059] Figure 1 is a method flowchart of the embodiment of the present application;

[0060] Figure 2 is a VGG19 pre-training model structure diagram provided by the embodiment of the present application;

[0061] Figure 3 (a) SURF registration result map of Stage 0 feature map, (b) SURF registration result map of Stage 1 feature map, (c) SURF registration result map of Stage 2 feature map, (d) SURF registration result map of Stage 3 feature map, (e) SURF registration result map of Stage 4 feature map. DETAILED DESCRIPTION

[0062] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict, and the present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0063] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. The description of the at least one exemplary embodiment is actually only illustrative, but not intended to limit the present application and its application or use in any way. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0064] It should be noted that the terms used herein are only intended to describe specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a reference to the presence of a feature, step, operation, device, component and / or combinations thereof.

[0065] Unless specifically stated otherwise, the relative arrangements of the components and steps illustrated in these embodiments and the numerical expressions and values set forth herein are not limiting of the scope of the present application. It should be understood that the various parts of the drawings are not necessarily drawn to scale, and that, for the purpose of convenience and clarity, not all components and steps can be shown in a given figure. Techniques, methods, and apparatus known to those of ordinary skill in the art can not be discussed in detail, but are intended to be a part of the present application. In all examples shown and discussed herein, any specific values are intended to be exemplary only, and are not to be construed as limiting. Other examples of the exemplary embodiments can have different values. It should be noted that like reference numerals and letters in the various figures indicate like elements, and thus, once an element is defined in one figure, it should not have to be discussed further in subsequent figures.

[0066] In the description of the present application, it should be understood that the orientation words such as "front, back, up, down, left, right", "transverse, vertical, perpendicular, horizontal" and "top, bottom" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and in the absence of the opposite description, these orientation words do not indicate and imply that the devices or elements referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the scope of protection of the present application: the orientation words "inner, outer" refer to the inner and outer of the contour of each component itself.

[0067] For the convenience of description, spatial relative terms such as "over", "above", "upper surface", "upper" and the like can be used herein to describe the spatial positional relationship of one device or feature with other devices or features as shown in the drawings. It should be understood that the spatial relative terms are intended to include different orientations in use or operation in addition to the orientation of the devices described in the drawings. For example, if the devices in the drawings are inverted, the device described as "above" or "over" other devices or structures will be positioned "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below" orientations. The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein are interpreted accordingly.

[0068] In addition, it should be noted that the use of the words "first", "second" and the like to define parts is only for the convenience of distinguishing the corresponding parts, and the above words have no special meaning unless otherwise stated, and therefore cannot be understood as a limitation on the scope of protection of the present application.

[0069] A self-homing method for an intelligent mobile robot based on multi-scale landmark registration, comprising the following steps:

[0070] S1: acquiring and storing panoramic images at a target position of the intelligent mobile robot, and capturing panoramic images at a current position of the robot;

[0071] S2: using a VGG19 pre-training model to extract features from the panoramic images at the target position and the current position respectively to obtain five different scale feature map matching pairs;

[0072] S3: using a SURF feature registration algorithm to perform image registration on the n feature map matching pairs respectively to obtain multi-scale registration road signs;

[0073] S4: Calculate the autonomous homing sub-directions of the intelligent mobile robot at different scales using multi-scale registration landmarks; adjust the weights of the homing sub-directions at different scales using a weight redistribution algorithm, and aggregate the homing sub-directions to obtain the autonomous homing direction of the mobile robot;

[0074] S5: Periodically update the autonomous homing direction of the intelligent mobile robot until the mobile robot autonomously homs to the target location.

[0075] Steps S1 / S2 / S3 / S4 / S5 are executed sequentially;

[0076] Furthermore, the VGG19 pre-trained model utilizes the feature extraction module of the VGG19 network. Specifically, the VGG19 pre-trained model is a VGG19 network pre-trained on the ImageNet dataset with the final fully connected layer removed.

[0077] Furthermore, the process of using the VGG19 pre-trained model to extract features from the panoramic images at the target location and the current location to obtain five feature map matching pairs at different scales is as follows:

[0078] The panoramic images of the target location of the intelligent mobile robot are respectively... T Panoramic image of the current location I C Input into the VGG19 pre-trained model;

[0079] Extracting panoramic images of the target location I T In the VGG19 pre-trained model, the output feature maps of Stage 0, Stage 1, Stage 2, Stage 3, and Stage 4 are denoted as f. T0 f T1 f T2 f Ti f T4 ;

[0080] Extract panoramic image of the current location I C In the VGG19 pre-trained model, the output feature maps of Stage 0, Stage 1, Stage 2, Stage 3, and Stage 4 are denoted as f. C0 f C1 f C2 f C3 f C4 ;

[0081] Feature maps of the same scale from the panoramic images of the target location and the current location are considered as a pair of matching maps, thus obtaining n feature map matching pairs of different scales {f T0 ,f C0}、{f T1 ,f C1}、{fT2 ,f C2}、{f T3 ,f C3}、{f T4 ,f C4}, abbreviated as {f Ti ,f Ci}, i = 0, 1, 2, 3, 4.

[0082] Furthermore, the steps for using the SURF feature registration algorithm to perform image registration on n feature map matching pairs to obtain multi-scale registration landmarks are as follows:

[0083] S31: For feature matching pairs {f} at each scale Tx ,f Cx Given x = 0, 1, 2, 3, 4, use the SURF feature matching algorithm to extract features from the two feature maps in the feature map matching pair.

[0084] The above features are then registered to generate feature matching pairs, denoted as the landmark pair {L} of the intelligent mobile robot at this scale. Ti,1 ,L Ci,1}, {L Ti,2 ,L Ci,2}…,{L Ti,n ,L Ci,n}, where each landmark is essentially its pixel location coordinates in the feature map, and n is the total number of landmark pairs extracted at this scale;

[0085] S32: Following step S31, obtain the land reference pairs for the intelligent mobile robot across all five scales.

[0086] Furthermore: the calculation of autonomous homing sub-directions for intelligent mobile robots at different scales using multi-scale registration landmarks specifically includes:

[0087] S41: For all landmark pairs at the i-th scale, calculate the direction vector for each landmark pair, and for the j-th landmark pair at the i-th scale {L Ti,j ,L Ci,j}, i = 0, 1, 2, 3, 4, j = 1, 2, ..., n; then the direction vector of the landmark pair is:

[0088]

[0089] Among them, {x Ti,j ,y Ti,j} is the land standard L Ti,j The pixel position coordinates, {x Ci,j ,y Ci,j} is the land standard L Ci,j The pixel position coordinates;

[0090] S42: Calculate the average direction vector of all landmark pairs in the i-th scale, that is, the autonomous homing sub-direction of the intelligent mobile robot in the current scale:

[0091]

[0092] Similarly, the autonomous homing sub-directions of the intelligent mobile robot in the 0th, 1st, 2nd, 3rd, and 4th scales can be calculated respectively, denoted as h0, h1, h2, h3, and h4 respectively.

[0093] Further, the weight adjustment algorithm is used to adjust the weights of the homing sub-directions in different scales, and the homing sub-directions are aggregated to obtain the autonomous homing direction of the mobile robot, which specifically includes:

[0094] S51: Assuming that the sizes of the feature maps in the 0th, 1st, 2nd, 3rd, and 4th scales are a0×b0, a1×b1, a2×b2, a3×b3, and a4×b4 respectively, the weight ω0 in the 0th scale is set to 1, and the weight redistribution in the 1st-4th scales is:

[0095]

[0096] where k = 1, 2, 3, 4, ω1, ω2, ω3, and ω4 are the weights in the 1st-4th scales respectively. 4.

[0097] S52: Calculate the autonomous homing direction of the mobile robot as h = ω0h0+ ω1h1+ ω2h2+ ω3h3+ ω4h4.

[0098]

[0099] Further, the autonomous homing direction of the mobile robot is updated in time until the mobile robot autonomously homes to the target position as follows:

[0100] S61: Use the calculated autonomous homing direction h of the mobile robot to control the movement of the mobile robot to the target position;

[0101] S62: After the mobile robot moves in the direction h for a period of time t, stop moving;

[0102] S63: After the robot stops moving, determine whether the robot has reached the target position. If so, stop all subsequent movement of the robot and the autonomous homing is complete. If not, control the robot to autonomously capture a panoramic image of the current position, overlay the previous panoramic image of the current position, and start step S62 again until the robot reaches the target position.

[0103] An autonomous homing device for an intelligent mobile robot based on multi-scale landmark registration, comprising:

[0104] ​a panoramic image capturing module for acquiring and storing panoramic images at the target position of the intelligent mobile robot and capturing panoramic images at the current position of the robot;

[0105] a multi-scale feature extraction module for extracting features from the panoramic images at the target position and the current position using a VGG19 pre-trained model to obtain five different scale feature map matching pairs;

[0106] an image registration module for registering the n feature map matching pairs using a SURF feature registration algorithm to obtain multi-scale registration landmarks;

[0107] a homing vector calculation module for calculating intelligent mobile robot autonomous homing sub-directions at different scales using the multi-scale registration landmarks, adjusting the weights of the homing sub-directions at different scales using a weight redistribution algorithm, and aggregating the homing sub-directions to obtain a mobile robot autonomous homing direction;

[0108] an updating module for updating the intelligent mobile robot autonomous homing direction at regular intervals until the mobile robot autonomously homing to the target position.

[0109] A computer readable storage medium having a computer program stored therein, wherein the computer program, when executed, performs the method of autonomous homing of an intelligent mobile robot based on multi-scale landmark registration.

[0110] Figure 1 is a method flowchart of an embodiment of the application;

[0111] Embodiment 1:

[0112] An autonomous homing method of an intelligent mobile robot based on multi-scale landmark registration, comprising the following steps:

[0113] Step 1: Pre-acquire and store panoramic images at the target position of the intelligent mobile robot, and capture panoramic images at the current position of the robot;

[0114] Step 2: Extract features from the panoramic images at the target position and the current position using a VGG19 pre-trained model to obtain five different scale feature map matching pairs;

[0115] Step 3: Register the n feature map matching pairs using a SURF feature registration algorithm to obtain multi-scale registration landmarks;

[0116] Step 4: Calculate intelligent mobile robot autonomous homing sub-directions at different scales using the multi-scale registration landmarks, adjust the weights of the homing sub-directions at different scales using a weight redistribution algorithm, and aggregate the homing sub-directions to obtain an intelligent mobile robot autonomous homing direction;

[0117] Step 5: Timing updates the autonomous homing direction of the intelligent mobile robot until the intelligent mobile robot autonomously homes to the target position.

[0118] Step 1 / Step 2 / Step 3 / Step 4 / Step 5 are executed in sequence;

[0119] The "pre-acquiring and storing the panoramic image at the target position of the intelligent mobile robot" in Step 1 specifically refers to pre-controlling the intelligent mobile robot to take a panoramic image at the target position, and storing the image in the intelligent mobile robot control system; The "capturing the panoramic image at the current position of the intelligent mobile robot" specifically refers to controlling the intelligent mobile robot to take a panoramic image at the current position after the homing behavior of the intelligent mobile robot starts, and temporarily storing the image in the intelligent mobile robot control system;

[0120] The step 2: using VGG19 pre-training model (remove) respectively to the panoramic image at the target position and the current position carries out feature extraction to obtain 5 different scale feature map matching pair including the following steps:

[0121] Step 2-1: Select VGG19 pre-training model; Figure 2 is the VGG19 pre-training model structure diagram provided by the embodiment of the application;

[0122] Step 2-2: respectively, the panoramic image I T at the target position of the intelligent mobile robot in step 1 and the panoramic image I C at the current position are input into the VGG19 pre-training model;

[0123] Step 2-3: extracting the output feature maps of Stage0, Stage1, Stage2, Stage3 and Stage4 of the VGG19 pre-training model in the panoramic image I T at the target position, respectively denoted as f T0 , f T1 , f T2 , f T3 and f T4 ;

[0124] Step 2-4: extracting the output feature maps of Stage0, Stage1, Stage2, Stage3 and Stage4 of the VGG19 pre-training model in the panoramic image I C at the current position, respectively denoted as f C0 , f C1 , f C2 , f C3 and f C4 ;

[0125] Step 2-5: Define the feature maps with the same scale from the target location and the current location as a matching pair, thereby obtaining five matching pairs of feature maps with different scales {f T0 ,f C0}、{f T1 ,f C1}、{f T2 ,f C2}、{f T3 ,f C3} and {f T4 ,f C4}, which are denoted as {f Ti ,f Ci}, i = 0, 1, 2, 3, 4.

[0126] The step 3: performing image registration on the five matching pairs of feature maps respectively by using the SURF feature registration algorithm to obtain multi-scale registration landmarks includes the following steps:

[0127] Step 3-1: for each matching pair of features {f Ti ,f Ci}, i = 0, 1, 2, 3, 4, at each scale, the features of the two feature maps in the matching pair are extracted respectively by using the SURF feature matching algorithm, and the features are registered to generate a matching pair of features, denoted as a landmark pair {L Ti,1 ,L Ci,1} of the intelligent mobile robot at the scale, {L Ti,2 ,L Ci,2}…, {L Ti,n ,L Ci,n}, wherein each landmark is essentially the pixel position coordinates thereof in the feature map, and n is the total number of landmark pairs extracted at the scale;

[0128] Step 3-2: by analogy with step 3-1, the landmark pairs of the intelligent mobile robot at all five scales are obtained, and all the landmark pair data are temporarily stored in the intelligent mobile robot control system;

[0129] The step 4: calculating the autonomous homing sub-directions of the intelligent mobile robot at different scales by using the multi-scale registration landmarks, specifically including:

[0130] S4-1: for all the landmark pairs at a certain scale, the direction vector of each landmark pair is calculated. Taking the first group of landmark pairs {L T0,1 ,L C0,1} at the 0th scale as an example, the direction vector of the landmark pair is:

[0131]

[0132] wherein, {xT0,1 , y T0,1} are pixel position coordinates of the landmark L T0,1 , y C0,1} are pixel position coordinates of the landmark L C0,1 , y C0,1} are pixel position coordinates of the landmark L

[0133] S4-2: Based on S4-1, the average direction vector of all landmark pairs at this scale is calculated, that is, the intelligent mobile robot autonomous homing sub-direction at the current scale:

[0134]

[0135] Similarly, the intelligent mobile robot autonomous homing sub-directions at the 1st, 2nd, 3rd, and 4th scales can be calculated respectively, and are denoted as h1, h2, h3, and h4 respectively.

[0136] The "adjusting the weights of the homing sub-directions at different scales by using a weight redistribution algorithm, and aggregating the homing sub-directions to obtain the intelligent mobile robot autonomous homing direction" specifically includes:

[0137] Step 5-1: Assuming that the sizes of the feature maps at the 0th, 1st, 2nd, 3rd, and 4th scales are a0×b0, a1×b1, a2×b2, a3×b3, and a4×b4 respectively, the weight ω0 at the 0th scale is set to 1, and the weight redistribution at the 1st-4th scales is:

[0138]

[0139] where k = 1, 2, 3, 4, and ω1, ω2, ω3, and ω4 are the weights at the 1st-4th scales respectively.

[0140] Step 5-2: The intelligent mobile robot autonomous homing direction is calculated as:

[0141]

[0142] The step 5: The process of updating the intelligent mobile robot autonomous homing direction in a timely manner until the mobile robot autonomously homes to the target position is as follows:

[0143] Step 6-1: The intelligent mobile robot is controlled to move towards the target position using the calculated intelligent mobile robot autonomous homing direction h;

[0144] Step 6-2: The intelligent mobile robot stops moving after moving in the direction h for a period of time t, and the specific value of t can be determined according to the actual situation, such as 0.5 seconds, 1 second, 2 seconds, etc. The shorter the time, the shorter the step length of the robot, the more frequent the update of the current position panoramic image, and the more frequent the update of the autonomous homing direction h, and vice versa.

[0145] Step 6-3: After the intelligent mobile robot stops moving, it is determined whether the intelligent mobile robot has reached the target position. If so, all subsequent movement behaviors of the intelligent mobile robot are manually stopped, and the autonomous homing is ended. If not, the intelligent mobile robot is controlled to autonomously capture a panoramic image at the current position, overlay the previous panoramic image at the current position, and restart Step 2 until the intelligent mobile robot reaches the target position.

[0146] The determination of whether the robot has reached the target position is achieved by comparing the picture taken after stopping moving with the target position picture. When the picture taken after stopping moving is consistent with the target position picture, it is determined that the intelligent mobile robot has reached the target position. When the picture taken after stopping moving is inconsistent with the target position picture, it is determined that the intelligent mobile robot has not reached the target position.

[0147] Figure 3 (a) SURF registration result map of Stage 0 feature map, (b) SURF registration result map of Stage 1 feature map, (c) SURF registration result map of Stage 2 feature map, (d) SURF registration result map of Stage 3 feature map, and (e) SURF registration result map of Stage 4 feature map.

[0148] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for autonomous homing of an intelligent mobile robot based on multi-scale land reference registration, characterized in that, Includes the following steps: Acquire and store panoramic images of the target location of the intelligent mobile robot, and capture panoramic images of the robot's current location; The VGG19 pre-trained model was used to extract features from the panoramic images at the target location and the current location to obtain five feature map matching pairs at different scales. The SURF feature registration algorithm is used to perform image registration on n feature map matching pairs to obtain multi-scale registration landmarks. Multi-scale registration landmarks are used to calculate the autonomous homing sub-directions of the intelligent mobile robot at different scales; a weight redistribution algorithm is used to adjust the weights of the homing sub-directions at different scales, and the homing sub-directions are aggregated to obtain the autonomous homing direction of the mobile robot. The autonomous homing direction of the intelligent mobile robot is updated periodically until the mobile robot autonomously returns to the target location. The step of adjusting the weights of different scale homing sub-directions using a weight redistribution algorithm and aggregating the homing sub-directions to obtain the autonomous homing direction of the mobile robot specifically includes: S51: Assume the dimensions of the feature maps at scales 0, 1, 2, 3, and 4 are respectively , , , as well as Then set the weights at scale 0. The weight redistribution for scales 1-4 is set to 1: （3） in ,but , , , These are the weights for scales 1 through 4, respectively. S52: Calculate the autonomous homing direction of the mobile robot as follows: （4）。 2. The method for autonomous homing of an intelligent mobile robot based on multi-scale land reference registration according to claim 1, characterized in that, The VGG19 pre-trained model utilizes the feature extraction part of the VGG19 network.

3. The method for autonomous homing of an intelligent mobile robot based on multi-scale land reference registration according to claim 2, characterized in that, The process of using the VGG19 pre-trained model to extract features from the panoramic images at the target location and the current location to obtain five feature map matching pairs at different scales is as follows: Panoramic images of the target location of the intelligent mobile robot are respectively... Panoramic image of the current location Input into the VGG19 pre-trained model; Extract panoramic image of target location The output feature maps of Stage 0, Stage 1, Stage 2, Stage 3, and Stage 4 in the VGG19 pre-trained model are denoted as follows: , , , , ; Extract panoramic image of the current location In the VGG19 pre-trained model, the output feature maps of Stage 0, Stage 1, Stage 2, Stage 3, and Stage 4 are denoted as follows: , , , , ; Feature maps of the same scale from the panoramic images of the target location and the current location are paired as a single matching pair, thus obtaining five feature map matching pairs of different scales. , , , , abbreviated as .

4. The method for autonomous homing of an intelligent mobile robot based on multi-scale land reference registration according to claim 1, characterized in that, The SURF feature registration algorithm was used to respectively... n The steps for image registration using feature map matching pairs to obtain multi-scale registration landmarks are as follows: S31: Feature matching pairs for each scale Using the SURF feature matching algorithm, features are extracted from the two feature maps in the feature map matching pair, and these features are registered to generate a feature matching pair, denoted as the landmark pair of the intelligent mobile robot at this scale. , …, Each landmark is essentially its pixel location coordinates in the feature map. n This represents the total number of land landmark pairs extracted at this scale; S32: Following step S31, obtain the land reference pairs for the intelligent mobile robot across all five scales.

5. The autonomous homing method for an intelligent mobile robot based on multi-scale land reference registration according to claim 1, characterized in that: The method of using multi-scale registration landmarks to calculate the autonomous homing sub-directions of intelligent mobile robots at different scales specifically includes: S41: For the first i For all landmark pairs at the scale, calculate the direction vector for each landmark pair, i.e., the first... i The first scale j Group of land standard pairs ,in, i =0,1,2,3,4 j =1,2,..., n Then the direction vector of the landmark pair is: （1） in, Landmark pixel position coordinates, Landmark The pixel position coordinates; S42: Calculate the... i The average direction vector of all landmark pairs at the current scale is the autonomous homing sub-direction of the intelligent mobile robot at the current scale: （2） Similarly, the autonomous homing sub-directions of the intelligent mobile robot can be calculated at scales 0, 1, 2, 3, and 4, respectively, and denoted as [examples of sub-directions]. , , , , .

6. The method for autonomous homing of an intelligent mobile robot based on multi-scale land reference registration according to claim 1, characterized in that: The process of periodically updating the mobile robot's autonomous homing direction until the mobile robot autonomously homs to the target location includes the following steps: S61: Calculated autonomous homing direction of the mobile robot Control the mobile robot to move to the target location; S62: In the mobile robot according to Shift direction for a period of time t Then, stop moving; S63: After the robot stops moving, determine whether the robot has reached the target position. If it has, stop all subsequent movement of the robot and end the autonomous return journey. If the target location is not reached, the robot will autonomously capture a panoramic image of its current location, overwrite the previous panoramic image of the current location, and restart S62 until the robot reaches the target location.

7. An autonomous homing device for intelligent mobile robots based on multi-scale land standard registration, characterized in that, include: Panoramic image capture module: used to acquire and store panoramic images of the target location of the intelligent mobile robot, and to capture panoramic images of the robot's current location; Multi-scale feature extraction module: Used to extract features from the panoramic images at the target location and the current location using the VGG19 pre-trained model to obtain five feature map matching pairs at different scales; Image registration module: used to perform image registration using the SURF feature registration algorithm. n Image registration is performed using feature map matching pairs to obtain multi-scale registration landmarks; Homecoming Vector Calculation Module: Used to calculate the autonomous homecoming sub-directions of intelligent mobile robots at different scales using multi-scale registration landmarks; adjust the weights of homecoming sub-directions at different scales using a weight redistribution algorithm, and aggregate the homecoming sub-directions to obtain the autonomous homecoming direction of the mobile robot; The step of adjusting the weights of different scale homing sub-directions using a weight redistribution algorithm and aggregating the homing sub-directions to obtain the autonomous homing direction of the mobile robot specifically includes: S51: Assume the dimensions of the feature maps at scales 0, 1, 2, 3, and 4 are respectively , , , as well as Then set the weights at scale 0. The weight redistribution for scales 1-4 is set to 1: （3） in ,but , , , These are the weights for scales 1 through 4, respectively. S52: Calculate the autonomous homing direction of the mobile robot as follows: （4） Update module: Used to periodically update the autonomous homing direction of the intelligent mobile robot until the mobile robot autonomously homs to the target location.

8. A computer-readable storage medium storing a computer program, wherein, When the computer program is executed, it performs the autonomous homing method for an intelligent mobile robot based on multi-scale land standard registration as described in any one of claims 1-6.

Images