A robot autonomous repositioning method, device, robot and storage medium

Abstract

The present invention discloses a robot autonomous repositioning method, device, robot and storage medium, the method comprising: obtaining a fusion map and a deep visual image and laser scanning data of the current position; wherein the fusion map is generated by a visual map, a visual word bag and a laser grid map; using the deep visual image to match in the fusion map, determining the robot's initial posture with the highest matching degree in the fusion map; when the matching degree is greater than or equal to a preset threshold, using the initial posture and laser scanning data to determine the robot's precise posture in the fusion map; when the matching degree is less than the preset threshold, controlling the robot to move to the next position, until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, using the initial posture and laser scanning data of the next position to determine the precise posture. This method can ensure the reliability and accuracy of the robot's autonomous repositioning at any position, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to a robot autonomous repositioning method, a device, a robot and a storage medium. Background Art

[0002] With the continuous development of robotics technology, autonomous mobile robots have been widely used in industries such as industry and service. These robots can use built-in map data for navigation and movement, but navigation and movement need to be based on the alignment of the robot's actual position and posture with the built-in map data. At the same time, in actual applications, the robot may restart, so how to reposition the robot when it restarts is an important technical issue in the field of robotics.

[0003] In the related art, in order to reposition the robot when it restarts, the robot is usually required to restart at a fixed position, or to manually set the initial position in the built-in map. The above method not only brings trouble to the user when using the robot, but also when the robot encounters an unexpected restart, it will not be able to determine the restart position, which will bring obstacles to the operation of the robot. Summary of the invention

[0004] The purpose of the present invention is to provide a robot autonomous repositioning method, device, robot and storage medium, which can use the depth vision image and laser scanning data collected by the robot at the current position for repositioning, thereby ensuring the reliability and accuracy of the robot's autonomous repositioning at any position and improving the user experience.

[0005] In order to solve the above technical problems, the present invention provides a robot autonomous repositioning method, comprising:

[0006] Acquire a fused map and a deep visual image and laser scanning data of the current location; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map;

[0007] Matching the deep vision image in the fused map to determine the initial posture of the robot with the highest matching degree in the fused map;

[0008] When the matching degree is greater than or equal to a preset threshold, determining the precise position and posture of the robot in the fused map using the initial position and posture and the laser scanning data;

[0009] When the matching degree is less than the preset threshold, the robot is controlled to move to the next position until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and the precise posture is determined using the initial posture of the next position and the laser scanning data.

[0010] Optionally, controlling the robot to move to a next position comprises:

[0011] Controlling the robot to move according to a preset mode and determining a preset number of candidate positions during the movement; the preset mode includes rotation and translation;

[0012] The initial position of the candidate position is set as the first candidate position, and the candidate position corresponding to the first candidate position with the largest matching degree is set as the next position;

[0013] The robot is controlled to move to the next position according to the preset mode.

[0014] Optionally, the determining the precise posture of the robot in the fused map by using the initial posture and the laser scanning data includes:

[0015] generating an initial region in the fused map according to the position of the initial pose;

[0016] The laser scanning data is used to perform matching in the initial area to determine the precise position and posture.

[0017] Optionally, the using the laser scanning data to perform matching in the initial area to determine the precise position and posture includes:

[0018] Downsampling the initial area to obtain a low-precision grid image;

[0019] Using the laser scanning data to match in the low-precision grid map, a second candidate pose is obtained;

[0020] In the fused map, the laser scanning data is matched with the second candidate pose to obtain the precise pose.

[0021] Optionally, after matching in the initial area using the laser scanning data, the method further includes:

[0022] When the matching fails, the initial area is adjusted, and the adjusted initial area is used to perform the step of matching in the initial area using the laser scanning data.

[0023] Optionally, the using the deep vision image to match in the fused map to determine an initial posture of the robot with the greatest matching degree in the fused map includes:

[0024] Extracting ORB feature points in the deep vision image, and inputting the ORB feature points into the visual word bag of the fused map for matching, to obtain a candidate image corresponding to the deep vision image in the fused map;

[0025] Matching the ORB feature points in the depth vision image with the ORB feature points in the candidate image to obtain ORB feature point pairs, and inputting the ORB feature point pairs into the RANSAC algorithm for processing to remove mismatched ORB feature point pairs;

[0026] Determine the number of point pairs of the processed ORB feature point pairs, and calculate the matching degree using the number of point pairs and the number of ORB feature points in the depth vision image;

[0027] The candidate image with the greatest matching degree is used as a matching image, and the matching image and the depth vision image are input into a PNP algorithm for calculation to obtain the initial pose.

[0028] Optionally, the generation process of the fused map includes:

[0029] Obtain the depth image, visible light image and laser scanning data to be processed at the current acquisition position;

[0030] Extracting ORB feature points of the visible light image, and updating the visual word bag using the ORB feature points of the visible light image;

[0031] Inputting the visible light image and the historical visible light image of the last acquisition position into the g2o graph optimization model to obtain the visual pose of the robot;

[0032] Using the laser scanning data to be processed to update the laser grid map, and using the laser scanning data to be processed to perform laser positioning in the updated laser grid map to obtain the laser position and posture of the robot;

[0033] Inputting the visual pose and the laser pose into an EKF model to obtain the current pose of the robot;

[0034] Establishing a binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image, and the depth image, and updating the visual map using the binding relationship;

[0035] The fused map is updated using the updated visual word bag, laser grid map and visual map.

[0036] The present invention also provides a robot autonomous repositioning device, comprising:

[0037] An acquisition module, used to acquire a fused map and a deep visual image and laser scanning data of a current location; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map;

[0038] A first matching module, used to use the deep vision image to match the fused map, and determine the initial posture of the robot with the highest matching degree in the fused map;

[0039] A second matching module, configured to determine the precise position and posture of the robot in the fused map using the initial position and posture and the laser scanning data when the matching degree is greater than or equal to a preset threshold;

[0040] The motion control module is used to control the robot to move to the next position when the matching degree is less than the preset threshold, until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and determine the precise posture using the initial posture of the next position and the laser scanning data.

[0041] The present invention also provides a robot, comprising:

[0042] Memory for storing computer programs;

[0043] A processor is used to implement the steps of the robot autonomous repositioning method as described above when executing the computer program.

[0044] The present invention also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the robot autonomous repositioning method as described above are implemented.

[0045] The present invention provides a robot autonomous repositioning method, comprising: obtaining a fused map and a deep visual image and laser scanning data of a current position; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map; using the deep visual image to perform matching in the fused map, and determining an initial posture of the robot with the greatest matching degree in the fused map; when the matching degree is greater than or equal to a preset threshold, using the initial posture and the laser scanning data to determine an accurate posture of the robot in the fused map; when the matching degree is less than the preset threshold, controlling the robot to move to a next position, until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and then using the initial posture of the next position and the laser scanning data to determine the accurate posture.

[0046] It can be seen that the method first uses the deep visual image collected by the robot at the current position to match in the fusion map to determine the initial posture. Since the fusion map carries the feature information of the visual map and the visual word bag, the method can determine the initial posture of the robot by matching the deep visual image with the fusion map; the method further uses the laser scanning data and the initial posture to determine the precise posture of the robot. Since the fusion map also carries the feature information of the laser grid map, and the matching accuracy of the laser scanning data is higher than that of the deep visual image, the method can use the laser scanning data and the initial posture to improve the robot repositioning accuracy; finally, when the robot encounters a failure in the initial posture matching, that is, when the matching degree of the initial posture is less than the preset threshold, the method will also autonomously control the robot to move to obtain an initial posture with a higher matching degree, thereby ensuring the reliability and accuracy of the robot's autonomous repositioning at any position, without restarting at a fixed position or manually setting, which can effectively avoid the inconvenience caused by the fixed position restart or manual setting in the related technology to the robot repositioning, thereby improving the user experience. The present invention also provides a robot autonomous repositioning device, a robot and a storage medium, which have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0048] Figure 1 A flowchart of a robot autonomous repositioning method provided by an embodiment of the present invention;

[0049] Figure 2 This is a structural block diagram of a robot autonomous repositioning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In the related art, in order to reposition the robot when it restarts, the robot is usually required to restart at a fixed position, or to manually set the initial position in the built-in map. The above method not only brings trouble to the user when using the robot, but also when the robot encounters an unexpected restart, it will be unable to determine the restart position, thereby bringing obstacles to the operation of the robot. In view of this, the present invention provides a robot autonomous repositioning method, which can use the depth vision image and laser scanning data collected by the robot at the current position for repositioning, thereby ensuring the reliability and accuracy of the robot's autonomous repositioning at any position, and improving the user experience. Please refer to Figure 1 , Figure 1 A flowchart of a robot autonomous repositioning method provided by an embodiment of the present invention, the method may include:

[0052] S101, obtaining a fused map and a deep visual image and laser scanning data of a current location; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map.

[0053] In an embodiment of the present invention, the fused map used for robot relocation is jointly generated by a visual map, a visual word bag and a laser grid map, wherein the visual map carries texture information of visible light images and is composed of visible light images collected at each position and posture orientation. The visual map can be used to quickly and effectively determine the position and posture of the robot when there is sufficient light at the current position; the visual word bag is trained based on the visual map, which contains the feature information of all visible light images in the visual map, and can classify the visible light images according to the feature information, which can further improve the matching efficiency of the visual map; the laser grid map is generated by laser scanning data, contains high-precision laser scanning information, and can accurately match the position and posture of the robot under any lighting conditions. Since the embodiment of the present invention uses a fused map composed of a visual map, a visual word bag and a laser grid map for robot relocation, the robot position can be determined effectively and accurately.

[0054] It should be noted that the embodiments of the present invention do not limit the generation method of the visual word bag and the laser grid map, and reference may be made to the relevant technologies of the visual word bag and the laser grid map. It is understandable that the image training set based on the visual word bag is a visual map, which is used to extract the feature information of the visible light image corresponding to different robot positions and postures, and classify the deep visual image of the current position according to the above feature information, so as to quickly find the candidate image with a higher matching degree in the visual map. Further, the embodiments of the present invention do not limit the generation method of the visual map, as long as the visual map is generated using the visible light image collected at each position and posture, and ensures that each posture corresponds to the visible light image one by one, and reference may be made to the relevant technologies of the robot visual map. Further, the present invention does not limit the generation method of the fused map, as long as the fused map can establish a corresponding relationship between the visual map and the laser grid map according to the robot's posture, and establish a corresponding relationship between the visual map and the visual word bag according to the feature information of the visible light image corresponding to each posture.

[0055] Further, it can be understood that in the process of generating a visual map, the robot's posture needs to be determined before the corresponding relationship between the posture and the visible light image can be established. The embodiment of the present invention does not limit the method of determining the robot's posture. For example, the visible light image collected at adjacent positions can be used for feature point collection and feature point matching, and the successfully matched feature points can be input into the G2O graph optimization model to obtain the robot's visual posture, and finally the visual posture is used as the robot's posture; laser scanning data and laser grid map can also be used for laser positioning to obtain the robot's laser posture, and the laser posture is used as the robot's posture; of course, the visual posture can be fused with the laser posture, and the fusion result can be used as the robot's posture. Considering that the matching efficiency of the visual posture is high, but its accuracy is greatly affected by light, and the matching efficiency of the laser posture is low, but its accuracy is less affected by light, therefore, in the embodiment of the present invention, the robot's posture can be generated by the visual posture and the laser posture. Combining the above two postures, the relationship between the efficiency and accuracy of the robot's posture determination can be effectively balanced, and thus a high matching efficiency and matching accuracy can be simultaneously ensured. It should be noted that the embodiments of the present invention do not limit the specific content of the g2o graph optimization model, and the present invention does not limit the specific content of laser positioning, and reference may be made to related technologies.

[0056] Furthermore, the embodiments of the present invention do not limit the specific forms of the depth vision image and the laser scanning data, and reference may be made to the relevant technologies. It is understandable that the depth vision image and the laser scanning data are collected at the same time. The embodiments of the present invention do not limit the collection of the depth vision image and the laser scanning data to periodic collection or real-time collection, and may be set according to actual application requirements. It is also understandable that the current position is the position corresponding to the current image collection moment.

[0057] S102: Use the deep vision image to perform matching in the fused map to determine the initial posture of the robot with the highest matching degree in the fused map.

[0058] Since the fused map contains the texture information of the visible light image, and a one-to-one correspondence is established between the visible light image and the robot's posture, the embodiment of the present invention can determine the initial posture of the robot by image matching.

[0059] It is understandable that, since the fusion map used in the embodiment of the present invention contains visual word bag information, the visual word bag can be used for image matching to screen out candidate images similar to the deep visual image. It should be noted that the embodiment of the present invention does not limit the specific process of matching using the visual word bag, and reference may be made to related technologies. For example, local image features can be extracted as visual words, and the matching scores of the visual words can be calculated using the TF-IDF model, and finally similar candidate images can be screened out according to the matching scores. Since the visual word bag mainly uses local image features and statistical features of local image features to screen candidate images, its matching accuracy may not meet the application requirements. Therefore, after the candidate images are screened out, the candidate images can be matched with the deep visual image for feature points, and the matching degree can be calculated using the matching number of feature points. Finally, the best candidate image can be selected based on the matching degree. It should be noted that. The embodiment of the present invention does not limit the specific image feature points, for example, it can be SIFT feature points, ORB feature points or other types of feature points, wherein SIFT feature points (Scale-invariant feature transform) are a local feature descriptor with scale and rotation invariance; ORB (Oriented FAST and Rotated BRIEF) is a feature point that combines FAST feature points and BRIEF feature descriptors, and also has scale and rotation invariance, and its acquisition efficiency is much higher than SIFT feature points. Considering the superior performance of ORB feature points, ORB feature points can be used as image feature points in the embodiment of the present invention. It should be noted that the embodiment of the present invention does not limit the acquisition method and process of ORB feature points, and reference can be made to related technologies; the embodiment of the present invention also does not limit the process of image matching using ORB feature points, and reference can be made to related technologies of ORB feature point matching. Finally, it should be noted that when constructing a visual map, ORB feature points can also be extracted from the visible light image collected at each position and posture, and the ORB features can also be integrated into the visual map.

[0060] In one possible scenario, using the deep vision image to match in the fused map, determining the initial posture of the robot with the greatest matching degree in the fused map may include:

[0061] Step 11: Extract the ORB feature points in the deep vision image, and input the ORB feature points into the visual word bag of the fusion map for matching, so as to obtain the candidate image corresponding to the deep vision image in the fusion map;

[0062] Step 12: Match the ORB feature points in the depth vision image with the ORB feature points in the candidate image to obtain ORB feature point pairs, and input the ORB feature point pairs into the RANSAC algorithm for processing to remove the ORB feature point pairs with incorrect matching;

[0063] Step 13: Determine the number of ORB feature point pairs that have been processed, and calculate the matching degree using the number of point pairs and the number of ORB feature points in the deep vision image;

[0064] Step 14: Take the candidate image with the largest matching degree as the matching image, input the matching image and the depth vision image into the PNP algorithm for calculation to obtain the initial pose.

[0065] It is understandable that there may be mismatches in the ORB feature points during the matching process, and unmatched ORB feature point pairs will affect the matching accuracy, so the RANSAC algorithm can be used to remove mismatched ORB feature point pairs. Furthermore, after selecting the candidate image, since the position and posture of the candidate image may deviate from the current position and posture corresponding to the depth vision image, the posture corresponding to the candidate image cannot accurately establish a corresponding relationship with the current position. Therefore, the ORB feature points of the candidate image and the ORB of the depth vision image can be input into the PNP algorithm, and a more accurate initial posture is obtained by estimating the relative position of the feature points in space.

[0066] S103: When the matching degree is greater than or equal to a preset threshold, the initial posture and laser scanning data are used to determine the precise posture of the robot in the fused map.

[0067] When the matching degree is greater than or equal to the preset threshold, it means that the candidate image matched meets the application requirements, and the initial posture can be determined to be relatively accurate. At this time, the laser scanning data can be further used to determine the precise posture of the robot. In the embodiment of the present invention, considering that the laser scanning data can determine the robot posture more accurately, but the amount of laser scanning data is relatively large, if the laser scanning data is directly used to match the complete fusion map, the efficiency is low and it is difficult to meet the real-time matching requirements; after determining the initial posture of the robot, the laser scanning data can be used to perform local matching with the fusion map around the initial posture, which can significantly improve the matching efficiency. Therefore, the embodiment of the present invention first uses the depth vision image to determine the initial posture, and then combines the initial posture with the laser scanning data to determine the precise posture. It can not only ensure the accuracy of the precise posture, but also avoid matching the laser scanning data with the complete fusion map, thereby improving the matching efficiency of the laser scanning data.

[0068] It should be noted that the embodiment of the present invention does not limit the specific value of the preset threshold, which can be set according to actual application requirements.

[0069] Furthermore, the embodiments of the present invention do not limit the specific method of using the initial posture and laser scanning data to determine the precise posture of the robot in the fused map. For example, the initial posture can be used to determine the visual area, and the laser scanning data can be used to determine the laser area, and finally the precise posture can be determined according to the degree of fusion between the visual area and the laser area; of course, the initial posture can also be used to determine the initial area in the fused map, and the laser scanning data can be used to match in the initial area to obtain the precise posture. Considering that the initial posture is used to determine the initial area in the fused map, and the laser scanning data is used to match in the initial area to obtain the precise posture, the matching amount of laser scanning data can be greatly reduced, thereby improving the matching efficiency. Therefore, in the embodiments of the present invention, the precise posture of the robot can be determined by using the initial posture to determine the initial area in the fused map, and using the laser scanning data to match in the initial area.

[0070] S104. When the matching degree is less than a preset threshold, the robot is controlled to move to the next position until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and the precise posture is determined using the initial posture of the next position and the laser scanning data.

[0071] When the matching degree is less than the preset threshold, it means that the matched candidate image does not meet the application requirements. At this time, the initial posture obtained will have a large deviation. The above situation is often due to the fact that the depth vision image collected at the current position does not contain enough image feature information, which leads to image matching failure. In view of the above situation, in an embodiment of the present invention, the robot will be automatically controlled to move to obtain depth vision images with sufficient image feature information from more postures.

[0072] It should be noted that the embodiment of the present invention does not limit the movement mode of the robot, for example, it can be rotation or translation. It is understandable that the depth visual image collected by the robot after the movement may still be unable to match the appropriate candidate image. At this time, if the robot is blindly controlled to move, it will be difficult to improve the efficiency of determining the initial posture of the robot. Therefore, in the embodiment of the present invention, each time the robot is controlled to move in a certain movement mode, a certain number of candidate positions will be determined, and the matching degree of each candidate position corresponding to the candidate image will be determined. Finally, the optimal candidate position is selected as the starting position of the next movement mode according to the size of the matching degree. In the above manner, the embodiment of the present invention can control the robot to move in a targeted manner, thereby improving the matching efficiency of the initial posture.

[0073] In one possible scenario, controlling the robot to move to a next position includes:

[0074] Step 21: Control the robot to move according to a preset mode, and determine a preset number of candidate positions during the movement; the preset mode includes rotation and translation;

[0075] Step 22: setting the initial position of the candidate position as the first candidate position, and setting the candidate position corresponding to the first candidate position with the largest matching degree as the next position;

[0076] Step 23: Control the robot to move to the next position according to the preset mode.

[0077] It should be noted that the embodiments of the present invention do not limit the method of determining candidate positions for a specific preset mode. For example, when the preset mode is rotation, the robot can be controlled to rotate in situ at a preset angle, and within the preset angle range, a candidate position is selected every time an interval angle is reached. The above preset angles and interval angles can be set according to actual application requirements; when the preset model is translation, the robot can be controlled to move a preset distance in a straight line, and a candidate position is selected every time an interval distance is reached. The above preset distances and interval distances can also be set according to actual application requirements. It can be understood that as long as the preset angle, interval angle, or preset distance, interval distance is set, the obtained candidate positions are a preset number. The present invention also does not limit the specific value of the preset number, which can be set according to actual application requirements.

[0078] Finally, it should be noted that in the process of controlling the robot's movement, if an obstacle occurs, the robot's obstacle avoidance device can be called to perform obstacle avoidance operations.

[0079] Based on the above embodiments, the method first uses the deep visual image collected by the robot at the current position to match it in the fusion map to determine the initial posture. Since the fusion map carries the feature information of the visual map and the visual word bag, the method can determine the initial posture of the robot by matching the deep visual image with the fusion map; the method further uses the laser scanning data and the initial posture to determine the precise posture of the robot. Since the fusion map also carries the feature information of the laser grid map, and the matching accuracy of the laser scanning data is higher than that of the deep visual image, the method can use the laser scanning data and the initial posture to improve the robot repositioning accuracy; finally, when the robot encounters a failure in the initial posture matching, that is, when the matching degree of the initial posture is less than the preset threshold, the method will also autonomously control the robot to move to obtain an initial posture with a higher matching degree, thereby ensuring the reliability and accuracy of the robot's autonomous repositioning at any position, without the need to restart at a fixed position or manually set it, which can effectively avoid the inconvenience caused by the fixed position restart or manual setting of the robot repositioning in the related technology, thereby improving the user experience.

[0080] Based on the above embodiment, the specific method of using the initial posture and laser scanning data to determine the precise posture of the robot in the fused map is introduced below. In one possible case, using the initial posture and laser scanning data to determine the precise posture of the robot in the fused map may include:

[0081] S201, generating an initial area in the fusion map according to the position of the initial posture.

[0082] It should be noted that the embodiments of the present invention do not limit the specific shape and size of the initial area, which can be set according to actual application requirements. The embodiments of the present invention also do not limit the specific method of determining the initial area, for example, the position corresponding to the initial posture is used as the center of the initial area for generation; if the initial area is a rectangle, the position corresponding to the initial posture can also be used as any corner of the rectangle.

[0083] S202: Use the laser scanning data to perform matching in the initial area to determine the precise position and posture.

[0084] The embodiment of the present invention does not limit the specific method of matching in the initial area using laser scanning data. For example, all fused maps in the initial area using laser scanning data can be directly used for complete matching, or a low-precision grid map can be generated using the fused map of the initial area, and then the laser scanning data and the low-precision grid map are used for preliminary matching. Finally, according to the results of the preliminary matching, a further fine matching is performed in the fused map to obtain an accurate posture. Considering that the amount of matching faced by laser scanning data is large, the embodiment of the present invention can use a low-precision grid map to reduce the amount of matching of laser scanning data, thereby improving matching efficiency.

[0085] In one possible scenario, matching in the initial area using laser scanning data to determine the precise pose may include:

[0086] Step 31: Downsample the initial area to obtain a low-precision grid map;

[0087] Step 32: Use the laser scanning data to match in the low-precision grid map to obtain a second candidate pose;

[0088] Step 33: In the fused map, the laser scanning data is matched with the second candidate pose to obtain an accurate pose.

[0089] It should be noted that the embodiment of the present invention does not limit the specific method of downsampling, and reference may be made to related technologies of raster image downsampling.

[0090] Finally, since there may be a situation where the laser scanning does not meet the requirements of the matching degree with the initial area, resulting in a matching failure, after the matching fails, the initial area can be adjusted, such as enlarged or displacement adjusted, and the adjusted initial area is used to continue matching. It should be noted that the embodiment of the present invention does not limit the size of the enlargement and the distance of the displacement adjustment, and can be set according to actual application requirements.

[0091] In one possible case, after matching in the initial area using the laser scanning data, the following steps may also be included:

[0092] Step 41: When the matching fails, the initial area is adjusted, and the adjusted initial area is used to perform the step of matching in the initial area using the laser scanning data.

[0093] Based on the above embodiments, this method can use the initial posture to determine the initial area in the fused map, and use the laser scanning data to match in the initial area to obtain an accurate posture, thereby greatly reducing the matching amount of laser scanning data, and ultimately achieving the effect of improving matching efficiency.

[0094] Based on the above embodiment, the generation process of the fused map is introduced below. In one possible case, the generation process of the fused map may include:

[0095] S301, obtaining a depth image, a visible light image and laser scanning data to be processed at a current acquisition position.

[0096] It is understandable that during the collection process, when encountering obstacles, the robot's obstacle avoidance device can be called to perform obstacle avoidance operations.

[0097] S302: Extract ORB feature points of the visible light image, and use the ORB feature points of the visible light image to update the visual word bag.

[0098] S303: Input the visible light image and the historical visible light image of the last acquisition position into the g2o graph optimization model to obtain the visual posture of the robot.

[0099] It should be noted that the embodiments of the present invention do not limit the specific content of the g2o graph optimization model, and reference may be made to related technologies.

[0100] S304: Update the laser grid map using the laser scanning data to be processed, and perform laser positioning in the updated laser grid map using the laser scanning data to be processed to obtain the laser position and posture of the robot.

[0101] It should be noted that the embodiments of the present invention do not limit the specific method of laser positioning, and reference may be made to related technologies. It is understandable that the laser pose and the visual pose need to be adjusted to the same coordinate system to achieve alignment of the laser pose and the visual pose.

[0102] S305 , input the visual posture and laser posture into the EKF model to obtain the current posture of the robot.

[0103] It should be noted that the embodiment of the present invention does not limit the specific content of the EKF model (Extended Kalman Filter), and reference may be made to related technologies.

[0104] S306: Establish a binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image, and the depth image, and use the binding relationship to update the visual map.

[0105] After establishing the binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image and the depth image, the depth information of the depth image can also be fused into the ORB feature points so that the ORB feature points have depth values, thereby better matching with the depth vision image.

[0106] S307: Update the fused map using the updated visual word bag, laser grid map and visual map.

[0107] Based on the above embodiments, the fusion map generated by the present method is jointly generated by the visual map, the visual word bag and the laser grid map, wherein the visual map carries the texture information of the visible light image and is composed of the visible light images collected at each position and posture orientation. The visual map can be used to quickly and effectively determine the position and posture of the robot when the current position is well lit; the visual word bag is trained based on the visual map, which contains the feature information of all visible light images in the visual map, and can classify the visible light images according to the feature information, which can further improve the matching efficiency of the visual map; the laser grid map is generated by laser scanning data, contains high-precision laser scanning information, and can accurately match the position and posture of the robot under any lighting conditions. Since the present method uses the visual map, the visual word bag and the laser grid map to generate the fusion map, a large amount of feature information can be retained for matching, so the reliability and accuracy of the robot relocation can be effectively improved.

[0108] A robot autonomous repositioning device, electronic device and storage medium provided in an embodiment of the present invention are introduced below. The robot autonomous repositioning device, electronic device and storage medium described below and the robot autonomous repositioning method described above can be referenced to each other.

[0109] Please refer to Figure 2 , Figure 2 A structural block diagram of a robot autonomous repositioning device provided by an embodiment of the present invention, the device may include:

[0110] The acquisition module 201 is used to acquire the fused map and the depth visual image and laser scanning data of the current position; wherein the fused map is generated by the visual map, the visual word bag and the laser grid map;

[0111] A first matching module 202 is used to match the robot in the fused map using the deep vision image to determine the initial posture of the robot with the highest matching degree in the fused map;

[0112] The second matching module 203 is used to determine the precise position and posture of the robot in the fused map using the initial position and posture and the laser scanning data when the matching degree is greater than or equal to a preset threshold;

[0113] The motion control module 204 is used to control the robot to move to the next position when the matching degree is less than a preset threshold, until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and then use the initial posture of the next position and the laser scanning data to determine the precise posture.

[0114] Optionally, the motion control module 204 may include:

[0115] A first motion control submodule is used to control the robot to move according to a preset mode and determine a preset number of candidate positions during the movement; the preset mode includes rotation and translation;

[0116] A setting submodule is used to set the initial posture of the candidate position to the first candidate posture, and set the candidate position corresponding to the first candidate posture with the largest matching degree to the next position;

[0117] The second motion control submodule is used to control the robot to move to the next position according to a preset mode.

[0118] Optionally, the second matching module 203 may include:

[0119] The initial region determination submodule is used to generate the initial region in the fusion map according to the position of the initial pose;

[0120] The precise pose determination submodule is used to match the laser scanning data in the initial area to determine the precise pose.

[0121] Optionally, the precise posture determination submodule may include:

[0122] A downsampling unit, used to downsample the initial area to obtain a low-precision grid map;

[0123] A first matching unit is used to match the laser scanning data in the low-precision grid map to obtain a second candidate pose;

[0124] The second matching unit is used to match the laser scanning data with the second candidate pose in the fusion map to obtain an accurate pose.

[0125] Optionally, the second matching module 203 may further include:

[0126] The adjustment submodule is used to adjust the initial area when the matching fails, and use the adjusted initial area to perform the step of matching in the initial area using the laser scanning data.

[0127] Optionally, the first matching module 202 may include:

[0128] The first matching submodule is used to extract ORB feature points in the deep vision image, and input the ORB feature points into the visual word bag of the fusion map for matching, so as to obtain the candidate image corresponding to the deep vision image in the fusion map;

[0129] The second matching submodule is used to match the ORB feature points in the depth vision image with the ORB feature points in the candidate image to obtain ORB feature point pairs, and input the ORB feature point pairs into the RANSAC algorithm for processing to remove the ORB feature point pairs with incorrect matching;

[0130] A matching degree calculation submodule is used to determine the number of point pairs of ORB feature points that have been processed, and calculate the matching degree using the number of point pairs and the number of ORB feature points in the depth vision image;

[0131] The initial pose determination submodule is used to take the candidate image with the largest matching degree as the matching image, input the matching image and the deep vision image into the PNP algorithm for calculation, and obtain the initial pose.

[0132] Optionally, the device may further include:

[0133] The second acquisition module is used to acquire the depth image, visible light image and laser scanning data to be processed at the current acquisition position;

[0134] A visual word bag update module is used to extract ORB feature points of visible light images and update the visual word bag using the ORB feature points of visible light images;

[0135] The visual pose determination module is used to input the visible light image and the historical visible light image of the last acquisition position into the G2O graph optimization model to obtain the visual pose of the robot;

[0136] A laser posture determination module is used to update the laser grid map using the laser scanning data to be processed, and to perform laser positioning in the updated laser grid map using the laser scanning data to be processed to obtain the laser posture of the robot;

[0137] The current posture determination module is used to input the visual posture and laser posture into the EKF model to obtain the current posture of the robot;

[0138] A binding module is used to establish a binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image, and the depth image, and to update the visual map using the binding relationship;

[0139] The updating module is used to update the fused map using the updated visual word bag, laser grid map and visual map.

[0140] An embodiment of the present invention further provides a robot, comprising:

[0141] Memory for storing computer programs;

[0142] A processor is used to implement the steps of the robot autonomous repositioning method as described above when executing a computer program.

[0143] It is understandable that the robot may also include sensors such as a wheel speed meter, a laser radar, a depth camera, an IMU inertial sensor, etc. The embodiment of the present invention does not limit the specific sensors, nor does it limit the specific configuration of the above sensors, which can be configured according to actual application requirements.

[0144] Since the embodiments of the electronic equipment part correspond to the embodiments of the robot autonomous repositioning method part, please refer to the description of the embodiments of the robot autonomous repositioning method part for the embodiments of the electronic equipment part, which will not be repeated here.

[0145] An embodiment of the present invention further provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the robot autonomous repositioning method of any of the above embodiments are implemented.

[0146] Since the embodiments of the storage medium part correspond to the embodiments of the robot autonomous repositioning method part, please refer to the description of the embodiments of the robot autonomous repositioning method part for the embodiments of the storage medium part, which will not be repeated here.

[0147] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0148] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0149] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0150] The above is a detailed introduction to a robot autonomous repositioning method, device, robot and storage medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A robot autonomous repositioning method, characterized in that: include: Obtaining a fused map and a deep visual image and laser scanning data of the current location; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map; Matching the deep vision image in the fused map to determine the initial posture of the robot with the highest matching degree in the fused map; When the matching degree is greater than or equal to a preset threshold, determining the precise position and posture of the robot in the fused map using the initial position and posture and the laser scanning data; When the matching degree is less than the preset threshold, the robot is controlled to move to the next position until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and the precise posture is determined using the initial posture of the next position and the laser scanning data; The controlling the robot to move to a next position comprises: Control the robot to move according to a preset mode, and determine a preset number of candidate positions during the movement; the preset mode includes rotation and translation. When the preset mode is rotation, the robot is controlled to rotate in situ by a preset angle, and within the preset angle range, a candidate position is selected every time an interval angle is reached; or when the preset mode is translation, the robot is controlled to move a preset distance in a straight line, and a candidate position is selected every time an interval distance is reached; The initial position of the candidate position is set as the first candidate position, and the candidate position corresponding to the first candidate position with the largest matching degree is set as the next position; Control the robot to move to the next position according to the preset mode; The generation process of the fusion map includes: Obtain the depth image, visible light image and laser scanning data to be processed at the current acquisition position; Extracting ORB feature points of the visible light image, and updating the visual word bag using the ORB feature points of the visible light image; Inputting the visible light image and the historical visible light image of the last acquisition position into the g2o graph optimization model to obtain the visual pose of the robot; Using the laser scanning data to be processed to update the laser grid map, and using the laser scanning data to be processed to perform laser positioning in the updated laser grid map to obtain the laser position and posture of the robot; Inputting the visual pose and the laser pose into an EKF model to obtain the current pose of the robot; Establishing a binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image, and the depth image, fusing the depth information of the depth image to the ORB feature points, and updating the visual map using the binding relationship; The fused map is updated using the updated visual word bag, laser grid map and visual map.

2. The robot autonomous repositioning method according to claim 1, characterized in that: The determining the precise posture of the robot in the fused map by using the initial posture and the laser scanning data includes: generating an initial region in the fused map according to the position of the initial pose; The laser scanning data is used to perform matching in the initial area to determine the precise position and posture.

3. The robot autonomous repositioning method according to claim 2, characterized in that: The using the laser scanning data to match in the initial area to determine the precise position and posture includes: Downsampling the initial area to obtain a low-precision grid image; Using the laser scanning data to match in the low-precision grid map, a second candidate pose is obtained; In the fused map, the laser scanning data is matched with the second candidate pose to obtain the precise pose.

4. The robot autonomous repositioning method according to claim 2, characterized in that: After matching in the initial area using the laser scanning data, the method further includes: When the matching fails, the initial area is adjusted, and the adjusted initial area is used to perform the step of matching in the initial area using the laser scanning data.

5. The robot autonomous repositioning method according to claim 1, characterized in that: The using the deep vision image to match in the fused map to determine the robot's initial posture with the highest matching degree in the fused map includes: Extracting ORB feature points in the deep vision image, and inputting the ORB feature points into the visual word bag of the fused map for matching, to obtain a candidate image corresponding to the deep vision image in the fused map; Matching the ORB feature points in the depth vision image with the ORB feature points in the candidate image to obtain ORB feature point pairs, and inputting the ORB feature point pairs into the RANSAC algorithm for processing to remove mismatched ORB feature point pairs; Determine the number of point pairs of the processed ORB feature point pairs, and calculate the matching degree using the number of point pairs and the number of ORB feature points in the depth vision image; The candidate image with the greatest matching degree is used as a matching image, and the matching image and the depth vision image are input into a PNP algorithm for calculation to obtain the initial pose.

6. A robot autonomous repositioning device, characterized in that: include: An acquisition module, used to acquire a fused map and a deep visual image and laser scanning data of a current location; wherein the fused map is generated by a visual map, a visual word bag and a laser grid map; A first matching module, used to use the deep vision image to match the fused map, and determine the initial posture of the robot with the highest matching degree in the fused map; A second matching module, configured to determine the precise position and posture of the robot in the fused map using the initial position and posture and the laser scanning data when the matching degree is greater than or equal to a preset threshold; A motion control module, configured to control the robot to move to a next position when the matching degree is less than the preset threshold, until the matching degree corresponding to the initial posture matched to the next position is greater than or equal to the preset threshold, and determine the precise posture using the initial posture of the next position and the laser scanning data; The motion control module comprises: A first motion control submodule is used to control the robot to move according to a preset mode and determine a preset number of candidate positions during the movement; the preset mode includes rotation and translation. When the preset mode is rotation, the robot is controlled to rotate in situ by a preset angle and select a candidate position every time an interval angle is reached within the preset angle range; or when the preset mode is translation, the robot is controlled to move a preset distance in a straight line and select a candidate position every time an interval distance is reached; A setting submodule, used to set the initial posture of the candidate position as the first candidate posture, and set the candidate position corresponding to the first candidate posture with the largest matching degree as the next position; A second motion control submodule, used for controlling the robot to move to the next position according to the preset mode; The device also includes: The second acquisition module is used to acquire the depth image, visible light image and laser scanning data to be processed at the current acquisition position; A visual word bag updating module, used for extracting ORB feature points of the visible light image and updating the visual word bag using the ORB feature points of the visible light image; A visual pose determination module, used for inputting the visible light image and the historical visible light image of the last acquisition position into a g2o graph optimization model to obtain the visual pose of the robot; A laser posture determination module, used to update the laser grid map using the laser scanning data to be processed, and to perform laser positioning in the updated laser grid map using the laser scanning data to be processed to obtain the laser posture of the robot; A current posture determination module, used for inputting the visual posture and the laser posture into the EKF model to obtain the current posture of the robot; A binding module, used to establish a binding relationship between the current posture and the visible light image, the ORB feature points of the visible light image and the depth image, fuse the depth information of the depth image to the ORB feature points, and update the visual map using the binding relationship; An updating module is used to update the fused map using the updated visual word bag, laser grid map and visual map.

7. A robot, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the robot autonomous repositioning method as claimed in any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the robot autonomous repositioning method according to any one of claims 1 to 5.

Images