Wheeled robot vision localization and map construction method and system based on multi-sensor fusion and medium

Abstract

The embodiment of the invention provides a wheeled robot visual localization and map construction method and system based on multi-sensor fusion and a medium, and the method comprises the steps: obtaining scene multi-dimensional information based on a multi-dimensional sensor, and obtaining an initial scene; establishing a scene coordinate system based on the initial scene, and calculating current position information of the wheeled robot based on the scene coordinate system; an initial scene image is collected, and obstacle position distribution information in a scene is obtained; generating moving track information based on the obstacle position distribution information and the current position information of the wheeled robot; controlling the wheeled robot to advance along the moving track based on the moving track information, updating a scene image in real time, and constructing and updating a scene map based on the updated scene image; the initial scene is obtained through the multi-dimensional sensor, and the obstacle distribution is analyzed, so that the moving track of the wheeled robot is accurately established, the positioning precision of the wheeled robot is improved, and the scene map is continuously updated in the moving process of the wheeled robot.

Description

Technical Field

[0001] This application relates to the field of visual positioning technology. Specifically, it relates to a method, system, and medium for visual positioning and map construction of a wheeled robot based on multi-sensor fusion. Background Art

[0002] The role of artificial intelligence in the robotics industry is becoming increasingly important. It enhances the capabilities of robots by improving their self-learning ability and applying it to multiple scenarios. Technologies such as deep learning and computer vision in artificial intelligence enable robots to better understand and execute tasks, improving production efficiency. In the field of mobile robots, artificial intelligence not only improves production efficiency but also ensures functional safety and information security, giving rise to new applications such as autonomous driving vehicles, collaborative robots, and drones. In existing methods for wheeled robot positioning and map construction, the initial scene cannot be accurately obtained based on multi-dimensional sensors, and it is difficult to continuously update the scene map during the movement of the wheeled robot. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method, system, and medium for visual positioning and map construction of a wheeled robot based on multi-sensor fusion. By obtaining the initial scene through multi-dimensional sensors and analyzing the obstacle distribution, the movement trajectory of the wheeled robot can be accurately established, improving the positioning accuracy of the wheeled robot, and continuously updating the scene map during the movement of the wheeled robot.

[0004] The embodiments of this application also provide a method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion, including:

[0005] Obtaining multi-dimensional information of the scene based on multi-dimensional sensors, and fusing the multi-dimensional information of the scene to obtain the initial scene;

[0006] Establishing a scene coordinate system based on the initial scene, and calculating the current position information of the wheeled robot based on the scene coordinate system;

[0007] Collecting the initial scene image, and obtaining the position distribution information of the obstacles in the scene based on the initial scene image;

[0008] Generating movement trajectory information based on the position distribution information of the obstacles and the current position information of the wheeled robot;

[0009] Controlling the wheeled robot to travel along the movement trajectory based on the movement trajectory information, and continuously updating the scene image in real time. Constructing and updating the scene map based on the updated scene image.

[0010] Optionally, in the method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, multi-dimensional information of a scene is obtained based on multi-dimensional sensors, and the multi-dimensional information of the scene is fused to obtain an initial scene, specifically including:

[0011] Obtain the height information, width information, and length information of the scene based on multi-dimensional sensors;

[0012] Obtain the volume information of the scene based on the height information, width information, and length information of the scene;

[0013] Obtain the shape information of the projection plane of the scene, and obtain the multi-dimensional information of the scene based on the shape information of the projection plane of the scene and the volume information of the scene;

[0014] Fuse the multi-dimensional information of the scene to obtain the scene space information, and obtain the initial scene based on the scene space information.

[0015] Optionally, in the method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, a scene coordinate system is established based on the initial scene, and the current position information of the wheeled robot is calculated based on the scene coordinate system, specifically including:

[0016] Obtain the shape information of the projection plane of the initial scene based on the initial scene, and calculate the center point of the projection plane based on the shape information of the projection plane;

[0017] Use the center point as the origin, the length of the projection plane as the X-axis, and the width direction of the projection plane as the Y-axis to establish a scene coordinate system;

[0018] Obtain the parameter information of the wheeled robot, and obtain the shape of the projection plane of the wheeled robot based on the parameter information of the wheeled robot;

[0019] Analyze the center point of the projection plane of the wheeled robot based on the shape of the projection plane of the wheeled robot;

[0020] Analyze the coordinates of the center point of the projection plane of the wheeled robot based on the scene coordinate system, and obtain the current position information of the wheeled robot according to the coordinates of the center point of the projection plane of the wheeled robot.

[0021] Optionally, in the method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, an initial scene image is collected, and the position distribution information of obstacles in the scene is obtained based on the initial scene image, specifically including:

[0022] Obtain the initial scene image and extract image features;

[0023] Compare the image features with the set features to obtain a feature deviation rate;

[0024] Determine whether the feature deviation rate is greater than or equal to the set feature deviation rate threshold;

[0025] If it is greater than or equal to the set characteristic deviation rate threshold, it is determined as a background feature, and the background feature is blurred.

[0026] If it is less than the set characteristic deviation rate threshold, it is determined as an obstacle feature, and obstacle position distribution information is generated based on the obstacle feature.

[0027] Optionally, in the method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, generating mobile trajectory information based on the obstacle position distribution information and the current position information of the wheeled robot specifically includes:

[0028] Obtain the obstacle position distribution information, and set a corresponding detection area based on the obstacle position distribution information.

[0029] Crop a sub-region image corresponding to the detection area based on the initial scene image, extract the obstacle features in the sub-region, and analyze the obstacle edge information in all sub-regions.

[0030] Analyze the blank areas between different obstacles based on the obstacle edge information in all sub-regions to obtain a blank area distribution map.

[0031] Obtain the current position information and target position information of the wheeled robot, screen out the blank areas between the current position and the target position based on the blank area distribution map, and establish mobile trajectory information.

[0032] Optionally, in the method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, controlling the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and updating the scene image in real time, and constructing and updating the scene map based on the updated scene image specifically includes:

[0033] Obtain the real-time position information and attitude information of the wheeled robot.

[0034] Analyze whether the wheeled robot deviates from the set mobile trajectory based on the real-time position information.

[0035] If it deviates, calculate the Euclidean distance between the real-time position of the wheeled robot and the set mobile trajectory, and adjust the attitude information of the wheeled robot based on the Euclidean distance.

[0036] If it does not deviate, explore the unknown area based on the camera on the wheeled robot, and update the scene image and the scene map.

[0037] Second aspect, embodiments of the present application provide a vision positioning and map construction system for a wheeled robot based on multi-sensor fusion. The system includes: a memory and a processor. The memory includes a program of a method for vision positioning and map construction of a wheeled robot based on multi-sensor fusion. When the program of the method for vision positioning and map construction of a wheeled robot based on multi-sensor fusion is executed by the processor, the following steps are implemented:

[0038] Obtain multi-dimensional information of the scene based on multi-dimensional sensors, fuse the multi-dimensional information of the scene, and obtain an initial scene;

[0039] Establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system;

[0040] Collect an initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image;

[0041] Generate movement trajectory information based on the position distribution information of obstacles and the current position information of the wheeled robot;

[0042] Control the wheeled robot to move along the movement trajectory based on the movement trajectory information, and update the scene image in real time. Construct and update the scene map based on the updated scene image.

[0043] Optionally, in the vision positioning and map construction system for a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, obtaining multi-dimensional information of the scene based on multi-dimensional sensors, fusing the multi-dimensional information of the scene, and obtaining an initial scene specifically includes:

[0044] Obtain the height information, width information, and length information of the scene based on multi-dimensional sensors;

[0045] Obtain the volume information of the scene based on the height information, width information, and length information of the scene;

[0046] Obtain the shape information of the scene projection plane, and obtain the multi-dimensional information of the scene based on the shape information of the scene projection plane and the volume information of the scene;

[0047] Fuse the multi-dimensional information of the scene to obtain scene space information, and obtain an initial scene based on the scene space information.

[0048] Optionally, in the vision positioning and map construction system for a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, establishing a scene coordinate system based on the initial scene, and calculating the current position information of the wheeled robot based on the scene coordinate system specifically includes:

[0049] Obtain the shape information of the initial scene projection plane based on the initial scene, and calculate the center point of the projection plane based on the shape information of the projection plane;

[0050] Taking the center point as the origin, the length of the projection plane as the X-axis, and the width direction of the projection plane as the Y-axis to establish a scene coordinate system;

[0051] Obtain the parameter information of the wheeled robot, and based on the parameter information of the wheeled robot, obtain the shape of the projection plane of the wheeled robot;

[0052] Analyze the center point of the projection plane of the wheeled robot based on the shape of the projection plane of the wheeled robot;

[0053] Analyze the coordinates of the center point of the projection plane of the wheeled robot based on the scene coordinate system, and obtain the current position information of the wheeled robot according to the coordinates of the center point of the projection plane of the wheeled robot.

[0054] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes a program for the method of visual positioning and map construction of a wheeled robot based on multi-sensor fusion. When the program for the method of visual positioning and map construction of a wheeled robot based on multi-sensor fusion is executed by a processor, the steps of the method of visual positioning and map construction of a wheeled robot based on multi-sensor fusion as described in any one of the above are implemented.

[0055] As can be seen from the above, a method, system and medium for visual positioning and map construction of a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application obtain multi-dimensional information of a scene through multi-dimensional sensors, fuse the multi-dimensional information of the scene to obtain an initial scene; establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system; collect an initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image; generate mobile trajectory information based on the position distribution information of obstacles and the current position information of the wheeled robot; control the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and update the scene image in real time, and construct and update the scene map based on the updated scene image; obtain the initial scene through multi-dimensional sensors and analyze the obstacle distribution, so as to accurately establish the mobile trajectory of the wheeled robot, improve the positioning accuracy of the wheeled robot, and continuously update the scene map during the movement of the wheeled robot. Description of the Drawings

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0057] Figure 1Flowchart of the vision positioning and map construction method for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application;

[0058] Figure 2 Initial scene construction flowchart of the vision positioning and map construction method for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application;

[0059] Figure 3 Flowchart of the wheeled robot position analysis method of the vision positioning and map construction method for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application. Detailed implementation manners

[0060] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Usually, the components of the embodiments of the present application described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application to be protected, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0061] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0062] Please refer to Figure 1 , Figure 1 which is a flowchart of a vision positioning and map construction method for a wheeled robot based on multi-sensor fusion in some embodiments of the present application. The vision positioning and map construction method for a wheeled robot based on multi-sensor fusion is used in a terminal device. The vision positioning and map construction method for a wheeled robot based on multi-sensor fusion includes the following steps:

[0063] S101. Obtain multi-dimensional information of the scene based on multi-dimensional sensors, and fuse the multi-dimensional information of the scene to obtain an initial scene;

[0064] S102. Establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system;

[0065] S103. Collect an initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image;

[0066] S104. Generate movement trajectory information based on the obstacle position distribution information and the current position information of the wheeled robot;

[0067] S105. Control the wheeled robot to move along the movement trajectory based on the movement trajectory information, and update the scene image in real time. Build and update the scene map based on the updated scene image.

[0068] It should be noted that the front wheels of the wheeled robot are composed of two independently driven wheels, which are used to change the speed, direction and size, so that the wheeled robot can move along the planned path. The rear wheels are used as passive wheels to stabilize the wheeled robot. Multidimensional information is fused through multi-sensor fusion to obtain a scene image, and the obstacle distribution is analyzed, so that the wheeled robot can accurately avoid obstacles according to the obstacle position.

[0069] Please refer to Figure 2 , Figure 2 is the initial scene construction flowchart of a method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion in some embodiments of the present application. According to the embodiments of the present invention, multi-dimensional information of the scene is obtained based on multi-dimensional sensors, and the multi-dimensional information of the scene is fused to obtain an initial scene, specifically including:

[0070] S201. Obtain the height information, width information and length information of the scene based on multi-dimensional sensors;

[0071] S202. Obtain the volume information of the scene based on the height information, width information and length information of the scene;

[0072] S203. Obtain the shape information of the scene projection plane, and obtain the multi-dimensional information of the scene based on the shape information of the scene projection plane and the volume information of the scene;

[0073] S204. Fuse the multi-dimensional information of the scene to obtain the scene space information, and obtain the initial scene based on the scene space information.

[0074] It should be noted that the scene is projected and mapped into a two-dimensional plane, so as to analyze the shape and size of the scene, and then accurately analyze the scene space information.

[0075] Please refer to Figure 3 , Figure 3 is the flowchart of the wheeled robot position analysis method of a method for visual positioning and map construction of a wheeled robot based on multi-sensor fusion in some embodiments of the present application. According to the embodiments of the present invention, a scene coordinate system is established based on the initial scene, and the current position information of the wheeled robot is calculated based on the scene coordinate system, specifically including:

[0076] S301. Obtain the shape information of the initial scene projection plane based on the initial scene, and calculate the center point of the projection plane based on the shape information of the projection plane;

[0077] S302. Use the center point as the origin, the length of the projection plane as the X-axis, and the width direction of the projection plane as the Y-axis to establish a scene coordinate system;

[0078] S303. Obtain the parameter information of the wheeled robot, and obtain the shape of the projection plane of the wheeled robot based on the parameter information of the wheeled robot;

[0079] S304. Analyze the center point of the projection plane of the wheeled robot based on the shape of the projection plane of the wheeled robot;

[0080] S305. Analyze the coordinates of the center point of the projection plane of the wheeled robot based on the scene coordinate system, and obtain the current position information of the wheeled robot according to the coordinates of the center point of the projection plane of the wheeled robot.

[0081] It should be noted that by establishing a scene coordinate system to calculate the coordinates of the wheeled robot within the coordinate system, accurate positioning of the wheeled robot can be achieved.

[0082] According to the embodiments of the present invention, collect the initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image, specifically including:

[0083] Obtain the initial scene image and extract the image features;

[0084] Compare the image features with the set features to obtain the feature deviation rate;

[0085] Judge whether the feature deviation rate is greater than or equal to the set feature deviation rate threshold;

[0086] If it is greater than or equal to the set feature deviation rate threshold, it is determined as the background feature, and the background feature is blurred;

[0087] If it is less than the set feature deviation rate threshold, it is determined as the obstacle feature, and the obstacle position distribution information is generated based on the obstacle feature.

[0088] It should be noted that by extracting the image features and analyzing and processing the features, the background features and obstacle features can be effectively screened out, the obstacle features are strengthened, the reflection effect is improved, and the position distribution of the obstacles can be accurately analyzed.

[0089] According to the embodiments of the present invention, generate the movement trajectory information based on the obstacle position distribution information and the current position information of the wheeled robot, specifically including:

[0090] Obtain the obstacle position distribution information, and set the corresponding detection area based on the obstacle position distribution information;

[0091] Crop the sub-region image corresponding to the detection region from the initial scene image, extract the obstacle features within the sub-region, and analyze the obstacle edge information within all sub-regions;

[0092] Analyze the blank regions between different obstacles based on the obstacle edge information within all sub-regions to obtain a blank region distribution map;

[0093] Obtain the current position information and target position information of the wheeled robot, filter out the blank regions between the current position and the target position based on the blank region distribution map, and establish mobile trajectory information.

[0094] It should be noted that by analyzing the edges of obstacles, the blank regions between obstacles can be accurately obtained. As long as the wheeled robot moves within the blank regions, it will not encounter obstacles. Therefore, when constructing the mobile trajectory, only the best trajectory is selected within the blank regions to achieve accurate obstacle avoidance for the wheeled robot.

[0095] According to an embodiment of the present invention, control the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and update the scene image in real time. Based on the updated scene image, construct and update the scene map, specifically including:

[0096] Obtain the real-time position information and attitude information of the wheeled robot;

[0097] Analyze whether the wheeled robot deviates from the set mobile trajectory based on the real-time position information;

[0098] If it deviates, calculate the Euclidean distance between the real-time position of the wheeled robot and the set mobile trajectory, and adjust the attitude information of the wheeled robot based on the Euclidean distance;

[0099] If it does not deviate, explore the unknown region based on the camera on the wheeled robot, and update the scene image and the scene map.

[0100] It should be noted that during the movement of the wheeled robot, the attitude is continuously adjusted to ensure that the wheeled robot can move along the set mobile trajectory, improve the positioning accuracy of the wheeled robot, and continuously explore the unknown region during the movement, and analyze the obstacle distribution within the unknown region, so as to continuously update the scene and the scene map.

[0101] In a second aspect, an embodiment of the present application provides a wheeled robot vision positioning and map construction system based on multi-sensor fusion. The system includes: a memory and a processor. The memory includes a program for the wheeled robot vision positioning and map construction method based on multi-sensor fusion. When the program for the wheeled robot vision positioning and map construction method based on multi-sensor fusion is executed by the processor, the following steps are implemented:

[0102] Obtain multi-dimensional information of the scene based on multi-dimensional sensors, fuse the multi-dimensional information of the scene, and obtain the initial scene;

[0103] Establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system;

[0104] Collect the initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image;

[0105] Generate mobile trajectory information based on the position distribution information of obstacles and the current position information of the wheeled robot;

[0106] Control the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and update the scene image in real time. Build and update the scene map based on the updated scene image.

[0107] It should be noted that the front wheels of the wheeled robot are composed of two independently driven wheels, which are used to change the speed, direction and size, so that the wheeled robot can move along the planned path. The rear wheels are used as passive wheels to stabilize the wheeled robot. The multi-dimensional information is fused through the multi-sensor fusion method to obtain the scene image, and the obstacle distribution is analyzed, so that the wheeled robot can accurately avoid according to the obstacle position.

[0108] According to the embodiment of the present invention, obtaining multi-dimensional information of the scene based on multi-dimensional sensors, fusing the multi-dimensional information of the scene, and obtaining the initial scene specifically includes:

[0109] Obtain the scene height information, width information and length information based on multi-dimensional sensors;

[0110] Obtain the scene volume information based on the scene height information, width information and length information;

[0111] Obtain the shape information of the scene projection plane, and obtain the multi-dimensional information of the scene based on the shape information of the scene projection plane and the scene volume information;

[0112] Fuse the multi-dimensional information of the scene to obtain the scene space information, and obtain the initial scene based on the scene space information.

[0113] It should be noted that the scene is projected onto a two-dimensional plane to analyze the shape and size of the scene, and then accurately analyze the scene space information.

[0114] According to the embodiment of the present invention, establishing a scene coordinate system based on the initial scene, and calculating the current position information of the wheeled robot based on the scene coordinate system specifically includes:

[0115] Obtain the shape information of the initial scene projection plane based on the initial scene, and calculate the center point of the projection plane based on the shape information of the projection plane;

[0116] Taking the center point as the origin, the length of the projection plane as the X-axis, and the width direction of the projection plane as the Y-axis to establish a scene coordinate system;

[0117] Obtain the parameter information of the wheeled robot, and based on the parameter information of the wheeled robot, obtain the shape of the projection plane of the wheeled robot;

[0118] Analyze the center point of the projection plane of the wheeled robot based on the shape of the projection plane of the wheeled robot;

[0119] Analyze the coordinates of the center point of the projection plane of the wheeled robot based on the scene coordinate system, and obtain the current position information of the wheeled robot according to the coordinates of the center point of the projection plane of the wheeled robot.

[0120] It should be noted that by establishing a scene coordinate system to calculate the coordinates of the wheeled robot within the coordinate system, the wheeled robot can be accurately positioned.

[0121] According to the embodiment of the present invention, collect the initial scene image, and obtain the position distribution information of obstacles in the scene based on the initial scene image, specifically including:

[0122] Obtain the initial scene image and extract the image features;

[0123] Compare the image features with the set features to obtain the feature deviation rate;

[0124] Judge whether the feature deviation rate is greater than or equal to the set feature deviation rate threshold;

[0125] If it is greater than or equal to the set feature deviation rate threshold, it is determined as the background feature, and the background feature is blurred;

[0126] If it is less than the set feature deviation rate threshold, it is determined as the obstacle feature, and the obstacle position distribution information is generated based on the obstacle feature.

[0127] It should be noted that by extracting the image features and analyzing and processing the features, the background features and obstacle features can be effectively screened out, the obstacle features are strengthened, the reflection effect is improved, and the position distribution of the obstacles can be accurately analyzed.

[0128] According to the embodiment of the present invention, generate the movement trajectory information based on the obstacle position distribution information and the current position information of the wheeled robot, specifically including:

[0129] Obtain the obstacle position distribution information, and set the corresponding detection area based on the obstacle position distribution information;

[0130] Crop the sub-region image corresponding to the detection area based on the initial scene image, extract the obstacle features in the sub-region, and analyze the obstacle edge information in all sub-regions;

[0131] Analyze the blank areas between different obstacles based on the edge information of obstacles in all sub-regions to obtain a distribution map of blank areas;

[0132] Obtain the current position information and target position information of the wheeled robot, and based on the distribution map of blank areas, screen out the blank areas between the current position and the target position to establish mobile trajectory information.

[0133] It should be noted that by analyzing the edges of obstacles, the blank areas between obstacles can be accurately obtained. As long as the wheeled robot moves within the blank areas, it will not encounter obstacles. Therefore, when constructing the mobile trajectory, only the best trajectory is selected within the blank areas to achieve accurate obstacle avoidance for the wheeled robot.

[0134] According to an embodiment of the present invention, control the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and update the scene image in real time. Based on the updated scene image, construct and update the scene map, specifically including:

[0135] Obtain the real-time position information and attitude information of the wheeled robot;

[0136] Based on the real-time position information, analyze whether the wheeled robot deviates from the set mobile trajectory;

[0137] If it deviates, calculate the Euclidean distance between the real-time position of the wheeled robot and the set mobile trajectory, and adjust the attitude information of the wheeled robot based on the Euclidean distance;

[0138] If it does not deviate, explore the unknown area based on the camera on the wheeled robot, and update the scene image and the scene map.

[0139] It should be noted that during the movement of the wheeled robot, the attitude is continuously adjusted to ensure that the wheeled robot can move along the set mobile trajectory, improve the positioning accuracy of the wheeled robot, and continuously explore the unknown area during the movement, and analyze the obstacle distribution in the unknown area, so as to continuously update the scene and the scene map.

[0140] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for the visual positioning and map construction method of a wheeled robot based on multi-sensor fusion. When the program for the visual positioning and map construction method of a wheeled robot based on multi-sensor fusion is executed by a processor, the steps of the visual positioning and map construction method of a wheeled robot based on multi-sensor fusion as described in any one of the above are realized.

[0141] A method, system, and medium for visual positioning and map construction of a wheeled robot based on multi-sensor fusion, which obtain multi-dimensional information of a scene through multi-dimensional sensors, fuse the multi-dimensional information of the scene to obtain an initial scene, establish a scene coordinate system based on the initial scene, calculate the current position information of the wheeled robot based on the scene coordinate system, collect initial scene images, obtain the position distribution information of obstacles in the scene based on the initial scene images, generate mobile trajectory information based on the obstacle position distribution information and the current position information of the wheeled robot, control the wheeled robot to travel along the mobile trajectory based on the mobile trajectory information, and update the scene images in real time, and construct and update the scene map based on the updated scene images. Obtain the initial scene through multi-dimensional sensors and analyze the obstacle distribution, so as to accurately establish the mobile trajectory of the wheeled robot, improve the positioning accuracy of the wheeled robot, and continuously update the scene map during the movement of the wheeled robot.

[0142] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0143] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units. They can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, in each embodiment of the present invention, the various functional units can all be integrated in one processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0145] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes: various media that can store program codes, such as removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0146] Alternatively, if the above integrated units of the present invention are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present invention, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

Claims

1. A wheeled robot visual positioning and map construction method based on multi-sensor fusion, characterized in that: include: Acquire multi-dimensional scene information based on multi-dimensional sensors, fuse the multi-dimensional scene information to obtain the initial scene; Establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system; Collecting an initial scene image, and obtaining obstacle position distribution information in the scene based on the initial scene image; Generate movement trajectory information based on obstacle position distribution information and current position information of the wheeled robot; The wheeled robot is controlled to move along the moving trajectory based on the moving trajectory information, and the scene image is updated in real time. The scene map is constructed and updated based on the updated scene image.

2. The wheeled robot visual positioning and map construction method based on multi-sensor fusion according to claim 1 is characterized in that: Based on the multi-dimensional sensor, the multi-dimensional information of the scene is obtained, and the multi-dimensional information of the scene is integrated to obtain the initial scene, which includes: Acquire scene height information, width information and length information based on multi-dimensional sensors; Acquire scene volume information based on scene height information, width information and length information; Acquire scene projection surface shape information, and obtain scene multi-dimensional information based on the scene projection surface shape information and scene volume information; The multi-dimensional information of the scene is integrated to obtain the scene spatial information, and the initial scene is obtained based on the scene spatial information.

3. The wheeled robot visual positioning and map construction method based on multi-sensor fusion according to claim 2 is characterized in that: A scene coordinate system is established based on the initial scene, and the current position information of the wheeled robot is calculated based on the scene coordinate system, specifically including: Acquire the projection surface shape information of the initial scene based on the initial scene, and calculate the center point of the projection surface based on the projection surface shape information; The scene coordinate system is established by taking the center point as the origin, the length of the projection surface as the X-axis, and the width of the projection surface as the Y-axis; Acquire parameter information of the wheeled robot, and acquire the projection surface shape of the wheeled robot based on the parameter information of the wheeled robot; Analyze the center point of the wheeled robot's projection surface based on its projection surface shape; The coordinates of the center point of the wheeled robot's projection surface are analyzed based on the scene coordinate system, and the current position information of the wheeled robot is obtained according to the coordinates of the center point of the wheeled robot's projection surface.

4. The wheeled robot visual positioning and map construction method based on multi-sensor fusion according to claim 3 is characterized in that: Collecting the initial scene image, and obtaining the obstacle position distribution information in the scene based on the initial scene image, specifically including: Obtain an initial scene image and extract image features; Compare the image features with the set features to obtain the feature deviation rate; Determine whether the characteristic deviation rate is greater than or equal to a set characteristic deviation rate threshold; If it is greater than or equal to the set feature deviation rate threshold, it is determined to be a background feature and the background feature is blurred; If it is less than the set feature deviation rate threshold, it is determined to be an obstacle feature, and obstacle position distribution information is generated based on the obstacle feature.

5. The wheeled robot visual positioning and map construction method based on multi-sensor fusion according to claim 4 is characterized in that: The moving trajectory information is generated based on the obstacle position distribution information and the current position information of the wheeled robot, including: Obtaining obstacle location distribution information, and setting a corresponding detection area based on the obstacle location distribution information; Based on the initial scene image, the sub-region image corresponding to the detection area is cut out, the obstacle features in the sub-region are extracted, and the obstacle edge information in all sub-regions is analyzed; Based on the obstacle edge information in all sub-areas, the blank areas between different obstacles are analyzed to obtain a blank area distribution map; The current position information and target position information of the wheeled robot are obtained, the blank area between the current position and the target position is filtered out based on the blank area distribution map, and the movement trajectory information is established.

6. The wheeled robot visual positioning and map construction method based on multi-sensor fusion according to claim 5 is characterized in that: Based on the moving trajectory information, the wheeled robot is controlled to move along the moving trajectory, and the scene image is updated in real time. Based on the updated scene image, the scene map is constructed and updated, specifically including: Obtain real-time position and posture information of the wheeled robot; Analyze whether the wheeled robot deviates from the set moving trajectory based on real-time position information; If there is a deviation, the Euclidean distance between the real-time position of the wheeled robot and the set moving trajectory is calculated, and the posture information of the wheeled robot is adjusted based on the Euclidean distance; If there is no deviation, the unknown area is explored based on the camera on the wheeled robot, and the scene image and scene map are updated.

7. A wheeled robot visual positioning and map building system based on multi-sensor fusion, characterized in that: The system includes: a memory and a processor, wherein the memory includes a program of a wheeled robot visual positioning and map building method based on multi-sensor fusion, and when the program of the wheeled robot visual positioning and map building method based on multi-sensor fusion is executed by the processor, the following steps are implemented: Acquire multi-dimensional scene information based on multi-dimensional sensors, fuse the multi-dimensional scene information to obtain the initial scene; Establish a scene coordinate system based on the initial scene, and calculate the current position information of the wheeled robot based on the scene coordinate system; Collecting an initial scene image, and obtaining obstacle position distribution information in the scene based on the initial scene image; Generate movement trajectory information based on obstacle position distribution information and current position information of the wheeled robot; The wheeled robot is controlled to move along the moving trajectory based on the moving trajectory information, and the scene image is updated in real time. The scene map is constructed and updated based on the updated scene image.

8. The wheeled robot visual positioning and map building system based on multi-sensor fusion according to claim 7 is characterized in that: Based on the multi-dimensional sensor, the multi-dimensional information of the scene is obtained, and the multi-dimensional information of the scene is integrated to obtain the initial scene, which includes: Acquire scene height information, width information and length information based on multi-dimensional sensors; Acquire scene volume information based on scene height information, width information and length information; Acquire scene projection surface shape information, and obtain scene multi-dimensional information based on the scene projection surface shape information and scene volume information; The multi-dimensional information of the scene is integrated to obtain the scene spatial information, and the initial scene is obtained based on the scene spatial information.

9. The wheeled robot visual positioning and map building system based on multi-sensor fusion according to claim 8, characterized in that: A scene coordinate system is established based on the initial scene, and the current position information of the wheeled robot is calculated based on the scene coordinate system, specifically including: Acquire the projection surface shape information of the initial scene based on the initial scene, and calculate the center point of the projection surface based on the projection surface shape information; The scene coordinate system is established by taking the center point as the origin, the length of the projection surface as the X-axis, and the width of the projection surface as the Y-axis; Acquire parameter information of the wheeled robot, and acquire the projection surface shape of the wheeled robot based on the parameter information of the wheeled robot; Analyze the center point of the wheeled robot's projection surface based on its projection surface shape; The coordinates of the center point of the wheeled robot's projection surface are analyzed based on the scene coordinate system, and the current position information of the wheeled robot is obtained according to the coordinates of the center point of the wheeled robot's projection surface.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a wheeled robot visual positioning and map building method program based on multi-sensor fusion. When the wheeled robot visual positioning and map building method program based on multi-sensor fusion is executed by a processor, the steps of the wheeled robot visual positioning and map building method based on multi-sensor fusion as described in any one of claims 1 to 6 are implemented.

Images