Method and device for collecting data to build a map for a mobile robot

Abstract

The present invention discloses a method and apparatus for collecting data for constructing a map for a mobile robot, which is applied to a mobile robot, and includes the steps of obtaining a target key node, moving to the target key node, and collecting data from the mobile robot's sensors during movement, and the target key node is determined based on at least topological information, and the topological information includes a logical relative position relationship between topological points and a logical connection relationship between the topological points, and beacons are provided at key nodes in an actual scene, and the topological points have a one-to-one correspondence with beacons at key nodes in the actual scene, and the logical connection relationship has a one-to-one correspondence with guide markers connected between key nodes, and during movement, the guide markers are used to determine the direction of movement, and the beacons are used to determine the target key node to be reached, without using position and posture information of any key node and any beacon.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to a Chinese patent application having application number 202111652063.1, filed on December 30, 2021, which is incorporated herein by reference.

[0002] The present invention relates to the field of machine vision, and in particular to a method of collecting data for building a map for a mobile robot. [Background technology]

[0003] With the development of technology, maps for navigation and positioning of mobile robots are being constructed automatically, whether it is visual navigation, laser navigation, inertial navigation, or a combination of various navigation methods. One of the automatic map construction methods is that the mobile robot moves according to a planned route and collects data for constructing the mobile robot's map in real time while moving, and the map construction software constructs the map based on the collected data.

[0004] While the mobile robot moves along a planned path, it is necessary to input preset position and orientation information to form a feedback for the movement of the mobile robot and intervene appropriately, and currently, all of these preset position and orientation information rely on a large amount of manual measurement work, such as measuring the true position and orientation of landmarks in a selected scene area, and measuring information for position and orientation calibration of landmarks since the current position and orientation needs to be calibrated based on landmarks during the movement of the robot. On the one hand, such manual measurement takes a lot of measurement time, and on the other hand, manual measurement errors are brought into the constructed map, thus affecting the accuracy of the constructed map. Summary of the Invention [Means for solving the problem]

[0005] The present invention provides a method for collecting data for building a map of a mobile robot to reduce manual work.

[0006] According to a first aspect of the present invention, there is provided a data collection method for constructing a map of a mobile robot applicable to a mobile robot, comprising: a step of moving to a target key node; and a step of collecting data from a sensor of the mobile robot during the movement, wherein the target key node is determined based on topological information, the topological information including a logical relative positional relationship between topological points and a logical connection relationship between the topological points, a beacon is provided at the key node in an actual scene, the topological point has a one-to-one correspondence with the beacon at the key node in the actual scene, the logical connection relationship has a one-to-one correspondence with a guide marker connected between the key nodes, during the movement, the guide marker is used to determine the direction of movement, and the beacon is used to determine the target key node to be reached, and the position and orientation information of any key node and any beacon is not used.

[0007] Preferably, the topological information further includes a beacon corresponding to the topological point, and the step of obtaining the target key node includes the step of obtaining a beacon of the target key node from a map building software, and the target key node is determined by the map building software based on the planned route, the topological information, and the topological point corresponding to the current key node reached by the mobile robot.

[0008] Preferably, during movement, using a guide marker to determine the direction of movement, using a beacon to determine the target key node to be reached, and not using the position and orientation information of any key node and any beacon includes: the mobile robot performs marker detection, and when it detects a guide marker, it moves according to the detected guide marker; when it detects a beacon, the mobile robot matches the beacon with the beacon of the current target key node from the map construction software; if the matching is successful, it determines that the current target key node has been reached, and requests the map construction software for the next target key node, so that the map construction software sends the beacon of the next target key node to the mobile robot; or the mobile robot sends the beacon to the map construction software, so that the map construction software matches the current beacon with the beacon corresponding to each topological point in the topological information, and if the matching is successful, it obtains the topological point corresponding to the current beacon, and sends the beacon corresponding to the next topological point to the mobile robot as the beacon of the next target key node of the mobile robot, and different beacons have different marking contents.

[0009] Preferably, the method further includes a step in which the mobile robot acquires topological information and a planned route from the map construction software, and the step of acquiring a target key node includes a step in which the mobile robot determines a target topological point based on the planned route, the topological information, and a topological point corresponding to a current key node reached by the mobile robot, and sets the key node corresponding to the target topological point as the target key node.

[0010] Preferably, during movement, using a guide marker to determine the direction of movement, and using a beacon to determine the target key node to be reached, without using the position and posture information of any key node and any beacon, includes: the mobile robot performs marker detection, and when a guide marker is detected, moves according to the detected guide marker, and when a beacon is detected, matches the beacon with a beacon corresponding to each topological point in the topological information, and when the matching is successful, determines that the current target key node has been reached and obtains the next target key node, and different beacons have different marking contents.

[0011] Preferably, the topological information further includes a beacon corresponding to the topological point, and during movement, determining a direction of movement using a guide marker and determining a target key node to be reached using a beacon, without using position and orientation information of any key node and any beacon, includes the mobile robot performing marker detection, and if it detects a guide marker, moving according to the detected guide marker, and if it detects a beacon, determining whether the currently detected beacon and the detected previous adjacent beacon are the same beacon based on a first distance between the currently detected beacon and the detected previous adjacent beacon recorded by an inertial odometer, and determining that the currently detected beacon and the detected previous adjacent beacon are not the same beacon if the first distance is equal to a set distance threshold, and determining a topological point corresponding to the currently detected beacon based on matching the order of the currently detected beacon and the detected previous adjacent beacon with the order of adjacent topological points in the planned route, and obtaining the next target key node, wherein different beacons have the same marking content.

[0012] Preferably, the topological information further includes a second distance between two topological points having a logical connection relationship, the second distance being a measured distance between two key nodes respectively corresponding to the two topological points, and the distance threshold being the second distance; and determining whether the currently detected beacon and the detected previous adjacent beacon are the same beacon based on the first distance between the currently detected beacon and the detected previous adjacent beacon recorded by an inertial odometer further includes determining that there is a false detection in the detection for the currently detected beacon if the first distance is not equal to the set distance threshold.

[0013] Preferably, a planned path set formed by all planned paths in the planned path is a subset of a movement path set along which the mobile robot moves during movement, the mobile robot moves at least once along each planned path in the planned path set, the step of collecting data from a sensor of the mobile robot during movement includes a step of collecting data from the sensor according to a first sampling frequency set during movement, and the step of the mobile robot detecting a marker includes acquiring and detecting a beacon according to a second sampling frequency set while the mobile robot remains in a moving state so that the current moving state is not interrupted, and the first sampling frequency and the second sampling frequency are related to the current moving speed of the mobile robot.

[0014] According to a second aspect of the present invention, there is provided a map building method comprising the step of building a map using data for building a map of a mobile robot, the data for building the map of the mobile robot being collected according to any of the methods for collecting data for building a map of a mobile robot.

[0015] Preferably, the step of constructing a map using data for constructing a map of the mobile robot includes the steps of synchronizing the collected data from each sensor, processing the collected image data to obtain key frames, performing global optimization based on the key frames to obtain global map information, and converting the map information into a map format.

[0016] According to a third aspect of the present invention, there is provided a data collection device for constructing a map of a mobile robot, comprising: a target key node acquisition module for acquiring a target key node; a movement control module for controlling a movement process to the target key node; and a collection module for collecting data from a sensor of the mobile robot during movement, wherein the target key node is determined based on at least topological information, the topological information including a logical relative position relationship between topological points and a logical connection relationship between the topological points; beacons are provided at key nodes in an actual scene, the topological points have a one-to-one correspondence with beacons at key nodes in the actual scene, and the logical connection relationship has a one-to-one correspondence with guide markers connected between key nodes; during movement, the guide markers are used to determine the direction of movement, and the beacons are used to determine the target key node to be reached. The data collection device for constructing a map of a mobile robot is provided, and does not use position and posture information of any key node and any beacon.

[0017] Preferably, the topological information further includes a beacon corresponding to a topological point, and the target key node acquisition module is configured to acquire the beacon of the target key node from a map construction software, and the target key node is determined by the map construction software based on the planned route, the topological information, and the topological point corresponding to the reached current key node.

[0018] Preferably, the movement control module includes a marker detection submodule configured to perform marker detection; and a movement control submodule configured to move according to the detected guide marker when a guide marker is detected, and to match the beacon with a beacon of a current target key node from the map construction software when a beacon is detected, and determine that the current target key node has been reached if the matching is successful, and request the map construction software for the next target key node so that the map construction software sends the beacon of the next target key node to the target key node acquisition module, or to match the current beacon with a beacon corresponding to each topological point in the topological information, and if the matching is successful, obtain a topological point corresponding to the current beacon and send the beacon corresponding to the next topological point to the target key node acquisition module as the beacon of the next target key node, where different beacons have different marking contents.

[0019] Preferably, the target key node acquisition module is configured to acquire topological information and a planned route from a map construction software, determine a target topological point according to the planned route, the topological information, and a topological point corresponding to a current key node reached, and set the key node corresponding to the target topological point as a target key node.

[0020] Preferably, the topological information further includes beacons corresponding to topological points, and the movement control module includes a marker detection submodule configured to perform marker detection, and a movement control submodule configured to move according to the detected guide marker when a guide marker is detected, and to match the beacon with a beacon corresponding to each topological point in the topological information when a matching is successful, and to determine that the current target key node has been reached and obtain the next target key node, wherein different beacons have different marking contents.

[0021] Preferably, the target key node acquisition module is configured to acquire topological information and a planned route from a map construction software, determine a target topological point according to the planned route, the topological information, and a topological point corresponding to a current key node reached, and set the key node corresponding to the target topological point as a target key node.

[0022] Preferably, the movement control module includes a marker detection submodule configured to perform marker detection; and a movement control submodule configured to, when a guide marker is detected, move according to the detected guide marker; when a beacon is detected, determine whether the currently detected beacon and the detected previous adjacent beacon are the same beacon based on a first distance between the currently detected beacon and the detected previous adjacent beacon recorded by an inertial odometer, and determine that the currently detected beacon and the detected previous adjacent beacon are not the same beacon if the first distance is equal to a set distance threshold; and determine a topological point corresponding to the currently detected beacon based on matching the order of the currently detected beacon and the detected previous adjacent beacon with the order of adjacent topological points in the planned route, and obtain a next target key node, wherein different beacons have the same marking content.

[0023] Preferably, the topological information further includes a second distance between two topological points having a logical connection relationship, the second distance being a measured distance between two key nodes respectively corresponding to the two topological points, and the distance threshold is the second distance, and the marker detection submodule is further configured to determine that there is a false detection in the detection for the currently detected beacon if the first distance is not equal to the set distance threshold.

[0024] Preferably, a planned path set formed by all planned paths in the planned path is a subset of a moving path set along which the mobile robot moves during movement, the mobile robot moves at least once along each planned path in the planned path set, the collection module is configured to collect data from the sensor of the mobile robot according to a first sampling frequency set during movement, and the marker detection submodule is configured to acquire and detect beacons according to a second sampling frequency set while remaining in a moving state so as not to interrupt the current moving state, and the first sampling frequency and the second sampling frequency are related to the current moving speed.

[0025] According to a fourth aspect of the present invention, there is provided a map construction system including a data collection device for constructing a map of a mobile robot as described in the third aspect above, and a map construction device for constructing a map using the data for constructing a map of the mobile robot.

[0026] Preferably, the map construction device includes a synchronization module for synchronizing collected data from the sensors of the mobile robot, a tracking module for processing the collected image data and obtaining key frames, an optimization module for performing global optimization based on the key frames to obtain global map information, and a map generation module for converting the map information into a map format.

[0027] According to a fifth aspect of the present invention, there is provided a mobile robot including an apparatus for collecting data for constructing a map of the mobile robot and / or the map construction system.

[0028] According to a sixth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to perform steps of any of the methods for collecting data for constructing a map of a mobile robot and / or steps of any of the methods for constructing a map.

[0029] The data collection method for constructing a map of a mobile robot provided by the present invention does not require obtaining position and orientation information of key nodes and beacons, uses guide markers to determine the direction of movement, uses beacons to determine the target key node to be reached, and collects data for constructing a map of the mobile robot while moving to the target key node without using position and orientation information of any key node and any beacon during movement. Since there is no need to actually measure to obtain position and orientation information of key nodes and beacons during movement, non-measurement map construction is supported, the convenience and degree of automation of data collection for constructing a map of a mobile robot while the mobile robot is moving can be improved, and map construction efficiency can be improved. Since there is no need to use position and orientation information of any key node and any beacon during movement, beacons can be flexibly installed, have good adaptability to actual scenes, the distance between beacons is not limited and can be theoretically infinite, the number of beacons required in the initial stage can be reduced, the efficiency of advance preparation can be improved, and complex routes for map construction can be supported when the implementation amount is small in the initial stage. The movement process is independent of navigation coordinates for map construction, does not require attention to positioning results, and the travel route is flexible. [Brief description of the drawings]

[0030] [Figure 1] FIG. 2 is a schematic diagram showing the flow of a method for collecting data to build a map of a mobile robot according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating a scene composition and a topological diagram according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing a flow of data collection for constructing a map of a mobile robot according to an embodiment of the present invention. [Figure 4] FIG. 11 is a schematic diagram showing a flow of data collection for constructing a map of a mobile robot according to another embodiment of the present invention. [Diagram 5]FIG. 2 is a schematic diagram showing a flow of map construction by map construction software according to the first embodiment of the present invention; [Figure 6] FIG. 11 is a schematic diagram showing a scene composition and a topological diagram according to a second embodiment of the present invention. [Figure 7] FIG. 11 is a schematic diagram showing a flow of moving to a target key node in a circular route manner according to embodiment 2 of the present invention; [Figure 8] FIG. 11 is a schematic diagram showing a flow of map construction by map construction software according to a second embodiment of the present invention. [Figure 9] FIG. 2 is a schematic diagram showing a data collection device for constructing a map of a mobile robot according to an embodiment of the present invention. [Figure 10] FIG. 1 is a schematic diagram illustrating a map construction system according to an embodiment of the present invention. [Figure 11] 1 is a schematic diagram illustrating a collection device, a map building system, or a mobile robot according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] In order to make the objectives, technical means and advantages of the present invention clearer, the present invention will be described in more detail below with reference to the drawings.

[0032] The data collection method for constructing a map of a mobile robot provided by the present invention collects data using the mobile robot's sensors while the mobile robot is moving to a target key node, and while moving, determines the direction of movement using guide markers and determines the target key node to be reached using beacons, without using position and orientation information of any key node and any beacon.

[0033] As shown in Fig. 1, Fig. 1 is a schematic diagram showing a flow of a method for collecting data for constructing a map for a mobile robot according to an embodiment of the present invention. The method is applied to the mobile robot side and includes the following steps 101 to 103.

[0034] In step 101, the target key node is obtained.

[0035] The target key node is determined based on the planned route and topological information, the topological information including logical relative positional relationships between topological points and logical connection relationships between the topological points, beacons are provided at the key nodes in the actual scene, the topological points have a one-to-one correspondence with the beacons at the key nodes in the actual scene, and the logical connection relationships have a one-to-one correspondence with the guide markers connected between the key nodes.

[0036] In step 102, the direction of movement is determined using the guide marker, the target key node to be reached is determined using the beacons, and movement to the target key node is performed without using position and orientation information of any of the key nodes and any of the beacons.

[0037] In step 103, data is collected from the sensors of the mobile robot during movement. During collection, the mobile robot collects data from the sensors of the mobile robot according to a set first sampling frequency, the first sampling frequency being related to a current moving speed of the mobile robot.

[0038] The present invention can improve the degree of automation of data collection for building a map for a mobile robot and support non-measurement map building.

[0039] In order to facilitate understanding of the present invention, the following description will be given in combination of sensor data collected by a specific collection method and map construction using the collected sensor data.

[0040] Example 1 In the first embodiment, marker information with different marking contents is used as a beacon, and a guide marker is used as a patrol route to construct a downward-looking visual texture map. The downward-looking visual texture map is a map that mainly consists of visual feature information of the ground and its relative positional relationship, and is mainly used for downward visual navigation of mobile robots. The main feature information of the map consists of the texture of the ground itself, so it is called a texture map.

[0041] First, the initial preparations are carried out.

[0042] In this embodiment, there is an existing route in the actual scene, and a topological diagram is created based on the existing route using map construction software, and a planned route is generated based on the topological diagram, which includes the logical relative position relationship between the topological points and the logical connection relationship between the topological points.

[0043] The topological points correspond one-to-one to the key nodes in the actual scene, and beacons with different marking contents, such as QR code (registered trademark, the same below) beacons, or one of letters, numbers, and icons, or any combination thereof, are provided on the ground at the positions of the key nodes, and the beacons at each key node are different, and the beacons corresponding to the topological points, the logical relative positional relationship between the topological points, and the logical connection relationship are stored in the topological diagram as attributes of the topological diagram. Based on the logical connection relationship between the topological points in the topological diagram, ribbons or elastic lines are pasted as guide markers between the key nodes to correspond to the logical connection relationship between the topological points in the topological diagram. As an exception, if the distance between the key nodes is less than a set distance threshold, it is not necessary to provide a guide marker.

[0044] Here, the key node does not include the overall position and orientation information of the key node, nor does it include the overall position and orientation information of the beacon, i.e., there is no need to manually measure the position and orientation information in the key node and the position and orientation information of the beacon.

[0045] Referring to FIG. 2, FIG. 2 is a schematic diagram showing a scene configuration and a topological diagram. In FIG. 2, the left diagram is an existing route in an actual scene, the middle diagram is a QR code and a guide marker provided, and the right diagram is a drawn topological diagram. Here, the topological diagram visualizes the logical relative positional relationship between topological points and the logical connection relationship between topological points, so it is not a full-scale image corresponding to the position and length of the existing route in the actual scene and the key node position, but a logical correspondence relationship. For example, in an actual scene, there is an existing route between key node A and key node B, and when a guide marker is provided between the above two key nodes, it does not need to be a straight line strictly, but may be a curve with curvature or a straight line that is easy for a mobile robot to travel. Also, for example, in an actual scene, key nodes A, E, and F are on the same horizontal line, but in the topological diagram, they do not need to be on the same horizontal line. From the viewpoint of computer program processing, the topological diagram is topological information.

[0046] The mobile robot is then used to collect data for constructing a map of the mobile robot.

[0047] Method 1: The mobile robot is controlled by map construction software to move so that the mobile robot collects sensor data while moving, and the travel path traveled by the mobile robot includes at least the planned path, i.e., the travel path traveled by the mobile robot may be more than the planned path, and it is sufficient to move along each path in the planned path at least once, so that the planned path set formed by all the planned paths in the planned path is a subset of the travel path set of the mobile robot.

[0048] As shown in Fig. 3, it is a schematic diagram showing a flow of a mobile robot according to an embodiment of the present invention collecting data for constructing a map of the mobile robot. Taking the beacon as an example, the mobile robot starts from a certain key node.

[0049] In step 301, the mobile robot reads the QR code information in the current beacon to perform initialization, and sends the beacon to the map construction software, so that the map construction software takes the topological point corresponding to the beacon as the starting point.

[0050] In step 302, the mobile robot receives a QR code corresponding to a target topological point determined by the map construction software based on the topological diagram, the planned route, and the currently arrived target key node, the target topological point corresponds to a next target key node adjacent to the currently arrived target key node, the key node where the QR code is located is the next target key node, and the current beacon read when the mobile robot initializes is a beacon in the target key node where the mobile robot has currently arrived.

[0051] The mobile robot determines the guide marker in the direction where the next target key node is located based on the received QR code, sets the direction of the guide marker as the current moving direction of the mobile robot, and moves according to the guide marker, or the mobile robot selects a guide marker in the direction where the received QR code (i.e., the next target key node) is located from the guide markers, sets the direction of the selected guide marker as the current moving direction of the mobile robot, and moves according to the guide marker. For example, when the mobile robot is at point C in Figure 2, if the target topological point is F, it moves according to the CF line segment, and if the target topological point is D, it moves according to the CD line segment.

[0052] In step 303, the mobile robot performs marker detection while moving and records data collected by the mobile robot's sensors in real time.

[0053] If a guide marker is detected, the mobile robot can move according to the currently detected guide marker to ensure the accuracy of the direction according to the guide marker.

[0054] If a beacon is detected, the mobile robot reads the QR code information in the beacon and matches the read QR code with the received QR code. If the matching is successful, it determines that the target key node has been reached and requests the next target key node from the map construction software, and returns to step 302 until all planned routes have been traversed at least once; otherwise, it determines that the target key node has not been reached, and the mobile robot moves according to the guide marker, and returns to step 303 until all planned routes have been traversed at least once.

[0055] In another manner of step 303, after the mobile robot reads the QR code in the beacon, it sends the read QR code to the map construction software, so that the map construction software matches the received QR code with the QR code corresponding to each topological point, and if the matching between the received QR code and the QR code corresponding to the target topological point is successful, it determines that the target key node has been reached, and sends a beacon corresponding to the next target topological point to the mobile robot, and the key node where the beacon is located is the target key node; otherwise, it determines that the target key node has not been reached, and the map construction software instructs the mobile robot to move according to the guide marker. If the mobile robot receives a beacon corresponding to the next target topological point until all planned routes are traversed, it returns to step 302, and if the mobile robot receives an instruction to move according to the guide marker, it returns to step 303.

[0056] In the process of detecting the marker, when the mobile robot reads the beacon, the mobile robot does not stop its current movement and acquires the beacon image according to the set second sampling frequency while maintaining the current running state, so that the mobile robot does not need to stop to collect the beacon image, which is advantageous to improving the efficiency of map construction. The second sampling frequency is related to the current moving speed.

[0057] Method 2: The mobile robot moves based on a planned path and a topological diagram.

[0058] As shown in Fig. 4, Fig. 4 is a schematic diagram showing a flow of a mobile robot according to another embodiment of the present invention collecting data for constructing a map of the mobile robot. Taking the beacon as an example, the mobile robot starts from a certain key node.

[0059] In step 401, the mobile robot reads the QR code information in the current beacon to perform initialization.

[0060] In step 402, the mobile robot obtains a planned path and a topological diagram in the selected scene area from the map building software, starting from the topological point corresponding to the current beacon.

[0061] In step 403, a target topological point and its corresponding QR code are determined based on the planned route, the topological diagram, and the currently arrived key node, and the key node where the QR code is located is the target key node, and a direction of movement is determined based on the received QR code, and movement is performed according to the guide marker.

[0062] In step 404, the mobile robot performs marker detection while moving and records data collected by the mobile robot's sensors in real time.

[0063] If a guide marker is detected, the mobile robot can move according to the currently detected guide marker to ensure the accuracy of the direction according to the guide marker.

[0064] If a beacon is detected, the mobile robot matches the currently detected beacon with the beacon corresponding to each topological point in the topological diagram, and if the matching is successful, it determines that the target key node has been reached, and determines the next target topological point and its corresponding beacon based on the planned path, the topological diagram, and the currently reached target key node, and moves from the currently reached target key node and returns to step 404 until it has traversed all the planned paths at least once; otherwise, it determines that the target key node has not been reached, and moves according to the guide marker, and returns to step 404 until it has traversed all the planned paths.

[0065] During the movement, in order to improve the efficiency of data collection, the mobile robot can move to the target key node based on the guide marker, and there is no need to feedback based on the position and attitude information of the key node and the beacon. In this way, the distance between the beacons is not limited, and the number of beacons required can be reduced, which is favorable for improving efficiency. In addition, the beacons can be flexibly installed, which improves the convenience of map construction. Furthermore, the movement path of the mobile robot can move according to the scene situation and the needs of data collection, and there is no need to stick to the planned path, as long as the planned path is a subset of the movement path set. For example, in FIG. 2, the movement path can be ABCDEFC, ABCFEDC, EDCFCBA, or CBAABCFEDC, etc., as long as the movement path includes AB path, BC path, CD path, ED path, EF path, and FC path. In this way, the movement path can be regardless of the navigation coordinates of the map construction, the movement path is flexible, and there is no need to pay attention to the positioning result during the movement process.

[0066] The data collected by the sensors includes at least a top-down ground texture image and beacon identification information, and may also include inertial navigation sensor data.

[0067] Finally, map construction is performed using map construction software based on the collected data.

[0068] The mobile robot transmits the collected data to the map-building software so that the map-building software performs map construction based on the collected data.

[0069] 5, which is a schematic diagram showing a flow of the map construction software according to the first embodiment of the present invention constructing a map. The map construction method includes the following steps:

[0070] In step 501, collected data from the mobile robot's sensors is synchronized.

[0071] Since the time stamps of the collected sensor data of the mobile robot are different, the collected sensor data of the mobile robot is matched.

[0072] In this embodiment, the collected data includes visual data from an image capture device, e.g., top-down ground texture images, and inertial data from an inertial sensor, e.g., wheel-based odometer data.

[0073] In step 502, visual data is processed to obtain keyframes and prepare them for subsequent optimization.

[0074] Here, the processing includes data pre-processing, feature extraction, matching and pose calculation, keyframe selection, etc.

[0075] In step 503, global optimization is performed based on the key frames to obtain global map information.

[0076] In this step, an optimization problem may be constructed, and global optimization may be performed based on the constructed optimization problem.

[0077] The optimization problem may be defined by state variables and their constraint relationships. In this embodiment, the state variables include the visual frame and the global beacon. The constraints may include an odometer constraint, a visual matching constraint, a loopback constraint, and a visual-beacon observation constraint. The initial values ​​of the optimization state variables may be given by the keyframes, or may be recalculated based on the constraint relationships before optimization.

[0078] The global optimization can be solved by iteratively solving a nonlinear least squares problem (eg, the LM algorithm).

[0079] After the optimization is completed, a globally consistent map information can be obtained.

[0080] In step 504, the map information is converted into a map format for positioning navigation and saved, completing the generation of the final map.

[0081] Example 2 In the second embodiment, the marker information having the same marking content is used as a beacon, and the guide marker is used as a patrol route, and a visual laser combined map is constructed based on the down-looking visual texture and laser data.

[0082] First, the initial preparations are carried out.

[0083] In this embodiment, when there is no existing route in the actual scene, for example, in a local open area in the scene, a topological diagram is created according to the needs of map construction using map construction software, and the topological diagram includes the logical relative position relationship between the topological points and the logical connection relationship between the topological points.

[0084] Based on the logical relative positional relationship between topological points, key nodes are provided in the actual scene, and beacons are provided in the key nodes, each beacon has marker information with the same marking content, and the beacons can be cross beacons, American beacons, etc. The key nodes do not include the overall position and orientation information of the key nodes, nor do they include the overall position and orientation information of the beacons, that is, there is no need to manually measure the position and orientation information in the key nodes and the position and orientation information of the beacons.

[0085] Based on the logical connection relationship between topological points, in the actual scene, ribbons or elastic lines are pasted as guide markers between key nodes to correspond to the logical connection relationship between topological points in the topological diagram. As an exception, when the distance between key nodes is smaller than a set distance threshold, no guide markers may be provided.

[0086] The logical relative positional relationships and logical connection relationships between topological points in a topological diagram are stored in the topological diagram as attributes of the topological diagram.

[0087] Referring to FIG. 6, FIG. 6 is a schematic diagram showing a scene configuration and a topological diagram according to the second embodiment of the present invention. In the figure, the left diagram shows the provided cross beacon and guide marker, and the right diagram shows the drawn topological diagram. Here, the topological diagram visualizes the logical relative positional relationship between the topological points and the logical connection relationship between the topological points, so it is not a full-scale image corresponding to the route and key nodes in the actual scene, but a logical correspondence relationship. For example, when providing a guide marker between two key nodes A and B in an actual scene, it does not need to be a straight line strictly, and it may be a curved line with curvature or a straight line that is easy for a mobile robot to travel.

[0088] A planned path is generated based on the topological diagram using map construction software, and the topological diagram and the planned path are sent to the mobile robot, where the planned path is a collection of directed paths consisting of topological points in the topological diagram.

[0089] The mobile robot determines a target key node based on the topological diagram, and moves to the target key node in a circular path manner.

[0090] As shown in Fig. 7, Fig. 7 is a schematic diagram showing the flow of moving to a target key node in a circular route manner in embodiment 2. The specific moving process is as follows:

[0091] The mobile robot gets a topological diagram and a planned path from the map-building software, and starts from some key node.

[0092] In step 701, after the mobile robot is initialized, it sets the topological point corresponding to the key node in the topological diagram as the starting point, and reads the beacon at the current key node.

[0093] In step 702, the mobile robot determines a target topological point in the topological diagram based on the planned path, the currently reached key node, and the topological diagram, sets the key node corresponding to the target topological point as the target key node, determines a direction of movement based on the target key node, and moves according to the guide marker.

[0094] For example, when the target topological point is at point C in FIG. 6, if the target topological point is F, the target moves along the CF line segment, and if the target topological point is D, the target moves along the CD line segment.

[0095] In step 703, the mobile robot performs marker detection while moving and records data collected by the mobile robot's sensors in real time.

[0096] If a guide marker is detected, then the currently detected guide marker can be followed to ensure directional accuracy according to the guide marker.

[0097] When a beacon is detected, it is determined whether the currently detected beacon and the previously detected adjacent beacon are the same beacon based on a first distance between the current key node and the previous adjacent key node recorded by an inertial odometer (i.e., the first distance between the currently detected beacon and the previously detected adjacent beacon).

[0098] If the first distance is equal to the set distance threshold, determine that the currently detected beacon and the detected previous adjacent beacon are different beacons, indicating that the mobile robot has moved to the next key node, and determine the topological point corresponding to the currently detected beacon based on matching the order of the adjacent beacons and the order of the adjacent topological points in the planned route, for example, in the figure, the planned route is ABCDEFC, when the mobile robot starts from the key node A and moves to the key node B according to the guide marker, the order of the adjacent beacons is AB, and determine the topological point corresponding to the beacon B according to the order of AB in the planned route, and execute step 704. Otherwise, determine that the currently detected beacon and the detected previous adjacent beacon are the same beacon, indicating that the mobile robot has not yet moved to the next key node, and return to step 703.

[0099] To avoid false detection of a beacon due to overlap of the beacon's features with existing features on the ground. For example, if the beacon is a cross beacon, misrecognizing the joint of the cross tiles on the ground as a cross marker will cause a mismatch between the cross marker and a topological point in the topological diagram.

[0100] In a situation where false detection is likely to occur, in addition to selecting a feature that can be distinguished from the ground, such as a rice feature, as a beacon, a second distance may be set between any two topological points in the topological diagram, the second distance being the measured distance between the beacons at two key nodes corresponding to the two topological points, and preferably, the two topological points are two topological points having a logical connection relationship.

[0101] In this manner, the second distance may be the distance threshold, and if the first distance is not equal to the second distance between the two topological points in the topological diagram, it may be determined that the current beacon has a false positive, thereby eliminating the false positive beacon through distance verification.

[0102] In step 704, the mobile robot determines a next target topological point based on the planned path, the currently reached target key node, and the topological diagram, and returns to step 703 until all the planned paths are traversed at least once. Here, the moving path along which the mobile robot moves includes at least the planned path, that is, the planned path is a subset of the moving path set of the mobile robot.

[0103] During the movement, in order to improve the efficiency of data collection, the mobile robot can move to the target topological point based on the guide marker, and does not need to feedback by the position and attitude information of the key node and the beacon. In this way, the distance between the beacons is not limited, and the number of beacons required can be reduced, which is favorable to improving efficiency. Furthermore, the movement path of the mobile robot can move according to the scene situation and the needs of data collection, and does not need to stick to the planned path, as long as the planned path is a subset of the movement path set. For example, in FIG. 6, the movement path can be ABCDEFC, can be ABCFEDC, can be EDCFCBA, or can be CBAABCFEDC, etc., as long as the movement path includes AB path, BC path, CD path, ED path, EF path, and FC path. In this way, the movement path can be regardless of the navigation coordinate of the map construction, the movement path is flexible, and there is no need to pay attention to the positioning result during the movement process.

[0104] In this embodiment, the data collected by the sensors includes at least a top-down ground texture image, inertial navigation sensor data, and laser data.

[0105] Finally, map construction is performed using map construction software based on the collected data.

[0106] The mobile robot transmits the collected data to the map-building software so that the map-building software performs map construction based on the collected data.

[0107] Referring to Figure 8, Figure 8 is a schematic diagram showing a flow of a map construction software according to embodiment 2 of the present invention constructing a map. The map construction method includes the following steps:

[0108] In step 801, collected data from the mobile robot's sensors is synchronized.

[0109] Since the timestamps of the collected sensor data of the mobile robot are different, the collected sensor data is matched.

[0110] In this embodiment, the collected data includes visual data from an image capture device, e.g., top-down ground texture imagery, inertial data from an inertial sensor, e.g., inertial odometer data, and laser data from a laser radar sensor.

[0111] In step 802, the visual and laser data are processed to obtain keyframes and prepare them for subsequent optimization.

[0112] Here, the processing includes data pre-processing, feature extraction, matching and pose calculation, keyframe selection, etc.

[0113] In step 803, global optimization is performed based on the key frames to obtain global map information.

[0114] In this step, an optimization problem may be constructed, and global optimization may be performed based on the constructed optimization problem.

[0115] The optimization problem may be defined by state variables and their constraint relationships. In this embodiment, the state variables include visual frame, laser frame, and global beacon. The constraints include global beacon. The constraints may include odometer constraint, visual matching constraint, loopback constraint, visual-beacon observation constraint, laser matching constraint, and visual laser association constraint. Initial values ​​of the optimization state variables may be given by the keyframe, or may be recalculated based on the constraint relationships before optimization.

[0116] The global optimization can be solved by iteratively solving a nonlinear least squares problem (eg, the LM algorithm).

[0117] After the optimization is completed, a globally consistent map information can be obtained.

[0118] In step 804, the map information is converted into a map format for positioning navigation and saved, completing the generation of the final map.

[0119] In the first and second embodiments of the present invention, the map may be constructed after all data collection is completed, or the map may be constructed while receiving the collected data, and the constructed map may be a visual map, a laser map, or a combined visual-laser map according to the needs and the information recorded in the sensor, and the present invention is not limited to a specific map construction method, and the map may be constructed using a SLAM map construction method based on optimization. The map construction software may run on the PC side, a server, or the mobile robot body.

[0120] As shown in FIG. 9, FIG. 9 is a schematic diagram showing a data collection device for constructing a map of a mobile robot according to an embodiment of the present invention, the device including: a target key node acquisition module, a movement control module, and a collection module.

[0121] The target key node acquisition module is used to acquire a target key node.

[0122] The migration control module is used to control the migration process to the target key node.

[0123] The collection module is used to collect data from the mobile robot's sensors while it is moving.

[0124] Here, the target key node is determined based on at least topological information, the topological information including logical relative positional relationships between topological points and logical connection relationships between the topological points, beacons are provided at key nodes in the actual scene, the topological points have a one-to-one correspondence with the beacons at the key nodes in the actual scene, and the logical connection relationships have a one-to-one correspondence with the guide markers connected between the key nodes.

[0125] During movement, the guide markers are used to determine the direction of movement, and the beacons are used to determine the target key node to be reached, without using position and orientation information of any of the key nodes and any of the beacons.

[0126] The movement control module includes a marker detection sub-module and a movement control sub-module.

[0127] The marker detection submodule is used to perform marker detection during movement, and when a guide marker is detected, it outputs a movement command to the movement control submodule, and when a beacon is detected, it sends a request command to the target key node acquisition module.

[0128] The movement control sub-module is used to move based on the movement command from the marker detection sub-module and the target key node information from the target key node acquisition module.

[0129] Please refer to FIG. 10, which is a schematic diagram showing a map construction system according to the present invention, which includes a collection device and a map construction device.

[0130] The collection device is used to collect data for building a map of the mobile robot.

[0131] The map builder is used to build a map using data for building a map for a mobile robot.

[0132] The map builder includes a synchronization module, a tracking module, an optimization module, and a map generation module.

[0133] The synchronization module is used to synchronize the collected data from the sensors of the mobile robot.

[0134] A tracking module is used to process the collected image data and obtain key frames.

[0135] The optimization module is used to perform global optimization based on the key frames to obtain global map information.

[0136] The map generation module is used to convert the map information into a map format.

[0137] 11, which is a schematic diagram showing a collection device, a map construction system, or a mobile robot according to an embodiment of the present invention, including a memory and a processor, a computer program stored in the memory, and when the processor executes the computer program, steps of constructing a data collection method for constructing a map of the mobile robot according to the embodiment of the present invention and / or steps of a map construction method are performed.

[0138] The memory may include Random Access Memory (RAM) and / or Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may be at least one storage device located remotely from the processor.

[0139] The processor may be a general-purpose processor including a Central Processing Unit (CPU), a Network Processor (NP), etc., or may be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0140] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs steps of the method for collecting data for constructing a map of a mobile robot of the embodiment of the present invention and / or steps of the method for constructing a map.

[0141] The embodiments of the apparatus / network side device / storage medium are basically similar to the embodiments of the method, so the description thereof will be brief, and reference may be made to some of the descriptions of the embodiments of the method for relevant parts.

[0142] In the present invention, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between those entities or operations. Also, the terms "comprise", "contain" or any other variation thereof are intended to include a non-exclusive inclusion, whereby a process, method, article or device that includes a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes the inherent elements of such process, method, article or device. In the absence of more limitations, an element qualified by the phrase "comprises a ..." does not exclude the presence of further identical elements in the process, method, article or device that includes said element.

[0143] The above is merely a preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for collecting data for constructing a map of a mobile robot, which is applied to a mobile robot, comprising the steps of: obtaining a target key node; moving to the target key node; and collecting data from sensors of the mobile robot while it is moving; The target key node is determined based on topological information, the topological information including a logical relative position relationship between topological points and a logical connection relationship between the topological points, a beacon is provided at the key node in the actual scene, the topological point corresponds one-to-one with the beacon at the key node in the actual scene, and the logical connection relationship corresponds one-to-one with the guide marker connected between the key nodes; During movement, a direction of movement is determined using a guide marker, a target key node is determined using a beacon, and position and orientation information of any key node and any beacon is not used. A method for collecting data for constructing a map of a mobile robot, comprising:

2. The topological information further includes a beacon corresponding to the topological point; The step of acquiring a target key node includes acquiring a beacon of the target key node from a map construction software, and the target key node is determined by the map construction software based on a planned route, the topological information, and a topological point corresponding to a current key node reached by the mobile robot. The collection method according to claim 1 .

3. During movement, the direction of movement is determined using the guide marker, the target key node is determined using the beacon, and the position and orientation information of any of the key nodes and any of the beacons is not used. The mobile robot performs marker detection; When a guide marker is detected, the robot moves according to the detected guide marker; If a beacon is detected, The mobile robot matches the beacon with a beacon of a current target key node from the map construction software, and if the matching is successful, determines that the current target key node has been reached, and requests the map construction software for the next target key node, so that the map construction software transmits the beacon of the next target key node to the mobile robot; Alternatively, the mobile robot transmits the beacon to the map construction software, so that the map construction software matches the current beacon with a beacon corresponding to each topological point in the topological information, and if the matching is successful, obtains the topological point corresponding to the current beacon, and transmits a beacon corresponding to the next topological point to the mobile robot as a beacon of a next target key node of the mobile robot; Different beacons have different marking content. The collection method according to claim 2 .

4. The method further includes the step of the mobile robot acquiring the topological information and the planned path from a map construction software; The step of acquiring the target key node includes a step of the mobile robot determining a target topological point based on the planned route, the topological information, and a topological point corresponding to a current key node reached by the mobile robot, and setting the key node corresponding to the target topological point as the target key node; The collection method according to claim 1 .

5. The topological information further includes a beacon corresponding to the topological point; During movement, the direction of movement is determined using the guide marker, the target key node is determined using the beacon, and the position and orientation information of any of the key nodes and any of the beacons is not used. The mobile robot performs marker detection; When a guide marker is detected, the robot moves according to the detected guide marker; When a beacon is detected, matching the beacon with a beacon corresponding to each topological point in the topological information; and when the matching is successful, determining that the current target key node has been reached and obtaining the next target key node; Different beacons have different marking content. The collection method according to claim 4 .

6. During movement, the direction of movement is determined using the guide marker, the target key node is determined using the beacon, and the position and orientation information of any of the key nodes and any of the beacons is not used. The mobile robot performs marker detection; When a guide marker is detected, the robot moves according to the detected guide marker; if detecting a beacon, determining whether the currently detected beacon and the previously detected adjacent beacon are the same beacon based on a first distance between the currently detected beacon and the previously detected adjacent beacon recorded by an inertial odometer; If the first distance is equal to a set distance threshold, determining that the currently detected beacon and the detected previous adjacent beacon are not the same beacon, and determining a topological point corresponding to the currently detected beacon based on matching an order of the currently detected beacon and the detected previous adjacent beacon with an order of adjacent topological points in a planned route, thereby obtaining a next target key node; Different beacons have the same marking content, The collection method according to claim 4 .

7. The topological information further includes a second distance between two topological points having a logical connection relationship, the second distance being a measured distance between two key nodes respectively corresponding to the two topological points; the distance threshold is the second distance; determining whether a currently detected beacon and a previously detected neighboring beacon are the same beacon based on a first distance between the currently detected beacon and the previously detected neighboring beacon recorded by an inertial odometer; and determining that there is a false positive in the detection for the currently detected beacon if the first distance is not equal to a set distance threshold. The collection method according to claim 6 .

8. a planned path set formed by all the planned paths in the planned path is a subset of a moving path set along which the mobile robot moves during movement, and the mobile robot moves at least once along each planned path in the planned path set; The step of collecting data from a sensor of the mobile robot during movement includes the step of collecting data from the sensor according to a first sampling frequency set during movement; performing marker detection by the mobile robot includes acquiring and detecting a beacon according to a set second sampling frequency while the mobile robot is in a running state so that a current running state is not interrupted; the first sampling frequency and the second sampling frequency are related to a current moving speed of the mobile robot; The collection method according to any one of claims 3, 5, 6 and 7.

9. constructing a map using the data for constructing a map of the mobile robot; The data for constructing a map of a mobile robot is collected according to the method for collecting data for constructing a map of a mobile robot according to any one of claims 1 to 8. A map construction method comprising:

10. The step of constructing a map using the data for constructing a map of the mobile robot includes: synchronizing collected data from sensors of the mobile robot; processing the collected image data to obtain key frames; performing global optimization based on the key frames to obtain global map information; converting the map information into a map format; 10. The method of claim 9.

11. A data collection device for constructing a map of a mobile robot, comprising: a target key node acquisition module for acquiring a target key node; a migration control module for controlling a migration process to the target key node; a collection module for collecting data from sensors of the mobile robot while moving; The target key node is determined based on topological information, the topological information including a logical relative position relationship between topological points and a logical connection relationship between the topological points, a beacon is provided at the key node in the actual scene, the topological point corresponds one-to-one with the beacon at the key node in the actual scene, and the logical connection relationship corresponds one-to-one with the guide marker connected between the key nodes; During movement, a direction of movement is determined using a guide marker, a target key node is determined using a beacon, and position and orientation information of any key node and any beacon is not used. A data collection device for constructing a map of a mobile robot, comprising:

12. A data collection device for constructing a map of a mobile robot according to claim 11; A map construction device for constructing a map using data for constructing a map of a mobile robot, A map construction system comprising:

13. The map construction device comprises: a synchronization module for synchronizing collected data from the sensors; a tracking module for processing the collected image data and obtaining key frames; an optimization module for performing global optimization based on the key frames to obtain global map information; a map generation module for converting the map information into a map format; 13. The system of claim 12.

14. A data collection device for constructing a map of a mobile robot according to claim 11, and / or a map construction system according to any one of claims 12 to 13. A mobile robot characterized by:

15. A computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to perform steps of the method for collecting data for constructing a map for a mobile robot according to any one of claims 1 to 8 and / or steps of the method for constructing a map according to any one of claims 9 to 10. A computer-readable storage medium comprising:

Images