Method for locating mobile device
By adopting the hierarchical structure priority sorting and incremental positioning mechanism of multi-source data in positioning calculation of mobile devices, the problem of reduced accuracy in SLAM positioning method in practice is solved, ensuring navigation accuracy and stability.
Patent Information
- Application Number
- CN202410601128.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-15
- Filing Date
- 2024-05-15
- Publication Date
- 2024-11-15
AI Technical Summary
When using SLAM-based positioning methods, mobile devices are prone to problems of reduced accuracy or complete failure in practice, such as loss of sensor data, wheel slip, hijacking and shading, resulting in reduced navigation accuracy.
By hierarchical structure prioritization of multi-source data in position and orientation calculations of mobile devices, different types of sensor data such as inertial measurement units, wheel odometers, and speed models are used to ensure reliable navigation when errors occur Information and improve positioning accuracy and stability through incremental positioning and data set comparison mechanisms.
It realizes that sufficient navigation accuracy can be maintained in case of sensor errors or unavailability, reduces the risk of navigation failure, and improves the autonomous navigation capabilities of mobile devices in complex environments.
Smart Images

Figure CN118962666A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for localizing a mobile device, in particular a vehicle or a robot that moves at least partially automatically, a system for data processing and a computer program for carrying out the method, and a mobile device. Background Art
[0002] Mobile devices, such as vehicles or robots that move at least partially automatically, typically move in the surroundings (especially in the surroundings to be processed or in the work area, such as at home), in the garden, in the factory hall or on the road, in the air or in the water. One of the basic problems of such or other mobile devices is to orient, that is, to know what the surroundings are like, that is, to know where obstacles or other objects are, and where their (absolute) position is. For this purpose, the mobile device can be equipped with different sensors, such as cameras, lidar sensors, radar sensors or inertial sensors, which can be used to detect the surroundings and the movement of the mobile device in two or three dimensions. This enables the mobile device to move locally, identify obstacles in time and bypass them. Summary of the invention
[0003] According to the invention, a method for locating a mobile device, a system for data processing and a computer program for carrying out the method, and a mobile device are proposed having the features of the independent claims. Advantageous embodiments are the subject matter of the dependent claims and the following description.
[0004] The invention generally relates to mobile devices that move or at least can move in the surroundings or there, for example in a work area. Examples for such mobile devices (or mobile working devices) are, for example, robots and / or drones and / or also vehicles that move partially or (fully) automatically (on land, water or air). For example, household robots, such as cleaning robots (for example in the form of suction robots and / or wiping robots), ground- or road cleaning devices, construction robots or weeding robots are considered as robots, but other so-called service robots are also considered, such as vehicles that move at least partially automatically, such as personnel transport vehicles or object transport vehicles (also so-called ground transport vehicles in warehouses, for example), but also flying vehicles, such as so-called drones or water vehicles.
[0005] Such a mobile device has in particular a control or regulating unit and a drive unit for moving the mobile device so that the mobile device can move in the surroundings, for example along a movement path. For this purpose, navigation information can be determined, i.e., for example, determined instructions: in which direction the mobile device should be driven in order to follow the movement path. This can then be implemented by the control or regulating unit and the drive unit. In general, this can be navigation of the mobile device.
[0006] Furthermore, the mobile device can have one or more sensors, by means of which the surroundings or information in the surroundings can be detected. These can be, for example, cameras, lidar sensors, radar sensors or inertial measurement units (or inertial sensors) and wheel odometers, by means of which the surroundings and the movement of the mobile device can be detected, for example, in two or three dimensions. However, other or additional sensors can also be provided, depending on the type of mobile device.
[0007] In this context, with regard to sensors, a distinction can be made in particular between different types of sensors, namely so-called device sensors and so-called environmental sensors. With the aid of device sensors, such as inertial measurement units and wheel odometers, device information, in particular device sensor data, can be detected and provided, which includes information about the position and / or movement of the mobile device, such as the speed, acceleration or yaw rate of the mobile device. With the aid of environmental sensors, such as cameras, lidar sensors or radar sensors, environmental information, in particular environmental sensor data, can be detected and provided, which includes information about the surroundings of the mobile device, such as also the distance of the mobile device (or the associated sensor) from objects in the surroundings.
[0008] A possible solution for determining the position and orientation of such a mobile device and thus also for navigating it is to use SLAM-based localization of the mobile device. SLAM (“Simultaneous Localization and Mapping”) refers to a method in robotics in which a mobile device, such as a robot, can or must simultaneously create a map of its surroundings and estimate its spatial position within the map. It is therefore used to detect obstacles and thus supports autonomous navigation.
[0009] There are different schemes for representing maps and locations in SLAM. Traditional methods for SLAM are usually based on geometric information, such as nodes and edges. Nodes and edges are typically components of a SLAM graph. The nodes and edges in a SLAM graph can be designed differently; traditionally, a node corresponds to, for example, the posture (position and orientation) of a mobile device or a specific environmental feature at a specific point in time, while an edge represents a relative measurement between a mobile device and an environmental feature. SLAM graphs are described in more detail in, for example, "Giorgio Grisetti, Rainer Kümmerle, Cyrill Stachniss, Wolfram Burgard, Graph-based SLAM Tutorial, IEEE Intelligent Transportation Systems Magazine, 2010, Vol. 2(4), pp 31-42" ("Giorgio Grisetti, Rainer Kümmerle, Cyrill Stachniss, Wolfram Burgard, IEEE Intelligent Transportation Systems Magazine, Vol. 2(4), pp 31-42, 2010").
[0010] Based on such a SLAM map, a map of the surrounding environment (environmental map) can be determined in which the mobile device moves. With each new data set with information for the surrounding environment and / or for the mobile device (the information is obtained by one or more sensors of the mobile device or is based on the sensors), the map (or SLAM map) can be expanded or updated.
[0011] In the case of positioning with the aid of SLAM, a positioning process can be used here. The positioning process generally includes providing device- and environment information (which is at least partially detected from the device and / or the surroundings by means of sensors of the mobile device), i.e., for example, providing the mentioned device- and / or environment information. Based on the device- and environment information, a data set is generated, which is then provided for positioning and / or navigating the mobile device. The environment information or environment sensor data can, for example, include so-called point clouds, such as those provided by typical lidar sensors. The device information can then be used in addition to determine the position and orientation (also collectively referred to as so-called attitude) of the mobile device for the point cloud.
[0012] Such a data set obtained during the positioning process can then be compared within the framework of the positioning with a previous data set, that is to say a data set obtained during a previous positioning process. In this case, an attempt is made to make the two data sets consistent, at least within a certain tolerance, in order to then determine the movement or trajectory of the mobile device. This can also be a so-called matching or scan matching (the latter particularly relates to the case of LiDAR sensors, which detect data within the framework of a scan). The comparison of a data set with a (typically immediately preceding) previous data set is also referred to as incremental positioning.
[0013] However, usually not every new data set is added to the aforementioned surroundings map; rather, new data sets can be added to the surroundings map only at certain time intervals and / or spatial intervals, for example. The data volume is thus kept low.
[0014] While positioning using SLAM usually works very well under good conditions, in practice, as has already been shown, a number of problems can occur, for example, in incremental positioning. Such problems can be, for example, loss of sensor data, wheel slip, so-called hijacking (this should be understood as the mobile device, mostly domestic robots, being picked up and put down again at another location), and covering of sensors (this then particularly concerns environmental sensors).
[0015] If the results of a positioning process, in particular an incremental positioning process in which errors occur, are simply continued to be used, this can lead to a reduction in accuracy (in positioning) or to a complete failure of the SLAM algorithm. Against this background, a possible solution for avoiding potential errors in the navigation of a mobile device is proposed.
[0016] One specific embodiment relates to a method for determining the position and orientation of a mobile device and thus in particular also for navigating the same, wherein the mobile device moves or is to move in an environment using SLAM-based positioning.
[0017] Data are provided here, which include information about the position and / or movement of the mobile device. This can be the device information already mentioned. The data are obtained from different types of sources. Sensors such as, for example, inertial measurement units and wheel odometers are considered as sources, but other sources are also considered, such as, for example, models or assumptions for the current speed. In this case, "different types of sources" should be understood in particular to mean that there are multiple sources, i.e., for example, a sensor and a model or two different, especially also different types of sensors.
[0018] It is provided here that one or more partial aspects of position and orientation (posture) are each assigned a source set consisting of different types of sources. In the corresponding set, the sources are in turn assigned a predetermined hierarchy in terms of priority. Position and orientation can have, for example, six degrees of freedom, three translational degrees of freedom (position) and three rotational degrees of freedom (orientation), and a partial aspect is to be understood as a part of these total degrees of freedom. However, it should also be mentioned that not all six degrees of freedom must be taken into account in position and orientation, so that, for example, in the case of a mobile device that moves only on a plane, the vertical direction can remain unconsidered. The predetermined hierarchy in terms of priority is to be understood as an order in which the sources of the set are arranged, according to which the sources are sorted.
[0019] A simple example is that the yaw angle (rotational degree of freedom) as a partial aspect of position and orientation is assigned to a source set that includes an inertial measurement unit, a wheel odometer, and a model for speed. The predetermined hierarchy in terms of priority can correspond, for example, to the order mentioned here for the three sources.
[0020] The availability of at least some of the different types of sources is then checked for the corresponding set (and therefore also for the corresponding partial aspects). In addition, it is determined whether the checked source is available. The availability of a source can be understood in particular as whether the source provides its information at all, or whether the source provides, for example, inaccurate information, which should therefore, for example, not be used.
[0021] Based on these data, the position and orientation of the mobile device, in particular the so-called pose, is determined in order to locate the mobile device.
[0022] Here, one or at least one of the multiple partial aspects is determined based on data from a selected source of the respectively assigned source set. In the example mentioned, the yaw rate is then determined, for example, based on data from an inertial measurement unit or based on data from a wheel odometer or based on data from a model for speed.
[0023] The following source is now used as the selected source from the corresponding source set, which is assigned the highest priority according to the corresponding hierarchy among the sources determined to be available. If all three sources are available in the example mentioned, the data of the inertial measurement unit are used in order to determine the yaw rate based on them. This also applies when at least the inertial measurement unit is available but, for example, the wheel odometer is not available. Conversely, when the inertial measurement unit is not available but the wheel odometer is available, the yaw rate is determined based on the wheel odometer.
[0024] This hierarchy makes it possible to always use the best possible source of the available sources, so that even in the case of possible errors that render a source unusable, sufficient accuracy is achieved. If, on the other hand, a plurality of sources are weighted relative to one another, even though possible minor inaccuracies can be compensated, greater deviations from erroneous sources can greatly distort the result. The hierarchy or order of the sources in such a set can be determined, for example, according to requirements and circumstances or according to the type of source.
[0025] It should be mentioned here that this only applies in particular to the respective sub-aspects, i.e. other sources and / or another hierarchy can be provided depending on the sub-aspects. For example, two specific sensors can be provided for two different sub-aspects, but with different hierarchy structures, for example because one sensor provides more accurate values for a sub-aspect than the other sensor, while the opposite is true for the other sub-aspect.
[0026] Likewise, the determination of whether a source is available is also applicable individually to each aspect or the corresponding set of sources. A specific sensor can be assigned to two different aspects, for example, but only be considered unavailable in one of the aspects. This can be the case, for example, if the sensor provides the identified inaccuracy and a sensor with more accurate data is still assigned to one aspect, but not to the other aspect. Conversely, if a sensor no longer provides data at all, it is no longer considered available in any of the aspects.
[0027] In one specific embodiment, the different types of sources include at least two of the following sources: one or more sensors, one or more position and / or motion models, and one or more position and / or motion specifications. In particular, the different types of sources include an inertial measurement unit, a wheel odometer, a target speed, a constant speed model, a constant speed, a constant position model, and a constant position.
[0028] Sensors, i.e., inertial measurement units and wheel odometers, provide current measured values, while the model can be used as a fallback. If, for example, no sensors are available any more, it can be assumed that the mobile device continues to move at the last measured or determined speed or is at the last determined position. However, if present, for example, a preset value, i.e., a target speed, can also be taken into account, which does not have to be constant. This applies not only to position and translational speed, but also to orientation and rotational speed.
[0029] In one specific embodiment, one or more of the aspects of the position and orientation include the position along the horizontal direction of movement, i.e., for example, along the x-direction and / or the y-direction (in the case of a Cartesian coordinate system). This aspect is then assigned a source set having the following predefined hierarchy: wheel odometer, target speed, constant speed model and constant position model.
[0030] In one specific embodiment, a partial aspect or one of the partial aspects of the position and orientation comprises a yaw angle. The partial aspect is assigned a source set having the following predetermined hierarchy: inertial measurement unit, wheel odometer, target speed, constant speed model and constant position model.
[0031] In one specific embodiment, one or more of the partial aspects of the position and orientation include a pitch angle and / or a roll angle. The partial aspect is assigned a source set having the following predefined hierarchy: inertial measurement unit, constant velocity and constant position.
[0032] The three mentioned partial aspects allow the most accurate possible determination of the position and orientation of the mobile device and thus navigation of the mobile device, that is to say even in the case of possible errors or other reasons (thus the source or sensor is not available). It should be mentioned that even if the proposed solution is only used in one of the partial aspects, an improvement is already achieved.
[0033] In one embodiment, when two sources of different types, each including a sensor, are assigned the same partial aspect of position and orientation, in particular the yaw angle, one of the two sources is determined to be unavailable if the data obtained by the two sources and corresponding to each other deviate from each other by more than a predetermined value. One of the two sources can include an inertial measurement unit and the other of the two sources can include a wheel odometer. However, this can also be the case in general with two different sensors, with the aid of which a specific partial aspect can be determined. In this case, the two sensors can provide different types of data, but the same partial aspect can be determined using these data. The corresponding data are then compared. An inertial measurement unit, such as a gyroscope, is used, for example, to determine the rotation rate (based on which the yaw rate can then be determined (or the yaw rate can be used as the rotation rate)), while the wheel odometer provides, for example, the length of the distance covered by the two wheels, which can be compared in order to determine the yaw rate based on this.
[0034] Likewise, the two sensors do not have to be sensors of different types, but can also be two sensors of the same type, for example two inertial measurement units, which, for example, have different qualities or are arranged at different locations on the mobile device.
[0035] It is particularly preferred that if the corresponding data from the source including the inertial measurement unit and the source including the wheel odometer deviate from each other by more than a predetermined value, the source including the wheel odometer is determined to be unavailable. In other words, when the corresponding values provided by the inertial measurement unit and the wheel odometer, for example, for the yaw rate, deviate greatly from each other, the inertial measurement unit is given priority. Because it has been proven that the inertial measurement unit provides more accurate measurement values than the wheel odometer, at least within the range of a shorter duration. When one of the two is wrong, it is more likely to be the wheel odometer.
[0036] However, as already mentioned, this does not necessarily mean that the wheel odometers are no longer used for other parts. For horizontal positions, the information from the wheel odometers can always be more accurate than the information from the model or the target speed. However, depending on the degree of deviation between the values of the inertial measurement unit and the values of the wheel odometers, for example, the wheel odometers can also be determined to be unusable for horizontal positions.
[0037] In one specific embodiment, a source, in particular a sensor, is determined to be unavailable when no data is or has been obtained from the source for at least one predetermined time period and / or when an error is detected for which it is predetermined that the source should be unavailable. In this way, further errors can be taken into account. When no data is obtained for a source or a sensor for at least one predetermined time period, the source can, as mentioned, be determined to be unavailable, in particular for a plurality of sub-aspects (when the source is associated with a plurality of sub-aspects). In this case, it can then be assumed, for example, that the sensor or the communication connection is defective.
[0038] Different possibilities are considered as errors, for which the source should be unavailable. For example, a so-called lifting sensor can be set at the mobile device, which is triggered when the mobile device (especially in the case of a household robot) is lifted. This can be, for example, a contact sensor, which is installed in the wheel suspension of the mobile device. When the lifting can now be detected, especially as long as the mobile device is lifted, the wheel odometer is determined to be unavailable, that is, the wheel odometer does not provide reasonable values when it is not in contact with the ground. However, such a lifting and later lowering can also be identified by an inertial measurement unit, for example. Another error is, for example, a step in the wheel odometer or its data. But it is also, for example, a slip in the yaw rate, which can be identified, for example, by comparing the data or values of the inertial measurement unit and the wheel odometer (this can occur, for example, when the mobile device moves laterally due to external forces). All these errors indicate, for example, that the wheel odometer should not be used, that is, the wheel odometer should be determined to be unavailable.
[0039] Another error is, for example, a stationary state of the robot, which can be detected based on the inertial measurement unit and / or the wheel odometer. This stationary state can be used, for example, to reduce the computational load and eliminate the accumulation of small errors (e.g. due to gradual drift of the inertial measurement unit) by avoiding unnecessary calculations during periods of inactivity of the mobile device.
[0040] In one specific embodiment, navigation information for the mobile device is also determined, more precisely based on the position and orientation of the mobile device. This can then include, in particular, the consideration of further sensor data, such as, for example, a lidar sensor, as has already been mentioned and as will be explained in more detail below.
[0041] One specific embodiment of the present invention relates to a method for localizing a mobile device and therefore in particular also for navigating the same, wherein the mobile device moves or is to move in an environment using SLAM-based localization.
[0042] In this case, device information, in particular device sensor data, detected by means of sensors of the mobile device is provided during the positioning process. The device information includes information about the position and / or movement of the mobile device. In addition, environmental sensor data detected by means of sensors of the mobile device are provided. The environmental sensor data includes information about the surroundings of the mobile device in order to obtain a current data set. The device information and environmental sensor data can be the information or data that has been explained in more detail above, i.e., information obtained, for example, by an inertial measurement unit and a wheel odometer, and, for example, by a lidar sensor, a radar sensor or a camera.
[0043] In the case of environmental sensor data, it may be the case that not all of these environmental sensor data are detected at the same point in time. This is the case, for example, with laser radar sensors, which usually scan the surroundings by means of a rotational or oscillating movement, which requires a certain duration. However, the mobile device continues to move during this period. In this case, depending on the desired accuracy, a correction can be carried out with respect to the position of the mobile device and, if necessary, its orientation, or further preprocessing can be carried out. This will be explained in more detail later.
[0044] For the positioning process, there is typically a data set of environmental sensor data, i.e., for example, a point cloud from a scan of a lidar sensor, while there are usually multiple data sets of device information or device sensor data, since the scanning frequency of an inertial measurement unit or a wheel odometer is usually greater than the scanning frequency of a lidar sensor.
[0045] Based on the device information, the temporary position and orientation of the mobile device (i.e., the temporary attitude, also referred to as the initial attitude) is then determined. For this purpose, for example, it is possible to determine from the device information what position and orientation the mobile device has relative to a previous positioning process. For a positioning process, in the case of a plurality of data sets of device information, for example, the last acquired data set or an average value of these data sets can be used.
[0046] A first check (or first check process) is then performed. This includes a first comparison: whether the current data set with the temporary position and orientation is consistent with one or more previously generated data sets or an existing environment map, preferably a data set provided in a previous (i.e., in particular, immediately previous) positioning process, according to at least one first consistency criterion. In the first check process, a check result is then obtained; if the data sets are consistent, the check result can be positive, otherwise, for example, negative.
[0047] In such an alignment, for example, an algorithm is used which works under consideration of a cost function which compares the two data sets (measurements) to be aligned as a function of the relative position between the measurements. An optimization algorithm can be used to determine the position which minimizes the cost function. When using a probability model, the cost function can be, for example, a negative log-likelihood function or another function which represents a measure of the quality of the alignment between the two data sets. In this case, certain tolerances can be taken into account, i.e. how exactly the two data sets must agree in order for the alignment to be determined as positive. This can be taken into account in a first check with the at least one first consistency criterion mentioned.
[0048] In particular, in the first comparison, starting from a temporary or initial posture, an optimum value for the posture, ie an adapted posture (or generally a position and orientation), is thus found, in particular if the first comparison or the first check is positive.
[0049] A second check (or second check process) is then performed. This includes a second comparison: whether the current data set is consistent with one or more previously generated data sets or existing environment maps, preferably data sets provided in a previous positioning process, according to at least one second consistency criterion. Both the first comparison and the second comparison can also be the matching or scan matching mentioned above.
[0050] In the second comparison, one or more parts of the device information (more precisely, in particular from the current positioning process, but also from previous positioning processes if necessary) are taken into account with a weighting that depends on the test result. This can be realized, for example, as an item (for example a penalty item) in a cost function (in the case of the optimization mentioned within the framework of the comparison), which item can be selected depending on the test result. As an alternative or in addition, in the second comparison, a current data set with a temporary position and orientation or with a position and orientation obtained in the first comparison can be used depending on the test result. In particular, when the first comparison is positive, the position and orientation obtained in the first comparison can be used, because it is then particularly accurate. Conversely, when the first comparison is negative, it is advantageous to use the temporary position and orientation, because, for example, no better posture was found in the first comparison.
[0051] The first comparison and the second comparison can be performed in the same manner, but if necessary with changed weights, for example as the terms of the cost function, or by the gesture on which the comparison is based. In this way, the current reliability of the device information, which is taken into account more or less, can always be determined as accurately and well as possible. In the first comparison, for example, standard weights can be used for the device information, and the weights in the second comparison can then differ from these standard weights if necessary.
[0052] When the second comparison is positive, a data set with the position and orientation obtained in the framework of the second comparison is provided for positioning and / or navigating the mobile device. Based on the weights in the second comparison, the position and orientation obtained in the optimization in the framework of the second comparison can be different from the position and orientation in the first comparison.
[0053] By means of these two comparisons, it is now possible to determine in the first comparison only the parameters for the second comparison, which is then the actual comparison (or scan matching), and then to continue to use this actual comparison as before. This allows for a more efficient and more precise positioning.
[0054] In one embodiment, the test result includes the first test result if the first comparison is negative. In this case, a first weight is used as a weight in the first test result, according to which at least a part of the device information is weighted more highly than in the first comparison. In order to be able to keep deviations of the position and orientation from the device information as low as possible, the deviations are then “penalized” more severely. This is advantageous, for example, when the environmental sensor data are not very accurate; this is indicated by a negative first comparison. The device information can then be more reliable.
[0055] In one embodiment, when the first comparison is positive and when the first comparison is also underdetermined, the test result includes the second test result. "Underdetermined comparison" should be understood in particular that the surrounding environment (environmental sensor data, i.e., data set originates from the surrounding environment) is featureless or has fewer features. So, for example, it can be related to a long corridor. In such a case, it can be so that all degrees of freedom cannot be reliably estimated. In the case of a laser radar sensor and a corridor, for example, the spacing and orientation relative to the wall of the corridor are easily determined, but it is difficult or impossible to estimate the movement in the direction of the corridor. In order to identify such a situation, for example, the eigenvalues of a submatrix can be considered, which relate to the position of the Hessian matrix of the optimization result from the first comparison. By comparing the ratio of the two eigenvalues with a threshold value, it can be determined whether the uncertainty along one direction is significantly greater than the uncertainty along another direction. Using this method, it is possible to determine the underdetermined situation in general and not only for a special corridor situation. So using this scheme, for example, it is also possible to identify the following situation, in which the mobile device is close to the wall on one side and there is a large empty space on the other side. As an alternative to this, however, it is also possible to use detectors or algorithms which explicitly recognize the shape of the corridor in the sensor data.
[0056] In this case, a second weight is used as a weight in the second check result, whereby at least a portion of the device information is weighted more highly than in the first comparison. Due to the insufficient precision of the environmental sensor data, the device information can be trusted more here. The weight can also be the same in the first check result and the second check result, for example.
[0057] In one embodiment, the test result includes a third test result when the first comparison is positive, when the first comparison is not undertimed, and when the device sensor data have been determined to be unreliable. "Device sensor data (or, if necessary, device information in general) are determined to be unreliable" should be understood in particular to mean that the relevant sensor, for example, provides erroneous values or should not be used for other reasons. It is then possible, for example, to regard the values provided by the relevant sensors (e.g. wheel odometers) as unreliable due to wheel slip, and to use a backup plan with respect to a constant speed or position or another model as described above. For this purpose, the posture (temporary posture or position and orientation) determined from the device sensor data can be compared with the posture determined within the framework of the first comparison; in this case, it is then possible to check whether the two postures deviate by more than a predefined threshold value or are consistent within a predefined tolerance. If a corridor has already been found before, it can be assumed, for example, that this is due to an inadequate comparison and that the device information is more likely to be correct.
[0058] In this case, a third weight is used as weight in the third check result, according to which, in particular, at least part of the device information is weighted less than in the first comparison.
[0059] Otherwise it could be assumed that a good alignment has been performed, a good agreement has been achieved, and therefore proprioception is more likely to be the cause of the problem.
[0060] In one embodiment, the test result includes a fourth test result when the first comparison is positive, when the first comparison is not under-timed, and when the device sensor data is not determined to be unreliable. The weights used in the first comparison, i.e., for example, standard weights, are used as weights in the fourth test result. For example, it can be provided as standard weights or for the first comparison that the position and orientation are taken less into account, and it can also be provided that, for example, a provisional position and orientation is used (thus also in the second comparison).
[0061] In one embodiment, it is further provided that the data set provided in the previous positioning process (with the associated position and orientation) is added to the existing surroundings map if at least one updating criterion exists, in particular including a detected error in the (current) positioning process. In contrast to the already mentioned conventional criteria, according to which the data set is added to the surroundings map, in the case of an error, the normal data set obtained thereby is added, so that a final update is then also achieved.
[0062] Here, at least one update criterion preferably includes at least one of the following update criteria. When the second comparison is negative, in particular when the second comparison is positive in the previous positioning process. When the device sensor data is determined to be unreliable, in particular when the device sensor data is not determined to be unreliable in the previous positioning process. When the first comparison is undertimed, in particular when the first comparison is not undertimed in the previous positioning process. Here, it is related to the following situation, in which different weights can be used as explained above. It can also be included that the mobile device was parked on the ground during the previous positioning process (and was lifted earlier). This can be identified by the mentioned lifting sensor, for example. Here, it can be considered that the device sensor data is incorrect. It can also be included that, at least temporarily, no environmental sensor data is obtained or provided during the positioning process. It can also be considered that there is unreliable data here. In addition, it is also possible to use the update criteria already mentioned, such as the expiration of a specific duration, a specific distance passed, or the number of scans performed (the data set obtained).
[0063] In one embodiment, adding a data set provided in a previous positioning process to an existing environment map includes checking whether the difference between the position and orientation of the mobile device determined based on the device information in the added data set and the position and orientation of the mobile device in the data set last added to the environment map is within a predefined tolerance. This can also involve a so-called fine match. In this way, it can be checked whether there is a drift in the position and orientation from the device sensor data and whether the drift is too small to be considered an error in the incremental positioning solution. If the difference is within the tolerance, the data set can be added, otherwise, for example, it is not added.
[0064] In one specific embodiment, navigation information is also determined for the mobile device, specifically based on a provided data set having a relevant position and orientation of the mobile device.
[0065] One embodiment relates to a method for error handling during navigation of a mobile device, wherein the mobile device moves or is to move in an environment using SLAM-based positioning. In this case, the positioning process is performed repeatedly.
[0066] In this case, in the positioning process, device and environment information are respectively provided, which are at least partially detected from the device and / or the surroundings by means of sensors of the mobile device. This can be the device sensor data and environment sensor data already mentioned, but also data obtained based on a model if necessary. A data set is then generated, more precisely based on the device and environment information. The data set is provided for positioning and / or navigating the mobile device. In particular, all further processing processes in the positioning process, as explained above, for example, can be included here.
[0067] However, during the positioning process, it is checked whether there is an error (or problem). Then the data set with an error in its positioning process is determined (or marked, for example) as a potentially erroneous data set, and the data set without an error in its positioning process is determined as a data set that complies with the regulations.
[0068] If a data set that complies with the regulations is then obtained after a potentially faulty data set, it is checked whether the potentially faulty data set can continue to be used. In this way, the decision on how to handle the error is then moved to the end, when the actual processing of the data set has already been completed in the individual (incremental) positioning process. If it is then determined that the error is not serious, for example, the data can nevertheless be used.
[0069] In one embodiment, the verification of the potentially erroneous data set includes the following verification: whether the compliant data set can be compared with one or more compliant data sets obtained before the potentially erroneous data set or existing environmental data sets according to at least one first consistency criterion. Figure 1 This can be achieved, for example, within the framework of a comparison or scan matching, as explained above, with the difference that one of the data sets to be compared is considered to be potentially erroneous here.
[0070] In one embodiment, a potentially erroneous data set is determined as an erroneous data set when the first check result is negative, that is to say when the data sets cannot be consistent. In this case, the data set is then no longer used further.
[0071] In one embodiment, the potentially erroneous data set is determined as a conforming data set when the first check result is positive and when the potentially erroneous data set is consistent with a data set obtained immediately before the potentially erroneous data set according to at least one second consistency criterion. This means that the comparison carried out within the framework of incremental positioning is positive, i.e. the error is at best minor or has little or no effect. The data set can then continue to be used without the need to take further measures.
[0072] Preferably, a potentially erroneous data set is also determined as an erroneous data set if the potentially erroneous data set does not correspond to a data set acquired immediately before the potentially erroneous data set according to at least one second consistency criterion. The error is then so great that even if the comparison between the correct data sets is successful, the comparison in the incremental positioning is unsuccessful, so that the data should nevertheless no longer be used.
[0073] In one embodiment, it is further provided that a new environment map is generated based on a data set that complies with the specification, which is obtained after the data set is determined to be faulty. In particular, however, the faulty data set is not used here. This new environment map can then be combined with the existing environment map, for example. In the case where the first check result is positive, but the data set is nevertheless ultimately faulty, these environment maps can also be combined immediately, since a sufficiently accurate comparison is already present.
[0074] In one embodiment, it is further provided that a conforming data set is added to the existing map of the surroundings when at least one update criterion exists. This can be one of the following cases: a data set obtained after the conforming data set has been determined to be erroneous. The mobile device has travelled an assigned distance since the conforming data set was last added to the existing map of the surroundings: a predetermined time period has elapsed since the conforming data set was last added to the existing map of the surroundings. A predetermined number of conforming data sets have been obtained since the conforming data set was last added to the existing map of the surroundings. Thus, in particular with the first-mentioned criterion, it is possible to use a data set that still conforms to the regulations to update the map before, if necessary, generating a new map.
[0075] In one embodiment, an error is determined to exist during the positioning process when at least one of the following error criteria is met: sensor data of a sensor of the mobile device is determined to be erroneous, for example because the sensor provides different data from other sensors or no longer provides data at all. An error can also be considered to exist when it is determined based on the sensor data that the mobile device has been lifted. This can be achieved, for example, based on the mentioned lifting sensor, or when using an inertial measurement unit. An error can also be considered to exist when it is determined based on the sensor data that the mobile device is moving, in particular horizontally, due to external forces. This can be determined, for example, using an inertial measurement unit, and if necessary also using a wheel odometer. An error can also be identified if the difference between the position and / or orientation of the mobile device determined based on the device sensor data in the added compliant data set and the compliant data set last added to the environment map is not within a predetermined tolerance. This means that the so-called fine matching mentioned was unsuccessful.
[0076] In one specific embodiment, navigation information for the mobile device is also determined, specifically based on the provided data set.
[0077] The system according to the invention or a computing unit for data processing, for example a controller or control unit of a mobile device or a server or another computer, is provided, in particular in terms of programming, for carrying out the method according to the invention, for example in one of the described embodiments.
[0078] The invention also relates to a mobile device having such a system for data processing or being configured to obtain navigation information determined as described above. The mobile device preferably also has a drive system and a control or regulating unit for moving the mobile device in accordance with the navigation information. The mobile device is preferably also configured to perform processing and can in particular be a mobile device as described above, i.e., for example, a cleaning robot or a lawnmower robot.
[0079] The implementation of the method according to the present invention in the form of a computer program or computer program product with a program code for executing all method steps is also advantageous, because this causes particularly low costs, especially when the controller implemented is also used for other tasks and therefore originally exists. Finally, a machine-readable storage medium is provided, which has a computer program as described above stored thereon. Suitable storage media or data carriers for providing computer programs are especially magnetic, optical and electrical memories, such as hard disks, flash memory, EEPROM, DVD, etc. It is also feasible to download programs via computer networks (Internet, intranet, etc.). Here, such downloading can be realized wired or wired or wirelessly (e.g., via WLAN-networks, 3G-, 4G-, 5G- or 6G-connections, etc.). BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Further advantages and embodiments of the invention are apparent from the description and the drawings.
[0081] The present invention is schematically shown according to an embodiment in the drawings and is described below with reference to the drawings. In which:
[0082] Figure 1 A mobile device with a surrounding environment is schematically shown for explaining the present invention,
[0083] Figure 2 The process for explaining the method of the present invention is schematically shown.
[0084] Figure 3 Schematically shows a further process for explaining the method of the present invention,
[0085] Figure 4 Schematically shows a further process for explaining the method of the present invention,
[0086] Figure 5 A further sequence is shown schematically for explaining the method of the invention. DETAILED DESCRIPTION
[0087] exist Figure 1 , a mobile device 100 in a surrounding environment 120, in particular in a working area, is shown schematically and by way of example for the purpose of explaining the present invention. The mobile device 100 is, by way of example, a vacuum cleaner robot having a control or regulating unit 102 and a drive unit 104 (with wheels) for moving the vacuum cleaner robot 100, for example, along a movement path 130. In addition, the vacuum cleaner robot 100 has, by way of example, a sensor 106 with a detection range, which is configured as a lidar sensor. For better illustration, the detection range is selected relatively small here; however, in practice the detection range can also be up to 360° (but, for example, at least 180° or at least 270°). The surrounding environment 120 can be detected by means of the sensor 106, that is, environmental sensor data or generally environmental information can therefore be generated.
[0088] In addition, the robot vacuum cleaner 100 has, by way of example, a sensor 107 configured as an inertial measurement unit and a sensor 108 configured as a wheel odometer. The wheel odometer can be assigned in particular to the two wheels of the drive unit 104, but this is not shown here for the sake of simplicity. Information about the position and / or movement of the robot vacuum cleaner 100 can be detected by means of the sensors 107 and 108, that is, device sensor data or generally device information can be generated.
[0089] Furthermore, the vacuum cleaner robot 100 has a computing unit 103, for example a controller, by means of which data can be exchanged with a higher-level system 110, for example via the indicated radio connection. In the system 110, for example, a movement path (or navigation information in general) can be determined, which is then transmitted to the system 103 in the vacuum cleaner robot 100, and the vacuum cleaner robot is then supposed to follow the movement path. However, it can also be provided that the movement path (or navigation information in general) is determined in the system 103 itself or is obtained there in another way. However, instead of the movement path or navigation information, the system 103 can also obtain control information, which is determined according to the movement path or navigation information, and the control or regulating unit 102 can move the vacuum cleaner robot 100 via the drive unit 104 according to this control information in order to follow the movement path. The movement path 130 is only indicated here by way of example.
[0090] The vacuum cleaner robot 100 is intended to move or navigate independently in the surroundings 120, for example, and in this case, for example, clean the floor. In addition, a plurality of different objects or obstacles, namely a wall 140 and a cabinet 142, are shown in the surroundings by way of example.
[0091] Although the invention is explained here and below in particular using a robot vacuum cleaner as an example, it is equally applicable to other mobile devices, such as, for example, a lawnmower robot or other autonomously driven vehicles. Instead of a lidar sensor, for example, a camera or a radar sensor can also be provided. In addition, a robot vacuum cleaner or a mobile device in general can also have other sensors, such as, for example, the lifting sensor already mentioned.
[0092] exist Figure 2 The method flow is schematically shown in the figure for explaining different embodiments for the sake of explanation. The method is generally used for mobile devices, such as for example according to Figure 1 The vacuum cleaner robot is navigated, which can also include determining the position and orientation of the mobile device and its positioning. Error handling can also be implemented here. The method described below includes different personalized aspects or implementation methods that can also be used individually.
[0093] In step 200, device and environment information is first provided. This device and environment information can include, in particular, environment sensor data 201, such as a lidar sensor, as well as device sensor data 202 of an inertial measurement unit and device sensor data 203 of a wheel odometer. This device and environment information can also include device information 204, such as, for example, a target speed of a vacuum cleaner robot. Likewise, data or information 205, such as a lifting sensor, can be provided (if necessary also as part of the device information). In general, all relevant information and sensor data can be provided in step 200.
[0094] It can be seen here that the device information originates from different types of sources, namely, in the exemplary embodiment shown, from an inertial measurement unit, a wheel odometer and a target speed.
[0095] When providing this information, and optionally also when collecting sensor data and / or other information, it is possible to check whether an error F is possibly present, ie, for example, a sensor is not providing data.
[0096] Based on this device information or corresponding data, the position and orientation of the mobile device for localizing the mobile device is determined in step 212. For example, the position and orientation 213 (hereinafter also referred to as attitude) can be determined relative to the last determined attitude.
[0097] In this case, for example, a plurality of partial aspects of the posture are each assigned a source set consisting of different types of sources. The partial aspect 213.1, which includes, for example, the position along the horizontal direction of movement, is assigned a source set 214.1 having the following predefined hierarchy: wheel odometer, target speed, constant speed model and constant position model.
[0098] Subaspects 213 . 2 include, for example, a yaw angle, to which a source set 214 . 2 having the following predefined hierarchy is assigned: inertial measurement unit, wheel odometer, target speed, constant speed model, and constant position model.
[0099] Partial aspects 213 . 3 include, for example, a pitch angle and / or a roll angle, to which a source set 214 . 3 having the following predefined hierarchy is assigned: inertial measurement unit, constant velocity, and constant position.
[0100] However, before step 212, it is checked in step 210 whether different types of sources are available for the corresponding source set. A source is unavailable, for example, if no data is or has been obtained from it for at least one predefined time period and / or an error is detected, for which it is predefined that the source should be unavailable; the latter can include, for example, that a lift sensor indicates that the mobile device is lifted, whereby the wheel odometer can be considered unavailable.
[0101] It can also happen, for example, that the data corresponding to each other obtained by the inertial measurement unit from the wheel odometer deviate from each other by more than a predetermined value. It can then be determined that the wheel odometer or a source including the wheel odometer is not available. In general, it can be determined in step 211 for each source whether the source is available.
[0102] In steps 210 , 211 , 212 , it can also be checked whether an error F may be present, ie, for example, a sensor does not provide data. An error can also be regarded as the case where a sensor or another source is determined to be unavailable.
[0103] In step 212, the partial aspects are determined individually, more precisely based on the source with the highest priority in the corresponding set, so that the position and orientation (attitude) are then obtained overall. It should be mentioned that six different degrees of freedom do not have to be determined here, so that depending on the situation, for example, a vertical position may not be required. Based on the position and orientation of the mobile device, navigation information for the mobile device can then be determined in the further process.
[0104] It should be mentioned that, for example, not only position and orientation can be processed separately, but also a distinction can be made between the yaw angle (which can be determined with the aid of the wheel odometer and the target speed, but can only be measured relative to the starting position) and the roll angle / tilt angle (which can, for example, only be measured with an inertial measurement unit, but is measured in an absolute coordinate system relative to the gravity vector).
[0105] As an alternative, as mentioned, for example, a constant velocity model or a constant position model can be used, in which the mobile device is assumed to move at a constant velocity and rotate at a constant angular rate or to be motionless and not rotated at all. Constant velocity is not applicable, for example, in the case of the first time step, because the velocity cannot yet be estimated. In addition, the output of the constant velocity model should be compared with the maximum velocity that the corresponding mobile device can reach, and should be discarded when this velocity is clearly exceeded, because an erroneous velocity estimate can be inferred. On the contrary, the constant position model is always used as a last resort.
[0106] To calculate the pose or trajectory (pose relative to the last pose), the start and end time points can take into account the time stamps of the time interval of interest and all time stamps within the interval associated with the sensor measurement. It is not necessary to assume that all sensors have the same scanning rate or perform their measurements synchronously. The data of all sensors can therefore be interpolated for all time stamps based on adjacent measurements, for example by linear interpolation or spline-based interpolation. In this way, a trajectory with high time resolution can be obtained.
[0107] If a lidar sensor is used to detect the environmental sensor data, then this environmental sensor data (or lidar scan) can first be preprocessed. One of the problems of rotating lidar sensors can be that not all points of the lidar scan are recorded at the same time, which leads to distortions of the scan. Therefore, the trajectory from the previous step can be used to calculate the positions of all scan points in such a way that they are recorded at the same point in time, for example at the beginning or end of the scan interval (correction).
[0108] If the provided trajectory cannot be used for the correction calculation (for example because of regression to a constant position), then for example a modified scan matcher can be used instead, which performs the correction during the scan matching.
[0109] In addition to the step of correcting the display, dynamic objects can also be identified in the measurement or processing of environmental sensor data at this stage. When using a lidar sensor, an algorithm for identifying changes can be used. In order to distinguish between static objects and dynamic objects, the current measurement can be compared with an earlier measurement (multiple measurements) or a local map (environmental map). By removing dynamic objects, such as animals, people, moving doors, etc., their influence on the tracking algorithm can be eliminated or at least reduced. If the posture from the previous step is not accurate enough for this due to previously identified problems, the identification of dynamic objects can be omitted in order to avoid erroneously removing the static part of the measurement.
[0110] When using a two-dimensional environmental sensor, such as a 2D-lidar, it can also be advantageous to perform leveling. The roll angle / tilt angle of the mobile device can be used during leveling in order to rotate the measurement (e.g., point cloud) so that it is parallel to the global coordinate system oriented to gravity. This can also be achieved using a 2D-matching algorithm even when the mobile device is tilted, for example, due to road unevenness. In addition, it can also be advantageous that the observed points are filtered according to their height. Observation of the ground or ceiling in a 2D-SLAM system is problematic because it leads to unstable measurements that move according to the angle of the sensor and should therefore not be used in the matching algorithm. In order to avoid such problems, points on or below the ground can be filtered according to the height of the point after leveling. It is feasible to do it more restrictively and only allow points in a vertical range of a few centimeters near the mounting height of the sensor. This avoids the unintentional detection of structures that are at another height and are usually invisible to the sensor.
[0111] Next, a data set should be determined based on the determined position and orientation 213 and the environmental sensor data 201, which is used for further positioning and / or navigation of the mobile device, in particular, also for expanding the environmental map if necessary. For this purpose, the position and orientation 213 should be provided as a temporary position and orientation 213 ', and the environmental sensor data 201 should be provided as the current data set 201 '.
[0112] In step 221, a first check is performed, which includes a first comparison: whether the current data set 201' with the temporary position and orientation 213' (that is, when the environmental sensor data are associated with the temporary posture) can be consistent with one or more previously generated data sets or existing environmental maps, for example, data sets provided in a previous positioning process, according to at least one first consistency criterion. In this case, a "localization process" is to be understood as a process in which a data set with a posture is determined, which data set is then used for further processing, for example, also added to the environmental map. A check result 225 is obtained in this case.
[0113] The first check or first comparison is, for example, an optimization method in which, starting from a current data set with a temporary posture, an attempt is made to make the current data set consistent with the aforementioned previous data set. For example, an adapted posture 213 "" can be obtained. This is, for example, so-called scan matching.
[0114] Next, a second check is performed in step 222, more precisely taking into account the check result 225. The second comparison is hereby performed to determine whether the current data set 201' is consistent with one or more previously generated data sets or existing environment maps, preferably data sets provided in a previous positioning process, according to at least one second consistency criterion.
[0115] However, in the second comparison, one or more parts of the device information are now taken into account using a weight 226 that depends on the test result 225. As an alternative or in addition, the current data set 201' is used according to the test result 225, which has a temporary position and orientation (posture) 213' or has a position and orientation 213' obtained in the first comparison. By way of example, four possible test results 225.1, 225.2, 225.3, 225.4 should be considered within the framework of the embodiment, which test results should be explained in more detail below.
[0116] The second check or second comparison is also, for example, an optimization method in which, starting from a current data set with a temporary or first comparison posture, an attempt is made to make the current data set consistent with the aforementioned previous data set. Then, for example, a re-adapted posture 213 "" can be obtained. This is also, for example, a so-called scan matching, in particular a second scan matching.
[0117] If the second comparison is positive, in step 230 the current data set 201 ′ with the pose 213 ″′ (which was obtained in the second comparison) is provided for positioning and / or navigating the mobile device.
[0118] In the context of these two checking or comparison processes, errors F (marked here by way of example at step 200 ) may also occur.
[0119] In step 340 , navigation information for the mobile device can then be determined, for example, in order to be able to navigate or move the mobile device.
[0120] In order to track the movement of the mobile device, the mentioned scan matching can be used. Matching algorithms can be used here, for example scan matching algorithms (such as Iterative Closest Point (ICP), Normal Distributions Transform (NDT), Correlation Scan Matching) or visual feature matching algorithms. These matching algorithms work under the consideration of a cost function, which compares two measurements (data sets) to be compared according to the relative position between the measurements. Depending on the use scenario, these measurements can be point clouds (in the case of LiDAR sensors), camera images, radar measurements, etc.
[0121] For example, an optimization algorithm is used to determine the position that minimizes the cost function. The cost function can be a negative log-likelihood function when using a probability model, or any other function that shows a measure for the quality of the alignment between two measurements. The optimization algorithm is usually started with a first estimate, which is then improved over time, for example by means of Newton's iteration method. Since the cost function will not be a convex function, the resulting posture depends on a good choice of the starting posture in terms of convergence. For this purpose, in principle, a posture determined based on the device sensor data (temporary posture) can be used to increase the probability that the algorithm converges to a global optimum and does not converge to a local optimum that is far from the actual value.
[0122] In order to incorporate a priori knowledge about the relative transformation (i.e. between two poses), the cost function can be extended by adding a term that describes the deviation from the previous pose. By introducing the additional term, the form of the cost function changes and the minimum of the cost function is shifted in the direction of the previous position, since deviations from the previous position are penalized at a higher cost even if the consistency itself is better. In this way, weights can then be taken into account during scan matching in order to, for example, allow more or less device sensor data to be included.
[0123] In such a solution, the main problem lies in the choice of weights for the device sensor data or the terms for the attitude determined thereby. If the weight is too large, the result depends only on the attitude and does not take into account information from the measurement (e.g. environmental sensor data of a lidar sensor), and if the weight is too small, the attitude is ignored. It has now been shown that a good weight for the attitude depends on the respective situation and can be adapted accordingly. When the attitude originates from a very reliable sensor that is not currently experiencing problems, its weight should be chosen to be larger. Conversely, when the attitude originates from an inaccurate sensor or the sensor is currently experiencing problems, the weight should be chosen to be significantly lower.
[0124] In order to find out a good choice of parameters for the matching algorithm, it is proposed to perform two scan matches. The first comparison is used to understand the current situation, and its results are used to adapt the parameters of the second comparison. The second comparison is then carried out with the adapted parameters in order to obtain the actual result. A particularly relevant situation that can be identified based on the first comparison is the situation of an environment without features, such as, for example, a corridor, in which the comparison is underdetermined, that is, not all degrees of freedom can be reliably estimated. In the case of a lidar sensor and a corridor, for example, the distance to the wall of the corridor and the orientation are easily estimated, but movements in the direction of the corridor are difficult or impossible to estimate.
[0125] In order to identify such a situation, for example, the eigenvalues of a submatrix can be taken into account, which submatrix relates to the position of the Hessian matrix of the optimization result from the first comparison. By comparing the ratio of two eigenvalues with a threshold value, it is determined whether the uncertainty in one direction is much greater than the uncertainty in the other direction. With this method, not only specific corridor situations are identified, but also other underdetermined situations are identified. For example, this approach also identifies the following situation: the mobile device is close to a wall on one side and there is a large empty space on the other side. Alternatively, a detector can also be considered, which explicitly recognizes the shape of the corridor in the sensor data.
[0126] Another problem that can be identified based on the first comparison is that the determination of the posture, i.e. the device sensor data, is unreliable, for example due to wheel slip that is not detected by the wheel slip-runout detector (e.g. by a wheel odometer), falling back to a constant speed / position, or due to other reasons. For this purpose, the posture determined based on the device sensor data is compared with the result of the first comparison, and it is checked whether the two postures deviate from each other by more than a predefined threshold. When a corridor was previously identified, it can be assumed that this should be attributed to the problem of insufficient comparison, and the posture can be considered to be correct with a greater probability. Otherwise, it can be assumed that a good comparison has been performed, and therefore the device sensor data is more likely to be the cause of the problem. In the latter case, the device sensor data can be considered unreliable.
[0127] This should be explained below based on Figure 3 It is necessary to explain in more detail when the test result can be used. Figure 2 Step 220 , in which a current data set 201 ′ with a temporary pose 213 ′ is provided.
[0128] A first check 221 or a first comparison is then performed. Standard weights can be used, for example. In step 301, it is checked whether the first comparison is positive. If the first comparison is negative (N), the test result should include, for example, a first test result 225.1. A first weight is then used as a weight, whereby, for example, at least a portion of the device information is weighted more highly than in the first comparison.
[0129] If the first comparison is positive (Y), it can be checked, for example, in step 302 whether the first comparison is underdetermined, i.e., whether, for example, a corridor has been identified. If this is the case (Y), the test result should, for example, include the second test result 225.2. The second weight is then used as a weight, according to which, for example, at least a portion of the device information is weighted compared to the first comparison. The first weight and the second weight can also be identical, for example.
[0130] If it is determined in step 302 that the first comparison is not undertimed (N), it can be checked in step 303 whether the device sensor data are determined to be unreliable. If so (Y), the test result can, for example, include a third test result 225.3. A third weight is then used as a weight, whereby at least a portion of the device information is weighted less than in the first comparison, for example. The device sensor data are then determined to be unreliable, for example, if the temporary position and orientation do not correspond to the position and orientation obtained within the framework of the first comparison within a predetermined tolerance.
[0131] If this is not the case (N) according to step 303, the test result should include, for example, fourth test result 225.4. In this case, for example, the weight used in the first comparison, ie, the standard weight, can be used as the weight.
[0132] A second check with a second comparison is then carried out, step 222 , wherein the first weighting, the second weighting, the third weighting or the fourth weighting is used, depending on the situation.
[0133] The data set 201 with the pose 213 ''' provided in step 230 can then be added to the existing environment map 235, step 232; in this case, the data set 201 with the pose 213 ''' can also be referred to as a so-called key frame. This can be done, for example, only if an update criterion 231 exists. The update criterion can include, for example, that no more data sets are added to the environment map for a certain duration or that the mobile device has moved a certain distance since the last addition.
[0134] In the case of an error F in the preceding process, it is possible that the data set obtained here is erroneous. In this case, this can be used as an update criterion 231 so that the data set obtained in the previous process (positioning process) (wherein no errors occurred later in particular) is added to the environment map 235.
[0135] The data set obtained in the previous process (positioning process) is added to the environmental map 235 in other update criteria, and the other update criteria may also be: the second comparison is negative, especially when the second comparison was also positive in the previous positioning process; the device sensor data is determined to be unreliable, especially when the device sensor data was not determined to be unreliable in the previous positioning process; when the first comparison is undertimed, especially when the first comparison was not undertimed in the previous positioning process; the mobile device was parked on the ground during the previous positioning process; or at least temporarily no environmental sensor data was obtained or provided during the positioning process.
[0136] The addition of a data set provided in a previous positioning process to an existing environment map or, in general, the addition of a data set to an environment map can in particular include a special check, in which it is checked whether the difference between the position and orientation (attitude) of the mobile device in the added data set determined based on the device information and the position and orientation (attitude) of the mobile device in the data set last added to the environment map lies within a predefined tolerance.
[0137] This is Figure 4 . For this purpose, multiple positioning processes n to n+4 are shown by way of example, which are respectively based on environmental sensor data and the data sets obtained therein. In addition, the posture can be determined separately as explained above based on the assigned device information or device sensor data. This is typically achieved by always referring to the last posture determined. It is then possible to determine the posture in positioning process n, for example, using two scan matches according to the aforementioned process. For positioning process n+1, the posture is determined relative to the posture of positioning process n based on the current device information. This is indicated by arrow 401 (dashed line). Using the posture determined in this way, positioning process n+1 can be performed, for example, using two scan matches. However, as mentioned above, the scan match itself is performed using the current data set with reference to the data set last added to the environmental map. This is indicated by arrow 402 (solid line).
[0138] When the data set of positioning process n is, for example, the last data set added, scan matching is also performed for positioning processes n+1, n+2, and n+3 relative to the data set of positioning process n, but the posture is always determined relative to the previous positioning process, that is, for example, in positioning process n+3 relative to positioning process n+2.
[0139] In this way, small deviations in the attitude can be accumulated. For this purpose, it is now proposed that when a data set is to be added to the environment map, the attitude is also determined relative to the attitude of the last added data set. When, for example, the data set of positioning process n+4 is to be added, the attitude is also determined relative to positioning process n based on the device information for positioning process n+4. If there is an excessively large deviation here, this can be regarded as an error, so that, for example, the current data set of positioning process n+4 is not used.
[0140] As already mentioned, in Figure 2 , a localization process is described by way of example, which includes in particular the provision of the required data in step 200 up to the provision of a data set 201 ', which is optionally added to the surroundings map, in step 230. As also described, during the localization process, it is possible to check in different steps whether an error F may be present. This can be done not only for a single localization process, but also for every localization process.
[0141] exist Figure 5 This is shown again in a very simplified manner in FIG. 5 for a positioning process 506. Here, it is schematically shown that there are a plurality of positioning processes in the positioning process, which are denoted by way of example by 501, 502, 503.
[0142] In this case, a check is performed in step 510 to determine whether an error F has occurred. If an error F has now occurred, the data set in whose positioning process an error has occurred is determined in step 512 as a potentially erroneous data set 513. This can be the case, for example, for the data set in the positioning process 502. The data set in whose positioning process no error has occurred is treated as a correct data set 514 in step 514. This can be the case, for example, for the data sets in the positioning processes 501 and 503.
[0143] In this case, different error criteria can be set, in the presence of which it is assumed or determined that an error is present in the positioning process. For example, sensor data of sensors of the mobile device can then be determined to be erroneous. It can be determined based on the sensor data that the mobile device has been lifted (in this regard, see the lifting sensors already mentioned). It can be determined based on the sensor data that the mobile device has been moved, in particular horizontally, by an external force. The difference between the position and / or orientation of the mobile device in the added compliant data set determined based on the device sensor data and the compliant data set last added to the environment map may not be within predefined tolerances (e.g. with respect to the position and orientation of the mobile device in the added compliant data set). Figure 4 as explained in more detail).
[0144] If a data set that complies with the specification is obtained after a potentially erroneous data set, then for the mentioned case, i.e. in the positioning process 503 or at the end of the positioning process, step 520 checks whether the potentially erroneous data set can continue to be used. In this way, although it is then noted that an error has occurred or has occurred, further processing of the error is postponed to the end.
[0145] The checking of the potentially erroneous data set in step 520 includes, for example, checking whether the compliant data set 515 (i.e., the data set from the current positioning process 503) can be compared with one or more compliant data sets obtained before the potentially erroneous data set or the existing environmental data set according to at least one first consistency criterion. Figure 1 In this case, the data set from the localization process 501 is preferably checked, which is carried out before the localization process 502 with the potentially erroneous data set. For example, scan matching can be performed here.
[0146] Here, a first test result 521 is obtained, which indicates whether the two data sets can be consistent. When the first test result 521 is negative (N), the potentially erroneous data set 513 can be determined as a (finally) erroneous data set, step 522.
[0147] When the first test result 521 is positive (Y), it can be checked whether the potentially erroneous data set is consistent with the data set obtained directly before the potentially erroneous data set according to at least one second consistency criterion. When this is the case (Y), the potentially erroneous data set can be determined as a compliant data set, step 524. Conversely, when this is not the case (N), the potentially erroneous data set can be determined as an erroneous data set, step 525.
[0148] In the case according to step 524, the data set can then continue to be used as usual and, for example, as already described above (see Figure 2 , step 232) adds it to the existing environment map, here represented by step 530. In other words, the comparison is successful here, and at most there are small errors in the pose estimation. This means that no (worth mentioning) errors ultimately occurred and errors were successfully avoided during incremental positioning. It can be considered that the pose estimation and the data assigned to the potentially erroneous data set are valid.
[0149] On the contrary, for the situation according to step 522 or 525, it can be provided that the erroneous data set is not used further, but a new environment map can be generated. This new environment map can then be combined with the existing environment map, for example. For the situation according to step 525, i.e. the first test result is positive, it can be considered that the error is relatively small, so that these environment maps can also be combined immediately because there is a sufficiently accurate comparison. The comparison is therefore also successful here, but is accompanied by a larger error in the attitude estimation: an error occurs during the incremental positioning, but the comparison can compare the last data set with the environment map for error recognition. This means that the error has caused a relatively small error in the case of incremental positioning in the greatest possible manner.
[0150] In the case according to step 522, i.e. the first check result is negative, these environmental maps can be combined later, if, for example, there is more information to check whether or how these environmental maps can still be combined. The comparison thus fails here. An error occurred during the incremental positioning, and the error detection comparison was unable to compare the latest data with the environmental map. This occurs, for example, in the following cases: the error is very serious or the mobile device has detected a new part of the surroundings. The latter means that the map at the location where the mobile device is located does not contain data that meets the regulations.
[0151] A recovery method can then be set up, which ensures that the SLAM state is transferred to a valid state after an error. To this end, the following steps can be performed. The problematic data set is deleted from the previous positioning process, and the last (compliant) data set is moved to a new map. Two non-intersecting maps are obtained, which are consistent. The SLAM system reports the position of the mobile system about the new map. This means that other map data is no longer available (temporarily), thereby ensuring that other modules of the system respond accordingly. Optionally, it can be set that if the comparison of the error recognition is successful, the two maps are immediately combined (see step 525), wherein the posture estimation from the comparison of the error recognition is used. The advantage of a clear combination is that the combination can be restored when it is later confirmed to be an error. Because it is uncertain what the exact trajectory from the last compliant data set to the current data set is, a new environmental map is created. The two maps can be recombined after the recovery method.
[0152] As already mentioned above, a conforming data set does not always have to be added to an existing map of the surroundings; rather, this can only be done if at least one updating criterion exists. This updating criterion can include, for example, that a data set obtained after the conforming data set has been determined to be erroneous. The last conforming data set obtained is then added. This updating criterion can also include that, since the conforming data set was last added to the existing map of the surroundings, the mobile device has traveled an assigned distance or a predetermined time period has elapsed, for which a predetermined number of conforming data sets have been obtained.
[0153] The proposed approach thus enables different innovative approaches, for example, a comprehensive approach, that is, one that can handle many different types of problems or errors that can occur at different stages of the overall algorithm and involve different sensor settings (as opposed to a specialized approach that can only handle a single problem).
[0154] Problems or errors can be forwarded to a later processing stage and then taken into account there. This makes it possible to avoid premature and inadequate decisions, in particular in situations where the correct response to the problem becomes clear only later (in contrast to conventional approaches, in which a processing stage can only silently output potentially erroneous results or terminate a process with a fatal error). Nevertheless, it can also be provided, for example, that an attempt is made to continue a regular positioning process or navigation despite a problem or error.
[0155] In addition, global map data can be used. The final decision on whether the incremental positioning (localization process) has failed is based on the global map data (environmental map). This final decision is postponed until there is problem-free data (data set that complies with the regulations) available for incremental positioning, and the erroneous data is removed from the map or not included at all. This mechanism allows SLAM to continue in extreme cases and allows riskier error avoidance scenarios, in which the common scenario with serious errors is interrupted or the risk of damaging the environmental map is risked.
Claims
1. A method for positioning a mobile device (100), in particular a vehicle or a robot that moves at least partially automatically, in particular a cleaning robot or a weeding robot, wherein: The mobile device moves or should move in a surrounding environment (120) using SLAM-based positioning, The method comprises, during positioning: Providing (200) device information (202, 203, 204), in particular device sensor data, detected by means of a sensor of the mobile device, wherein the device information comprises information about the position and / or movement of the mobile device, and providing environmental sensor data (201) detected by means of a sensor of the mobile device, wherein the environmental sensor data comprises information about the surroundings of the mobile device, in order to obtain a current data set (201'), determining (212) a temporary position and orientation (213') of the mobile device based on the device information, a first check (221), comprising a first comparison of whether the current data set (201') with the provisional position and orientation (213') is consistent with one or more previously generated data sets or existing environment maps, preferably data sets provided in a previous positioning process, according to at least one first consistency criterion, in order to obtain a check result (225), Taking into account the check result (225), a second check (222) is performed, which includes a second comparison: whether the current data set is consistent with one or more previously generated data sets or existing environment maps, preferably data sets provided in a previous positioning process, according to at least one second consistency criterion, wherein, in the second comparison, one or more parts of the device information are taken into account with a weight (226) that depends on the test result (225), and / or wherein, in the second comparison, depending on the test result, a current data set (201`) with a temporary position and orientation (213`) or with a position and orientation (213``) obtained in the first comparison is used, and When the second comparison is positive, a current data set with the position and orientation (213"``) obtained in the framework of the second comparison is provided (230) for positioning and / or navigating the mobile device.
2. The method according to claim 1, wherein: When the first comparison is negative, the test results include a first test result (225.1), and In this case, a first weight (226.1) is used as weight in the first check result, according to which at least part of the device information is weighted more highly than in the first comparison.
3. The method according to claim 1 or 2, wherein: When the first comparison is positive and when the first comparison is under-determined, the test result includes a second test result (225.2), and In this case, a second weight (226.2) is used as a weight in the second check result, according to which at least a part of the device information is weighted in particular compared to the first comparison.
4. A method according to any one of the preceding claims, wherein: When the first comparison is positive, when the first comparison is not under-timed, and when the device sensor data has been determined to be unreliable, the test result includes a third test result (225.3), and In this case, a third weight (226.3) is used as weight in the third check result, according to which at least part of the device information is weighted less than in the first comparison.
5. The method according to claim 4, wherein: The device sensor data are determined to be unreliable when the provisional position and orientation do not correspond to the position and orientation obtained in the framework of the first comparison within a predefined tolerance.
6. A method according to any one of the preceding claims, wherein: When the first comparison is positive, when the first comparison is not under-timed, and when the device sensor data is not determined to be unreliable, the test result includes a fourth test result (225.4), and The weight (226.4) used in the first comparison is used as the weight in the fourth test result.
7. The method according to any one of the preceding claims, further comprising: The data set provided in a previous positioning process is added (232) to the existing environment map (235) when at least one updating criterion (231) exists, which includes in particular detected errors (F) in the positioning process.
8. The method according to claim 7, wherein: The at least one update criterion includes at least one of the following update criteria: the second comparison is negative, in particular when, moreover, the second comparison was positive in a previous positioning process; the device sensor data has been determined to be unreliable, in particular when furthermore the device sensor data has not been determined to be unreliable in a previous positioning process; When the first comparison is under-timed, in particular when the first comparison was not under-timed in a previous positioning process; The mobile device has been parked on the ground during the previous positioning process; The surrounding sensor data are not acquired or provided during the positioning process at least temporarily.
9. The method according to claim 7 or 8, referring back to claim 3, wherein: Adding the data set provided in the previous positioning process to the existing environment map (235) includes: A check is made as to whether the difference between the position and orientation of the mobile device in the added dataset, which is determined based on the device information, and the position and orientation of the mobile device in the dataset last added to the environment map, lies within a predefined tolerance.
10. The method according to any one of the preceding claims, further comprising: Navigation information for the mobile device is determined based on the provided data set.
11. System for data processing, comprising means for implementing the method according to any of the preceding claims.
12. A mobile device comprising a system according to claim 11 and / or configured to obtain navigation information determined according to the method according to claim 10 and configured to navigate based on the navigation information, The mobile device preferably utilizes a control or regulating unit and a drive unit in order to move the mobile device in accordance with the navigation information.
13. The mobile device according to claim 12 is constructed as a vehicle with at least partially automated movement, in particular a personnel transport vehicle or an object transport vehicle, and / or is constructed as a robot, in particular as a household robot, such as a cleaning robot, a floor- or road-cleaning device or a lawn mowing robot, and / or is constructed as a drone.
14. A computer program comprising instructions which, when executed on a computer, cause the computer to carry out the method steps of the method according to claim 1, when the program is executed on the computer.
15. A computer-readable storage medium having stored thereon the computer program according to claim 14.
