METHOD FOR SEMANTIC SEGMENTATION OF AN IMAGE AND DEVICE

Description

[0001] The present invention relates to a method for the semantic segmentation of an image that has been captured by an environmental sensing means of a device, in particular one that moves automatically, such a device, as well as a computing system and a computer program for carrying it out. Background of the invention

[0002] Mobile processing devices such as robotic lawnmowers, vacuum cleaners, mops, or other household robots typically move within a defined environment, such as a garden or an apartment. A fundamental problem lies in determining the permissible processing area; in the case of a robotic lawnmower, for example, how far the lawn to be mowed extends, ensuring that the lawn is cut as completely as possible while preventing the robot from straying beyond the lawn boundaries, such as onto a road or similar obstacle.

[0003] The article "XU YU-SYAN ET AL: Dynamic Video Segmentation Network, ARXIV, June 1, 2018 (2018-06-01), pages 1-10" deals with the semantic segmentation of an image. The article referenced therein, "CORDTS MARIUS ET AL: The Cityscapes Dataset for Segmantic Urban Scene Understanding, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, June 27, 2016 (2016-06-27), pages 3213-3223, XP033021503", makes a connection to moving vehicles. Disclosure of the invention

[0004] According to the invention, a method for the semantic segmentation of an image, a device, a computing system, and a computer program for carrying it out, comprising the features of the independent claims, are proposed. Advantageous embodiments are the subject of the dependent claims and the following description.

[0005] The invention relates to preferably mobile devices that move in an automated manner, such as preferably a robot, e.g., a robotic lawnmower. Although the invention will be explained below primarily in relation to a robotic lawnmower, other mobile devices or robots are also conceivable, in particular household robots, such as a vacuuming and / or mopping robot, floor or street cleaning devices, at least partially automated vehicles, or even drones.

[0006] It is generally desirable for a robotic lawnmower, for example, to remain within the boundaries of the garden or lawn, as there may be areas unsuitable for optimal operation, such as ponds, mud, gravel paths, flower beds, etc. For this purpose, a wire or cable can be laid in the garden or lawn, especially along the edges or where the robotic lawnmower is intended to mow. This wire or cable can be detected by a sensor in the robot. This is also referred to as a lawn edge, where such a wire is laid. While this allows the robot's movement to be limited as desired, laying the wire or cable requires considerable effort.

[0007] Another way for a (mobile) device to recognize specific features in its environment, such as a lawn edge in the case of a robotic lawnmower, is to use environmental sensing devices or one or more sensors, such as cameras or video cameras, thermal imaging cameras, radar sensors, lidar sensors, laser rangefinders, ultrasonic sensors, inertial sensors, and / or odometers. In particular, the present invention focuses on sensing devices or sensors that enable a visual representation of the environment, preferably cameras or video cameras. However, depth information, which can be displayed graphically or as (digital) images, can also be obtained using other sensors, such as lidar or ultrasonic sensors. Such an image of the environment can then be analyzed to identify specific things or objects in the environment, such as...to recognize or identify the aforementioned lawn edge in the case of a robotic lawnmower.

[0008] Semantic segmentation is particularly relevant here. In this process, each pixel in an image is assigned to one of several classes; that is, the pixels are labeled or classified. Typically, features of the pixels are first determined or extracted, and the class is then assigned based on these features. Such features can include, for example, shape, color, context, pattern, light variation, and / or image context. For instance, a pixel could be assigned the color green, and the surrounding pixels could also all be green. This would suggest grass.

[0009] Various attributes can be used as classes. In the example of a robotic lawnmower, two classes can be used in a simple case: "lawn" and "not lawn." This means that for each pixel in the image, it is determined whether it shows lawn or not. It goes without saying that more than two classes can be used to also recognize paths, roads, buildings, vehicles, or people, and assign them to a pixel. Another possibility—similar to the "not lawn" class—is a "background" class, to which everything not assigned to another class is assigned. All these labels add more information compared to pure object recognition, which (only) identifies objects in an image.

[0010] In principle, classification does not need to be performed for every single pixel; it can also be performed for segments or parts of the image, each comprising several pixels. Such segments can be predefined or created during semantic segmentation, possibly with varying sizes within the image. For this purpose, one or more artificial neural networks or, more generally, pattern recognition methods based on artificial intelligence are preferably used, which will be discussed in more detail below.

[0011] In this way, a classified output image (a kind of map) can be generated from the input image, indicating the location of specific objects or features. In the simple example with the two classes "lawn" and "not lawn," a lawn edge can be detected or determined by a boundary between areas of the image assigned to either "lawn" or "not lawn" (the corresponding output image could, for example, be purely black and white). The output image is then used to control the device; for example, it can stop or turn before or upon reaching a lawn edge. It is also conceivable to differentiate between, for example, lawn that still needs mowing and lawn that has already been mowed.

[0012] It should be mentioned at this point that while cameras or video cameras on or in such a mobile device are preferably considered for capturing such (digital) images, within the scope of the present invention, as already mentioned, environmental representations captured with other sensors such as the aforementioned lidar or ultrasonic sensors or thermal cameras, or possibly obtained through further processing, are also to be understood as images; semantic segmentation can also be applied there.

[0013] By repeatedly or continuously performing this procedure for a current image of the environment, especially when the (mobile) device is moving, the mobile device can be operated or controlled based on the current resulting image; for example, a robotic lawnmower can be controlled in such a way that it only drives (and mows) up to a detected lawn edge, but then continues along the lawn edge, but not beyond it.

[0014] Since semantic segmentation adds more information to an image, it can be used, especially in the field of self-driving cars, to (better) recognize multiple objects and their respective classes, such as cars, people, traffic signs, road obstacles, etc., in a single image. Other autonomous robots can also use semantic segmentation to obtain more information about their environment; for example, robotic arms can use it to determine which objects to select, while drones can use it to recognize sky boundaries and obstacles.

[0015] However, this semantic segmentation typically requires high computing power. Therefore, the use of semantic segmentation in small (mobile and especially autonomous) devices or vehicles remains challenging due to the high computational demands and the complex architectures of deep neural networks required to perform the semantic segmentation quickly and accurately.

[0016] Within the scope of the present invention, it is therefore proposed to select a region, i.e., a portion of the image, and to use a higher proportion of the (available) computing power of an executing computer system in this region during semantic segmentation than for the rest of the image, i.e., the unselected region, at least with respect to a portion of the selected region within the image. In other words, the available computing power (at least insofar as it is used or can be used for semantic segmentation) is used or divided in such a way that more computing power is used per portion of the image for the selected region than for the rest. If, for example, the selected region comprises half of the image (i.e., half of the pixels present in the image), more than half of the computing power is used for the selected region, e.g., three-quarters or more.

[0017] The selected area does not necessarily have to be a contiguous area; it can also be separate sub-areas of the image.

[0018] This allows for the fast yet accurate identification and recognition of features within a selected area—that is, a region of the image (and thus the surrounding area) deemed particularly relevant with regard to certain characteristics—while simultaneously recognizing and identifying features in the rest of the image. Compared to distributing computing power evenly across the entire image, the available processing power can therefore be used more efficiently. Similarly, while maintaining the same speed and accuracy in feature recognition within the selected area, the required computing power can be reduced. This enables its use even in smaller (mobile) devices, which typically have less processing power available.

[0019] Artificial intelligence, specifically deep learning (preferably artificial neural networks), is used to perform semantic segmentation of an image and identify relevant segments, for example, within the lawn (in the case of a robotic lawnmower). This is achieved with low computational resources and high accuracy by focusing on relevant regions without losing the overall image context. This approach uses a dataset specific to the environment in which such a device or robot would operate to train a deep learning network. This network then focuses on segmenting and defining, for example, the correct lawn or grass boundaries and other relevant objects within the lawn, such as trees, houses, garden tools, people, etc. This allows for particularly quick and easy selection of the area of ​​interest within the image.

[0020] This approach differs from similar approaches used in the field of self-driving cars or related applications involving semantic segmentation. Semantic segmentation requires significant processing power to be both accurate and fast. This is also true for small devices or vehicles, especially those operating close to the ground, but their computing power is generally more limited than that of hardware used in cars. Furthermore, the image used for semantic segmentation (i.e., the input image) cannot (or at least should not) be cropped to the area of ​​interest, as the context (i.e., the rest of the image outside the selected area) is also crucial for a smaller and more accurate network (artificial neural network).The image is thus used as input, a specific area is focused on (selected) without losing the context of the complete image, while making it possible to focus difficult computational operations on such an area in order to improve both throughput and accuracy.

[0021] The proposed approach is also more robust to varying changes in the image environment, such as shifts in the camera's viewing angle. It is also robust to changes in ambient light, weather conditions, seasons, and the presence of similar objects in the garden, among other factors. A major problem with artificial neural networks for semantic segmentation is their complexity and time-consuming nature when accuracy is required; however, when speed is a requirement, accuracy inevitably decreases as the network size decreases. The proposed approach now makes it possible to achieve both accuracy by preserving the entire image and high speed by using a smaller network that focuses primarily on relevant (the selected) areas of the image.

[0022] The proposed approach is more robust because it applies more computing power to the focused or selected area without losing contextual information for the entire image. The area of ​​interest or selection would typically be the ground surface, but the network needs information about weather conditions (e.g., sunny or cloudy) to adapt to the current ground surface. For example, if it's sunny, the grass or lawn usually appears greener to the camera than if it's cloudy. If the network were distorted in terms of direction or color, errors would occur in either of these scenarios. With contextual information, however, it can be better generalized (adapted).

[0023] The proposed approach is particularly unique compared to other deep learning solutions in that it is geared towards the designed environment (e.g., garden and outdoor area) and towards devices or robots that work close to the ground (or closer than, for example, self-driving cars).

[0024] The segmented image can be used, for example, to define the lawn edge or grass boundary of lawn areas. This definition has proven accurate enough to replace conventional boundary wires. A robot operating closer to the ground must focus more on objects and boundaries in its vicinity, without losing sight of more distant objects. This approach can also be used to teach the device or robot the boundaries of a garden, for example, as it learns or codes its new boundaries. This helps it recognize the presence of grass, soil, roads, water, dirt, flowerbeds, weeds, etc., within the garden or at ground level. Segmentation can also aid in localization by providing context for the image objects and helping the robot understand their surroundings.

[0025] Most semantic segmentation methods work by analyzing the entire image and performing the same set of computational operations on each area, resulting in the same, or evenly distributed, use of computing power. This is useful in other areas due to the unpredictability of the appearance of objects in the image (e.g., in self-driving cars). However, especially with small autonomous robots, the operating area and perspective are well-defined due to the robot's dimensions, camera configuration, placement, and orientation. For example, by analyzing multiple internal datasets (e.g., through prior training), the area where, for instance, grass typically appears in the image can be averaged, and after projecting this onto the image, a region of interest can be identified and selected.

[0026] Nevertheless, the proposed approach is also suitable for other devices in which features in images of the environment are to be recognized.

[0027] The area in the image for which the higher processing power is used is selected based on the position of the environmental sensing device (e.g., the camera) relative to a plane or surface (specifically, the ground) on which the device is moving. Advantageously, the area in the image is also selected based on the device's current position within the environment, ideally in relation to the device's operating range within that environment. The process for selecting and obtaining the area of ​​interest is primarily mathematical, for example, by analyzing the camera's orientation relative to the ground plane and defining a boundary for the robot's operating range; for example, the robot should be able to detect all objects within a three-meter radius.

[0028] In the context of the present invention, an artificial neural network (or deep learning network) preferably receives an image (or image pair) after it has been preprocessed by a localization engine and performs dimensionality reduction via, for example, small or few convolutional layers of the artificial neural network, typically alongside a user-defined layout of batch normalization, pooling mechanisms, and activation functions. Features are then extracted. This can initially be done for the entire image.

[0029] Image pairs can be captured to generate depth information or to detect it within the network. This increases the information content. For example, recognizing walls is challenging because they typically have a color and minimal texture; if depth information is available, they can be more easily recognized and correctly segmented.

[0030] A localization engine is used in particular to correct the captured image, i.e., especially the distortion of the lens, and possibly also to perform color correction.

[0031] Dimensional reduction helps, for example, to reduce the number of inputs or input values ​​(for the neural network) to facilitate processing by performing convolutions and focusing on relevant features. An image with, for example, only 720 x 480 pixels and three color bits already results in 1,036,800 input values, which is usually too many for processing. Pooling achieves "down-sampling" by selecting specific values ​​to pass to subsequent layers (e.g., "max pooling" in a 2x2 matrix would select and pass the maximum value, thus reducing a 2x2 pixel size to 1x1).

[0032] Activation functions are similar to logic gates in an integrated circuit. They determine which neurons are activated at each stage of the neural network and thus decide the final output (for the grass or lawn section, for example, neurons that focus on specific grass patterns, textures, and colors would be activated). Batch normalization mitigates the problem of internal covariant shift and smooths the objective function by standardizing the inputs in each batch.

[0033] In addition, a region of interest within the image is selected or extracted in order to apply a further set of larger convolutions or a higher number of convolutions, possibly alongside other operations, to that selected area. This allows the artificial neural network to consider the entire image and focus heavier convolution kernels (i.e., those with a higher weight) on the region of interest or the selected area.

[0034] In other words, more calculations are performed in the selected area than in the rest of the image. This can be achieved, for example, by first performing feature recognition on the entire image, and then performing additional calculations on the selected area, resulting in more accurate or better feature recognition there. Additional neural networks can then be used in the selected area. Alternatively, feature recognition can be performed separately (from the outset) for the selected area and the rest of the image. In this case, different artificial neural networks can be used for the selected area and the rest of the image, differing, for example, in depth and / or the number of layers.

[0035] Calculations (convolution, pooling, batch normalization, activations) for the entire image are simpler compared to those performed on the area of ​​interest (selected area). Because the entire image is evaluated, no contextual information is lost; instead, more computation is performed on the area of ​​interest, allowing the focus to be on segmentation. For example, the upper part of the image typically contains the (less relevant) sky, which doesn't include any classes helpful for lawn edge detection; however, it is relevant to determine whether it is cloudy, sunny, etc. The lower part of the image typically contains grass and nearby objects (close to the robot); therefore, more calculations are performed there to obtain more accurate segmentation.

[0036] Since the result of feature recognition is scaled down (due to the aforementioned dimensionality reduction), some values ​​(used to determine or extract the features) can be stored beforehand to improve the accuracy of the upscaling. Values ​​can be stored during downscaling and then reused during upscaling to obtain a more precise output. For example, if a 10x10 matrix is ​​reduced to a 2x2 matrix and then enlarged back to a 10x10 matrix, the output (i.e., the resulting image) would be blurry and inaccurate. However, if intermediate values ​​are stored in a 5x5 matrix, these stored intermediate values ​​can be used during upscaling. The extracted features can then be assigned to specific classes (e.g., grass, road, dirt, people, house, etc.).The variables are selected, chained, and assigned threshold values, and then scaled up to a feature map, meaning the probabilities of each of these classes are stored in the image. This then represents the resulting image. For a more detailed explanation, please refer to the sections on the figures, in particular... Figure 2 , referred.

[0037] A computing system according to the invention, for example a computing unit such as a control unit of a robot, is, in particular in terms of programming, equipped to carry out a method according to the invention.

[0038] The invention further relates to a device, in particular a mobile device, with such a computing system (e.g., as a control unit) and an environmental sensing device, such as a camera for capturing an image of the surroundings. The device is preferably designed as a robot, in particular as a robotic lawnmower, as a household robot, e.g., a vacuuming and / or mopping robot, as a floor or street cleaning device, as an at least partially automated vehicle, or as a drone.

[0039] Implementing a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is also advantageous, as this incurs particularly low costs, especially if an executing control unit is already available for other tasks. Finally, a machine-readable storage medium is provided with a computer program stored on it as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical, and electrical storage media, such as hard drives, flash memory, EEPROMs, DVDs, etc. Downloading a program via computer networks (Internet, intranet, etc.) is also possible. Such a download can be wired or wireless (e.g., via a WLAN network, a 3G, 4G, 5G, or 6G connection, etc.).

[0040] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawing.

[0041] The invention is schematically illustrated in the drawing using an exemplary embodiment and is described below with reference to the drawing. Brief description of the drawings

[0042] Figure 1 schematically shows a device according to the invention in a preferred embodiment in an environment. Figure 2 shows an image captured by an environment sensing means of a device and a resulting image generated therefrom by means of semantic segmentation. Figure 3 schematically shows a sequence of a method according to the invention in a preferred embodiment. embodiment(s) of the invention

[0043] In Figure 1A schematic representation of a device 100 according to the invention in a preferred embodiment is shown in an environment 150, e.g., a property. The device 100 can, in particular, be a robotic lawnmower with wheels 120, a computer system 110 designed as a control unit, and a camera 130 with a field of view 132. For better illustration, the field of view 132 is chosen to be relatively small here; in practice, however, the field of view can be, for example, at least 180° or at least 270°.

[0044] The surroundings or property 150 here include, for example, various areas, namely lawn 152 (a large lawn area at the bottom of the picture as well as two smaller ones on the left and top right of the picture), a path 154 which extends from left to right in the picture and has two branches going upwards, as well as a wall or garden wall 156 between these two branches of the path.

[0045] The robotic lawnmower 100 is intended to move autonomously across the property and mow lawn 152. For this to work, it's crucial that the robotic lawnmower 100 recognizes a lawn boundary, in this case, a lawn edge 160, between the large lawn area 152 below and the path 154. This is necessary so that the robotic lawnmower 100 doesn't drive over this lawn edge 160, or at least doesn't continue mowing if it does. It's conceivable that the robotic lawnmower is allowed or even supposed to travel along path 154 to reach the other lawn areas, but doesn't mow while on path 154. Furthermore, recognizing the lawn edge 160 is important so that it can navigate along it and mow the area.

[0046] In Figure 2 The upper illustration shows an image 200, which was taken by an environmental detection device such as the camera 130 of the robotic lawnmower 100. Figure 1The image has been captured. It shows the various pieces of lawn 152, the path 154, and the wall 156, as well as a sky 158. It is already apparent that a large part of image 200 consists of lawn, which is primarily due to the low ground position of the robotic lawnmower 100 and its camera 130.

[0047] The lower illustration shows image 210, which is a result image generated from image 200 using semantic segmentation. In this process, features are extracted from image 200, for example, for each pixel, and these features are then used to assign each pixel to one of several classes. For example, only the classes "lawn" 212 and "no lawn" 214 will be used, which may be sufficient for a robotic lawnmower. In the result image, these two classes are represented as white (212) and hatched (214); in practice, this could be in black and white or, with more than two classes, in different colors; various other features that can be digitally processed can also be used.

[0048] As mentioned, in semantic segmentation the entire image 200 can be processed uniformly, particularly during feature extraction. However, within the scope of the present invention, a region of the image 200 is to be selected in which, relative to a portion of the selected region of the (entire) image 200, a higher proportion of the computing power of the executing computer system, e.g., the control unit 110, is required. Figure 1 , is used for the remainder of image 200. As an example, such a selected area is labelled 220; this is a lower, middle area of ​​image 200.

[0049] The selection of this area 220 can be made particularly depending on the current position of the robotic lawnmower 100 within the environment 150, as well as due to the fact that its camera is located relatively close to the ground. For example, it can be taken into account that a large part of the image in the lower area will be grass, which is due to the specific viewing direction of a robotic lawnmower's camera. Based on the robot's configuration and the camera's placement and orientation, it is possible to determine, at least approximately, where the horizon line (under the sky 158 in Figure 2 The image assumes a level surface. Therefore, the lower middle part of the image will generally contain the more relevant features (lawn or lawn edge).

[0050] It can therefore be assumed that the lawn edge 160 is located in the selected area 220, at least if the robot is close enough to the lawn edge 160 so that semantic segmentation with increased computing power compared to the rest is meaningful; thus, the lawn edge 160 can be accurately determined despite the overall low computing power.

[0051] In Figure 3 The diagram schematically illustrates the sequence of steps in a preferred embodiment of a method according to the invention, in the form of a flowchart. An exemplary sequence of steps that can be performed is shown.

[0052] In step 300, an image, such as image 200, is created. Figure 2The image is captured and transmitted to the computer system. There, the image – as an input image – can optionally be pre-processed in step 302 using an image processing system to correct lens distortions, lighting conditions, perspective, etc. The (optionally pre-processed) image is then forwarded to an artificial neural network 304 for further processing.

[0053] In step 306, a dimension reduction is performed. Since the calculation cannot (or should not) be performed for every pixel of the image due to the size (data volume), the image dimension is reduced using convolution, pooling, batch normalization, and activation functions, as explained above.

[0054] Then, in step 308, features 310 of the image are extracted, for example, based on several factors such as shape, color, context, pattern, light variance, and image context. By definition, a feature is, for example, an individually measurable property or characteristic of a phenomenon. For the context of a neural network, this would mean, for example, training specific neurons to focus on certain properties of the image; some neurons could focus on color, pattern, texture, etc. Extracting features means, in particular, that certain regions in an image activate specific neurons, and their value is passed on to the network for analysis, threshold determination, and ultimately segmentation into the specific class.

[0055] Some of these features are stored in step 312 to ensure easy and fast upscaling later, meaning they help scale the result to the size of the original input image without losing resolution. The network also learns the stored values; this is more efficient than interpolating values ​​or nearest neighbors because contextual information is added. For example, to upscale a square, typically only four points are needed; to upscale a circle, significantly more points would be required—at least in image processing.

[0056] Furthermore, in step 314, context-aware pooling, batch normalization, activation functions, and other operations can be performed. "Context-aware" in this context means that the network extracts contextual information, for example: "This area of ​​the image looks like a door, a window, and a wall. Therefore, the entire area is most likely a house." If something is then found in this "house" that most likely appears to be sky, the information can be discarded because it does not fit the context. This facilitates, or even results in, the detection of "grass" in the middle of the sky, as this is context-aware and impossible.

[0057] In step 316, an area of ​​the image (e.g., area 220) is selected. Figure 2) selected; this area can be determined or calculated in two ways, namely, for example, by analyzing an existing dataset of images and averaging the percentage of the (current) image that contains grass. Alternatively, a method can be used that depends on the physical properties of the robotic lawnmower, the camera's placement, and its orientation relative to the ground area.

[0058] In step 318, the more difficult convolutions or a larger set of operations using the neural network are then concentrated on the selected area, while a smaller set of operations is applied to the rest of the image.

[0059] The resulting feature map is then, in step 320, classified into its corresponding classes, e.g., but not limited to: "lawn" and "no lawn", as in Figure 2The results of the calculation for the rest of the image and the focused calculation in the selected area of ​​the image are displayed and concatenated into a resulting classified matrix 322. The classified matrix (i.e., an intermediate result) is then upscaled in a decoding phase using the residuals (the stored features or values) from the encoding phase, step 324, in order to obtain, in step 326, a result image as output that is similar in size to the input image.

[0060] The resulting image can then be mapped and, if necessary, further processed (e.g., indexing of classes, separation of output classes, thresholds) and passed on to possible applications.

[0061] Various arrangements or sequences of this approach or the steps can be chosen; as has been shown, for example, selecting the area of ​​interest after dimensionality reduction has little impact on accuracy and can even achieve better inference times. Context-aware feature selection can also be performed across the entire image or (only) in the selected area, depending on the application and the desired focus on that area.

Claims

1. Method for semantic segmentation of an image (200) that has been captured by an environment capturing means (130) of an automatedly moving device (100), by means of a computing system (110) having a computing power, wherein the image (200) is obtained and a region (220) is selected in the image, wherein the region (220) in the image is selected depending on a position of the environment capturing means (130) in the device (100), with respect to an underlying surface on which the device (100) is moving, wherein, in the context of the semantic segmentation, segments, in particular pixels, of the image (200) are each assigned one of a plurality of classes (212, 214), wherein, relative to a proportion of the image constituted by the selected region, a higher proportion of the computing power is used for the selected region (220) of the image than for the rest of the image by virtue of more computation operations being carried out in the selected region (220) of the image than in the rest of the image, wherein a classified result image (210) is generated and in particular output, and wherein the result image is used to control the device (100).

2. Method according to Claim 1, wherein the segments of the image (200) are each assigned one of a plurality of classes by virtue of features (310) being determined for each of the segments of the image, and wherein the respective class is assigned on the basis of the features.

3. Method according to Claim 2, wherein the determination of the features for the selected region (220) and for the rest of the image is performed using artificial intelligence-based pattern recognition methods, in particular artificial neural networks (304), which have a different depth and / or a different number of layers.

4. Method according to Claim 3, wherein, before the determination of the features, the image (200), in particular only the rest of the image, is downscaled with regard to dimensions to be taken into account.

5. Method according to Claim 4, wherein the image (200) or the rest of the image is upscaled again after the determination of the features and before the assignment of the classes (212, 214).

6. Method according to any of the preceding claims, wherein the region (220) in the image is selected depending on a current position of the device (100) within the environment (150).

7. Method according to any of the preceding claims, wherein the device (100) used is a robot, in particular a robotic lawnmower, a domestic robot, such as e.g. a robotic vacuum cleaner and / or mop, a floor or road cleaning device, an at least partially automated vehicle, or a drone.

8. Computing system (110) which is designed to carry out all the method steps of a method according to any of the preceding claims.

9. Device (100), in particular mobile device, comprising an environment capturing means (130) for capturing an image (200) of the environment (150) and a computing system (110) according to Claim 8.

10. Device (100) according to Claim 11, which is in the form of a robot, in particular a robotic lawnmower, a domestic robot, e.g. a robotic vacuum cleaner and / or mop, a floor or road cleaning device, an at least partially automated vehicle, or a drone.

11. Computer program which causes a computing system (110) to carry out all the method steps of a method according to any of Claims 1 to 7 when it is executed on the computing system (110).

12. Machine-readable storage medium with a computer program according to Claim 11 stored thereon.