Method for creating an environment map
Through camera detection and weighted average processing of mobile self-propelled devices, the problem of unstable image quality in the environmental map is solved, high-quality and stable environmental map updates are achieved, and distortion and brightness fluctuations in the map are reduced.
Patent Information
- Application Number
- CN202510158584.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-02-14
- Filing Date
- 2025-02-13
- Publication Date
- 2025-08-15
AI Technical Summary
In the prior art, environmental maps created by mobile self-propelled devices are susceptible to floor unevenness, pitch motion and lighting changes, resulting in unstable and distorted image quality, making it difficult to achieve high-quality environmental map updates.
Multiple images of the floor area are detected by the camera, transformed using known parameters and device position orientation, combined image representations with weighted averages, and updated image content in the environmental map to reduce distortion and lighting-related brightness fluctuations.
The stability and high-quality update of the environmental map are achieved, the distortion and brightness fluctuations in the map are reduced, and the coherence and coordination of the image are improved.
Smart Images

Figure CN120477654A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for creating an environment map of an environment area for operating a mobile self-propelled device, in particular a floor cleaning device such as a suction and / or sweeping and / or wiping robot, a mobile self-propelled device, a computer program product, and a computer-readable data carrier. Background Art
[0002] Mobile autonomous devices, such as vacuum robots, have the task of autonomously cleaning the entire floor surface as much as possible. In particular, these autonomous devices are supposed to perform the laborious, repetitive task of floor cleaning on behalf of their users. The robots regularly carry out cleaning orders or are dispatched by the user to clean defined floor surfaces. For this purpose, the cleaning robots can be operated, for example, via a smartphone application that displays a map of the surroundings. This map typically displays a floor plan of a living area, typically with its walls, furniture, and objects, as outlines.
[0003] The representation in the environment map mostly involves the outlines of boundary walls and objects, so that a kind of floor plan of the living area is displayed. Apart from their outlines and their relative positions to each other, the different rooms are often not distinguishable from each other.
[0004] To enable users to better identify real locations in the application's environment map and thus facilitate the operation of the robot, a graphical map can be used. In this case, camera images of the robot are recorded and gradually used to implement a floor representation in the application. This allows the environment map to be realistically represented in the application, including a graphical representation of the robot.
[0005] The conversion of a robot's camera image into a map image for an application is traditionally based on a transformation that assumes that the floor detected by the robot using its camera is completely flat and that the robot, and therefore the camera, always has an unchanging orientation, such as a tilt. Floor unevenness and the robot's pitching motion during travel can lead to distortions in the map image, which can vary depending on the situation and direction of travel.
[0006] Other influences on the image representation may be changing lighting conditions, such as daylight or artificial light, reflections or glare. Such situations may also have a negative impact on the illustrated map, for example when the floor surface is represented in sections with different brightness.
[0007] Furthermore, if an already existing image representation of the environment map is simply replaced by a newly recorded image representation, there is a risk that the illustrated map deteriorates or the image quality fluctuates, since there is no guarantee that the newly recorded image representation has better image quality than the already existing image representation.
[0008] Publication DE 102018132428 A1 discloses a method for photomosaic floor mapping performed by a robot, in which the image of the robot's camera is converted into a bird's-eye view by means of a transformation. The camera's field of view is recorded as an image, cropped into parts or segments, converted into a plan view, and combined with other images. A combined, illustrated map is formed without the involvement of the user. To ensure high image quality, the quality of each recorded image is determined, and in the case of multiple detections of the same segment with different quality versions, the lower-quality version is replaced by the higher-quality version. However, simply replacing one version of a segment with another can disadvantageously result in visible fluctuations in the image content (e.g., brightness), which can reduce the overall quality of the environment map. Summary of the Invention
[0009] The object of the present invention is to provide an improved method for creating an environment map, in which a stable continuity of the map image representation over time can be achieved and in which the quality of the depicted environment map can be improved during its creation and during its further (continuous) updating.
[0010] This object is achieved by a method for creating an environment map of an environment for operating a mobile vehicle having the features of claim 1. Advantageous embodiments and developments are the subject matter of the dependent claims.
[0011] According to the invention, a method for creating an environment map of an environment in which the mobile autonomous device, in particular a floor cleaning device such as a vacuum and / or sweeping and / or wiping robot, is operated by means of a camera and a processing device of the device, comprises the following method steps:
[0012] a) using a camera to detect (n-1) images of the floor area of the grid cells of the environment area;
[0013] b) transforming the (n-1) images using known parameters of the camera, its position and orientation at the device, such that a bird's-eye view first image representation of the floor area of the grid cell is produced;
[0014] c) combining the first image representations by performing a weighted averaging and inserting the first averaged image representation of the floor area into the environment map by including the position and orientation of the device relative to the environment map;
[0015] d) detecting at least one nth image of a floor area of the grid cell and transforming the nth image into a second image representation;
[0016] e) combining the first and second image representations of the grid cell while performing a weighted averaging of all image representations; and
[0017] f) Overwriting the first average image representation in the environment map with the second average image representation.
[0018] The method according to the invention advantageously improves the quality of the illustrated environment map during its creation and during its further (continuous) updating. This results in a visually better representation of the floor surface with fewer distortions, fewer lighting-related brightness fluctuations, and thus an overall more harmonious, coherent image surface.
[0019] The device has a large number of sensors with which it perceives its surroundings, in particular lidar sensors, which are used to create a map of the environment with outlines of walls and obstacles. In addition, the device has at least one camera that can detect objects in front of the device and is used, for example, for object recognition and classification. The camera's field of view (FoV) can also detect sections of the floor. These parts of the image can be used to perform photomosaic floor mapping. In this case, the image content is transformed using the known parameters of the camera and its position and orientation at the device so that it corresponds to a bird's-eye view of the recorded floor area. By including the device's position and orientation relative to the environment map at the time of the image recording, the corresponding partial surface in the map representation can be filled with the image representation.
[0020] The device uses a camera to capture a large number of images from alternating directions, under varying lighting conditions, and sometimes at varying pitch angles relative to obstacles, thresholds, etc., due to floor unevenness, for example. To combine these images into the resulting floor surface, a weighted average is calculated for the previous and new image representations of the floor surface. Instead of completely overwriting the image representation of the currently captured area of the floor with the new image representation, the previous and new representations are merged. The more frequently a floor section is captured by the camera, the more stable the stored image representation of that area remains. Because the device captures floor areas multiple times over time, individual erroneous recordings due to pitch movements or unfavorable glare do not affect the overall representation. Consequently, individual "problem images" have little impact or are promptly suppressed. Furthermore, automatic averaging is performed for lighting conditions such as daylight or artificial light from lamps, as well as for different viewing angles.
[0021] Mobile autonomous devices are particularly understood to be floor cleaning devices that autonomously clean floor surfaces, particularly in the home. This includes, in particular, suction, mopping, and / or sweeping robots, such as robot vacuum cleaners. These devices preferably operate in operation (cleaning mode) without user intervention or with minimal user intervention. For example, the device autonomously moves through a predetermined room to clean the floor according to a predetermined and programmed method strategy.
[0022] The floor surface to be treated can be understood as each room surface to be cleaned. In particular, subareas of individual rooms, individual surfaces of an apartment, individual rooms of an apartment and / or the entire floor surface of an entire apartment or house also belong to this.
[0023] A grid cell can be understood in particular as a subregion of the surrounding area. In particular, the surrounding area is composed of a plurality of grid cells, which are identical or at least similar in particular in their size, shape, orientation, etc.
[0024] An environment map is particularly understood to be any map suitable for representing the environment of a floor treatment area with all its walls, obstacles and objects. For example, an environment map can show a floor treatment area in a sketched manner with the furniture and walls contained therein.
[0025] An obstacle can be understood to be any object and / or item that is arranged in the floor treatment area, for example placed or standing there, and that influences, in particular hinders and / or interferes with the treatment by the mobile device, such as furniture, walls, curtains, carpets, etc.
[0026] Preferably, a map of the environment with obstacles is preferably displayed in an application on the portable input device. This serves in particular to visualize possible interactions for the user.
[0027] In the present case, an input device is understood to be, in particular, any device that is portable for a user, is arranged outside the mobile self-propelled device, in particular externally and / or distinctly therefrom, and is suitable for displaying, providing, conveying and / or transmitting data by means of an interface, such as a mobile phone, smartphone, tablet and / or computer or laptop.
[0028] An application, in particular a control application and / or a cleaning application for the device, is installed on the input device, which application is used to communicate with the mobile self-propelled device and the input device and, in particular, allows visualization of the floor treatment area, i.e., the house or apartment or residential area to be cleaned. The application preferably displays the area to be cleaned as an environment map, including any obstacles, to the user.
[0029] A camera can be understood to be, in particular, any image recording device which is suitable for recording images of its surroundings, preferably with high image quality. The camera is here part of the device and is, in particular, integrated in the device.
[0030] The camera detects (n-1) images of the floor area of a grid cell of the environment area, where n is an integer greater than 1. The camera further detects the nth image in a time-shifted manner and optionally detects other images of the floor area of the same grid cell.
[0031] A processing device can be understood in particular as any device suitable for handling, processing, combining, storing, transforming and / or rewriting image representations. The processing device is part of the device and, in particular, is integrated into the device. The (n-1) images are transformed by means of the processing device so that a first bird's-eye view image representation of the floor area of the grid cell is generated. Furthermore, the processing device transforms the nth image and optionally other detected images into a second image representation, with a time offset. Furthermore, the processing device then merges the first and second image representations of the grid cell and thereby updates the average image representation of the environment map by rewriting. Each image representation is also preferably stored in the processing device, in particular in a memory of the processing device.
[0032] In one advantageous embodiment, the surrounding area is divided into a plurality of grid cells, wherein method steps a) to f) are performed in each grid cell. By preferably completely dividing the surrounding area into grid cells, a realistic representation of the entire floor covering of the surrounding area can be advantageously ensured. This ensures long-term stability of the image representation of the entire surrounding area. The entire surrounding area can be represented with high precision and detail-accurately.
[0033] In another advantageous embodiment, (n-1) images are captured during a survey and / or cleaning run, and the nth image and optionally further images are captured during a subsequent cleaning run. After being commissioned at the user's site, the device is dispatched on a survey run, during which the device surveys the environment, creates a map of the environment, and simultaneously integrates the floor surface into the environment map as a graphical map. While creating the graphical map, and possibly also later during a subsequent cleaning run, the device's camera records (n-1) images of the floor surface. The nth image and any subsequent images are recorded during subsequent cleaning runs and used to update the environment map by taking into account a weighted average of all previous image representations.
[0034] In another advantageous embodiment, (n-1) images are recorded from alternating directions with different lighting conditions and / or at different elevation angles. These recorded (n-1) images are aggregated to reduce erroneous recordings in terms of their weighting, so that they barely affect the overall representation of the floor surface. Preferably, the nth image and subsequent images are also recorded from alternating directions with different lighting conditions and / or at different elevation angles.
[0035] In another advantageous embodiment, the second average image representation is obtained by multiplying the first average image representation by the number of detected first image representations, adding the second image representation to this result, and dividing this result by n, where n is an integer greater than 1. This results in the following formula:
[0036] B nij,ij =[(B (n-1)ij,ij *(n-1) ij )+D nij,ij ]:n ij
[0037] Among them D n is the new image representation of the camera, B n is the average image representation stored in memory, ij is the index of the grid cell, and n ij is the number of detections for this grid cell. This results in a clustering of the image representation for the grid cells. In the case of sections of the floor surface that are particularly frequently detected by the camera, individual problematic images have little influence on the overall result or are suppressed in a timely manner. This can advantageously result in automatic averaging of lighting conditions and different viewing angles. To this end, the image representation stores not only the previous image representation (B) of each grid cell, but also the number of the previous (n-1) detections for each grid cell.
[0038] In another advantageous embodiment, the first and second image representations include color channels and / or color settings, to which the weighted averaging is applied. For the color representation of the image representations, the formula is applied, for example, individually to the color channels (RGB) and / or the color settings (HSV).
[0039] In another advantageous embodiment, the detection of (n-1) images is limited to a maximum, so that even after many detections, the new image representation still retains a certain influence. This allows, in particular, changes in the apartment to be gradually incorporated into the image representation of the environment map. In particular, the weighting of the first average image representation is overwritten. This means that if n-1+x images are recorded, these images belong to the first average image representation, but their weighting is not set to n-1+x, but to n-1.
[0040] In particular, weighting based on the number of previous detections can result in changes in the apartment, such as rearranging furniture, being barely or only very slowly incorporated into the image representation. By performing a reset, the user can set the weighting factor n-1 (the number of first detected image representations) to 1 or 0, giving the new image representation a stronger impact, or overwrite the old image representation without necessarily deleting the entire environment map, allowing unchanged areas to remain in the environment map. This reset does not necessarily have to be applied to the entire floor surface; rather, it can be applied to selected areas of the environment map using a function within the application. Alternatively, the user can have the device perform a resurvey.
[0041] In another advantageous embodiment, the similarity between the first image representation and the second image representation influences the second average image representation. In this case, not only the number of previously detected grid cells in the image representation is taken into account, but also the similarity between the new image representation and the previous image representation. The above formula is thus modified, for example, as follows:
[0042] B nij,ij =[B (n-1)ij,ij +(D nij,ij *W Dn,ij )](1+W Dn,ij )
[0043] W Dn,ij =F:[(n-1) ij *(D nij,ij -B (n-1)ij,ij )]
[0044] Where W Dn,ij is a weight based on the number of previous image representations and the difference between image representations; F can be an arbitrary factor, where F>0 and preferably F=1.
[0045] If the similarity is not only determined by the difference with the previous image representation, but is based on the distribution of all previous image representations, then the standard deviation of the image representation is used. The weighting factors are then, for example, as follows:
[0046] W Dn,iij =F:[σ nij,ij *(D nij,ij -μ nij,ij )]
[0047] where σ nij,ij is the standard deviation over all image representations (previous and current), and μ nij,ijis the average over all image representations (previous (first) image representation and current (second) image representation) for a grid cell; F may be an arbitrary factor, where F>0 and preferably F=1.
[0048] The less similar the image representations at different points in time are for a grid cell, the lower the weight given to the new image representation. Therefore, if the same floor area is repeatedly detected multiple times, this has a stabilizing effect on the graphical environment map. Short-term disturbances can be advantageously suppressed.
[0049] In another advantageous embodiment, the device moves along the boundary lines of the floor area when capturing the (n-1)th image, the nth image, and any subsequent images. In particular, the device does not move transversely to the boundary lines. Boundary lines can be found, for example, at the edges of carpets or between floor surfaces of different coverings, such as at the transition between wooden floors and tiles. If the device moves obliquely or transversely to these boundary lines, the transitions can be perceived differently from different directions. For example, due to the physical height of the carpet, the carpet edge appears flatter to the device than from the perspective of the adjacent floor covering, potentially leading to distortions in the image representation. However, if the device moves along the boundary lines, these distortions can be offset.
[0050] In another advantageous embodiment, the environment map is processed using image processing. Preferably, the device checks the created graphical environment map using the image processing device after the exploration trip or additionally at specific time intervals. Morphological operations such as region growing, erosion, dilation, or closing can be used to create connected regions that can preferably be assigned to specific floor coverings. In an additional or alternative step, edge detection algorithms such as the Canny algorithm or the Sobel operator can be applied to identify significant boundary lines in the image representation. In another optional processing step, geometric shapes such as lines and circles can be determined, for example, using a Hough transform.
[0051] The detection of existing boundary lines can be further improved by using additional sensors. For example, an ultrasonic sensor integrated into the device can infer the type of floor covering. An acceleration sensor or gyroscope can detect the device's inclination when passing a boundary line (such as the edge of a carpet). The current consumption of the device's travel drive, brush roller motor, side brush motor, or other actuators in contact with the floor can infer changes in the foundation. Any combination of the above methods can also be used.
[0052] The device then evaluates the image representation where the boundary lines are located. The device preferably performs targeted improvements along these boundary lines in order to refine the graphical environment map, particularly in the area of adjacent floor coverings, and minimize distortions. The image representation in the area of the boundary lines is particularly preferably aggregated with an increased weighting with the previous image representation or replaces it in order to obtain an improved graphical environment map.
[0053] The present invention further relates to a mobile self-propelled device, in particular a floor cleaning device such as a suction and / or sweeping and / or wiping robot, which comprises a camera and a processing device and is configured to carry out the method according to the invention for creating an environmental map of an environmental area.
[0054] It goes without saying that, in addition to the method and the device, a computer program product also falls within the scope of the present invention, the computer program product comprising instructions which, when executed by a device, cause the device to perform the method according to the present invention. Likewise, a computer-readable medium having such a computer program product stored thereon also falls within the scope of the present invention.
[0055] Each feature, configuration, embodiment and advantage associated with the method also applies in conjunction with the device, the computer program product and the computer-readable medium according to the invention, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] The invention is explained in more detail with reference to the following embodiments of the invention which are presented by way of example only. In which:
[0057] Figure 1 A schematic diagram of a mobile self-propelled vehicle configured to carry out the process according to the invention for creating an environmental map of an environmental area is shown,
[0058] Figure 2 shows a flow chart of an embodiment of a method according to the invention for weighted merging of a new (second) image representation and a previous (first) image representation,
[0059] Figure 3 A schematic diagram showing an embodiment of an environment map created by the method according to the present invention and allowing the user to reset the environment map,
[0060] Figure 4 A schematic diagram showing an embodiment of creating an environment map using the method according to the present invention and processing boundary lines, and
[0061] Figure 5 A flow chart of an embodiment of a method for improving a boundary line according to the present invention is shown. DETAILED DESCRIPTION
[0062] exist Figure 1 , a mobile self-propelled device 10 is shown, which is in particular a suction robot. The suction robot perceives its environment using various sensors, in particular a lidar sensor 1, which is used to create an environment map with outlines of walls, objects and obstacles. In addition, the suction robot has a camera 2, which can detect objects in front of the suction robot and is used, for example, for object recognition and object classification. The camera's field of view also detects sections of the floor covering. These parts of the image representation are used to perform photomosaic floor mapping. In this case, the image content is transformed using the known parameters of the camera and its position and orientation at the suction robot, so that a bird's-eye view of the recorded floor covering is generated. The image representation can be used to fill in the corresponding floor surface in the environment map, while including the robot's position and orientation relative to the environment map at the time of the image recording.
[0063] To this end, after being put into operation, the suction robot is sent on a reconnaissance tour, during which it surveys the environment, creates a map of the environment, and simultaneously integrates the floor surface into the environment map as a graphical map. During the reconnaissance tour and during the suction robot's subsequent cleaning tours, the camera 2 repeatedly re-acquires the floor surface from alternating directions, under different lighting conditions, and sometimes at first at different pitch angles. To improve the quality of the graphical environment map during its creation and during its subsequent continuous updating, the present invention generates a weighted average of the previous and new image representations of the floor surface. This advantageously enables a harmonious, coherent representation of the floor surface to be produced in the environment map with minimal distortion and lighting-related brightness fluctuations.
[0064] At present, the image representation for the current floor area is therefore not completely overwritten with a new image representation, but rather the previous first image representation is merged with the new second image representation of the same floor area, resulting in a stable image representation. In this case, the floor area is divided into individual grid cells. For example, weighted averaging is applied individually to the color channels or color settings. To this end, for each grid cell, not only the previous first image representation but also the number of all previous first detections for each grid cell are stored in the suction robot, in particular in its processing device. The created average image representation is stored in the environment map and, in the case of an updated average image representation, is overwritten with the updated average image representation. The current average image representation is obtained from the previous (first) image representation of the grid cell multiplied by the number of previous (first) image representations for this grid cell plus the current (second) image representation divided by the number of image detections for this grid cell (number of first + second image representations).
[0065] This weighted averaging of all image representations allows distorted or blurred images to be suppressed promptly or to have little effect from the outset. This method automatically averages lighting conditions and different viewing angles. The number of previous first image representations is preferably limited to a maximum value so that even after a large number of image acquisitions, new image representations contribute the necessary weight to the average image representation. Changes in the surrounding area are gradually incorporated into the environment map.
[0066] To further stabilize the environment map, the similarity between the new (second) image representation and the previous (first) image representation can also be considered. Weighting is performed using a weighting factor based on the number of previous image representations and the differences between the image representations. Alternatively, the similarity can be determined based on the distribution of all previous image representations using the standard deviation of the image representation.
[0067] exist Figure 2 A flow chart for weighted averaging is shown in FIG. In a first method step 3, the suction robot performs an exploratory walk and uses its camera to detect (n-1) images of the floor area of a grid cell of the environment area, where (n-1) is the number of detected images and n is an integer greater than 1. These (n-1) images are transformed using the known parameters of the camera and its position and orientation at the device using a processing device of the suction robot, so that (n-1) first image representations of the floor area of the grid cell, taken from a bird's-eye view, are generated. When performing weighted averaging, these (n-1) first image representations are combined to form a first average image representation.
[0068] In method step 4, the vacuum robot creates a graphical environment map based on the first average image representation of the floor area, including the device's position and orientation relative to the environment map. In method step 5, the environment map, along with the first image representations, their number, and the first average image representation, is stored in a processing device of the vacuum robot.
[0069] When a cleaning order is received by the suction robot (step 6), the suction robot collects additional current images of the floor area of the same grid cell. In particular, at least one additional image (the nth image), preferably a plurality of additional images, is recorded from different positions. These additional current images are transformed using the known parameters of the camera, its position and orientation at the device, to produce at least one second bird's-eye view image of the floor area of the grid cell.
[0070] In step 7, the vacuum robot aggregates the first and second image representations of the grid cells while performing a weighted averaging of all image representations, thereby forming a second average image representation. This second average image representation and the updated weighting factors are stored in the vacuum robot's processing device (step 8). Furthermore, the first average image representation stored in step 5 is overwritten with the second average image representation from the environment map. The modified and updated environment map is then displayed to the user in an application on their smartphone (step 9).
[0071] Preferably, the user is provided with a function for resetting the illustrated environment map, without necessarily having to delete the entire environment map, so that unchanged areas remain in the map. In this case, the weighting factor n-1 is set to 1 or 0, so that the new image representation has a stronger impact or the old image representation is overwritten once. This reset does not have to be applied to the entire floor surface, but can be limited to selected areas of the map via a function in the application. Figure 3 , such a reset area 12 is shown in the surroundings map 11 , with which the user is offered the option of overwriting the previous map representation in the set area 12 during the next cleaning run.
[0072] The creation of the illustrated environment map can be further improved if the suction robot does not move transversely to the boundary lines between floor surfaces of different coverings when detecting the image of the floor area, but moves along these boundary lines. In order to identify such boundary lines, the existing illustrated environment map is processed using image processing means. Connected areas of the same floor covering can be created by morphological operations such as region growing, erosion, expansion or closing. In a further or alternative step, an edge extraction algorithm (e.g. Canny algorithm, Sobel operator) can be applied to draw out significant boundary lines in the image. In a further optional step, geometric shapes (lines, circles) can be determined (e.g. by means of a Hough transform).
[0073] The aforementioned boundary line detection is preferably improved by using additional sensors. For example, an installed ultrasonic sensor can infer the type of ground surface. When driving over the edge of a carpet, an installed acceleration sensor or gyroscope can detect the robot's inclination. For this purpose, the current consumption of the travel drive, brush roller motor, side brush motor, or other actuators in contact with the floor can also be used to infer changes in the ground surface.
[0074] The suction robot then evaluates the previous map image representation, where the boundary line 13 is located, as for example in Figure 4 The identified boundary line 13 is shown in Figure 4After identifying these boundary lines 13 , the robot performs a targeted refinement movement along these boundary lines 13 in order to refine the illustrated surroundings map, in particular in the area of the adjacent floor covering, and to minimize distortions.
[0075] The process of the method for improving the boundary line 13 in the environment map is as follows Figure 5 In step 14, the suction robot performs an exploration run. Subsequently, the suction robot creates a graphical map of the environment from the image representation of the camera, such as in combination with Figure 2 The process flow of the method according to the embodiment of the present invention is as explained above (step 15). The image representation is checked using image processing and boundary lines are extracted (step 16). The suction robot now travels along these extracted boundary lines and detects new images for the new image representation there (step 17). In the final step 18, the environment map is improved using the newly recorded image representations at the boundary lines.
Claims
1. A method for creating an environment map (11) of an environment area in which the mobile self-propelled device (10), in particular a floor cleaning device such as a vacuum and / or sweeping and / or wiping robot, by means of a camera (2) and a processing device of the device, comprising the following method steps: a) detecting (n-1) images of the floor area of the grid cells of the environment area using the camera (2); b) transforming the (n-1) images using known parameters of the camera (2), its position and orientation at the device (10) so as to produce a bird's-eye first image representation of the floor area of the grid cell; c) combining the first image representations by performing weighted averaging and inserting the first averaged image representation of the floor area into the environment map (11) by including the device position and orientation relative to the environment map (11); d) detecting at least one nth image of a floor area of the grid cell and transforming the nth image into a second image representation; e) combining the first and second image representations of the grid cell while performing a weighted averaging of all image representations; and f) Overwriting the first average image representation in the environment map (11) with the second average image representation. 2 . The method according to claim 1 , wherein the environmental area is divided into a plurality of grid cells and method steps a) to f) are performed in each grid cell. 3 . The method according to claim 1 , wherein (n−1) images are recorded during an exploration run and / or a cleaning run, and an nth image is recorded during a subsequent cleaning run. 4 . The method according to claim 1 , wherein (n−1) images are recorded from alternating directions with different lighting conditions and / or at different elevation angles. 5 . The method according to claim 1 , wherein the second average image representation is obtained by multiplying the first image representation by the number of detected first image representations, adding the second image representation to the result and dividing the result by n. 6 . The method according to claim 1 , wherein the first and second image representations comprise color channels and / or color settings, to which the weighted averaging is applied. The method of claim 5 , wherein the detection of (n−1) images is limited to a maximum.
8. The method of claim 5, wherein during calculation of the second average image representation, weights for the first average image representation are set to 1 or 0. 9 . The method according to claim 1 , wherein the similarity between the first image representation and the second image representation is influenced in a second average image representation.
10. The method according to claim 1, wherein the device is moved along a boundary line (13) of the floor area when detecting the (n-1) image and the nth image.
11. The method according to claim 1, wherein the environment map (11) is processed by image processing means in order to determine the boundary lines of the floor area.
12. A mobile self-propelled device (10), in particular a floor cleaning device such as a suction and / or sweeping and / or wiping robot, comprising at least one camera (2) and a processing device, and being configured to create an environmental map (11) of an environmental area according to one of the preceding claims.
13. A computer program product comprising instructions which, when the program is executed by a device (10), cause the device (10) to perform the method according to one of the preceding claims 1 to 11.
14. A computer-readable data carrier having stored thereon the computer program product according to claim 13.
Citation Information
Patent Citations
Photomosaic soil mapping
DE102018132428A1