Environment map construction method, device, and storage medium

Abstract

Embodiments of the present application provide an environment map construction method and device and a storage medium. In the embodiments of the present application, an environment map containing an unoccupied area is constructed in combination with a walking track of an autonomous mobile device; and on this basis, a plurality of sub-areas existing in the unoccupied area are identified in combination with positions of spatial segmentation bodies existing in the environment area; and an environment map similar to a house type map is constructed by adding outer walls to the plurality of sub-areas. The environment map is closer to the intuitive feeling of a user and is convenient for the user to understand. The user can interact with the autonomous mobile device more conveniently and accurately based on the environment map, and the quality of human-computer interaction based on the environment map is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an environmental map construction method, device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, the functions of self-moving robots are becoming increasingly powerful, and the demand for interaction with users is also constantly increasing. In light of this, self-moving robots are beginning to present users with environmental maps constructed using Simultaneous Localization and Mapping (SLAM) technology, hoping that users can interact with the self-moving robot through these maps and more flexibly instruct it to perform tasks. For example, a robotic vacuum cleaner presents an environmental map to the user, hoping that the user can flexibly and conveniently instruct the robot to perform targeted cleaning tasks in specific areas. However, the problem is that the human-computer interaction based on environmental maps is not ideal. Summary of the Invention

[0003] This application provides an environmental map construction method, device, and storage medium to construct an environmental map that is easy for users to understand and improve the quality of human-computer interaction based on the environmental map.

[0004] This application provides an environmental map construction method applicable to autonomous mobile devices. The method includes: acquiring a first environmental map corresponding to traversed environmental areas, the first environmental map including unoccupied areas; identifying area dividing lines in the unoccupied areas based on the positions of spatial dividers in the environmental areas, the area dividing lines dividing the unoccupied areas into multiple sub-areas; adding external walls to the multiple sub-areas to obtain a second environmental map, the second environmental map including the multiple sub-areas and their external walls.

[0005] This application also provides a method for controlling an autonomous mobile device, applicable to a terminal device. The method includes: in response to a display operation, displaying an environmental map, the environmental map including multiple passable sub-areas and their outer walls within a corresponding environmental area; in response to a trigger operation on the environmental map, sending a control command to the autonomous mobile device to cause the autonomous mobile device to perform a specified operation on a target object in the environmental area; wherein the target object corresponds to a triggered map location on the environmental map, and the map location is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0006] This application embodiment also provides an autonomous mobile device, including: a device body, the device body having a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program to: acquire a first environmental map corresponding to a traversed environmental area, the first environmental map including unoccupied areas; identify area dividing lines in the unoccupied areas based on the positions of spatial dividers in the environmental areas, the area dividing lines dividing the unoccupied areas into multiple sub-areas; add external walls to the multiple sub-areas to obtain a second environmental map, the second environmental map including the multiple sub-areas and their external walls.

[0007] This application embodiment also provides a terminal device, including: a memory, a processor, and a display; the memory is used to store a computer program; the processor, coupled to the memory, is used to execute the computer program for: in response to a display operation, displaying an environmental map through the display, the environmental map including multiple passable sub-areas and their outer walls in a corresponding environmental area; in response to a trigger operation on the environmental map, sending a control command to an autonomous mobile device to cause the autonomous mobile device to perform a specified operation on a target object in the environmental area; wherein the target object corresponds to a triggered map position on the environmental map, and the map position is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0008] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the methods provided in this application.

[0009] In this embodiment, an environmental map containing unoccupied areas is constructed by combining the walking trajectory of the autonomous mobile device. Based on this, the locations of spatial dividers in the environmental area are further combined to identify multiple sub-areas within the unoccupied areas. By adding exterior walls to the multiple sub-areas, an environmental map similar to a floor plan is constructed. This environmental map is closer to the user's intuitive experience and easier for the user to understand. Based on this environmental map, the user can interact with the autonomous mobile device more conveniently and accurately, which is conducive to improving the quality of human-computer interaction based on the environmental map. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0011] Figure 1This application provides a schematic flowchart of an environmental map construction method according to an exemplary embodiment;

[0012] Figure 2a A schematic diagram of the map state showing unoccupied areas in an environmental map provided for an exemplary embodiment of this application;

[0013] Figure 2b Is Figure 2a A map showing the state of multiple sub-regions is obtained by adding region dividing lines to the unoccupied areas.

[0014] Figure 2c This is for Figure 2b A schematic diagram of the map state obtained by adding external walls to multiple sub-regions;

[0015] Figure 2d Yes Figure 2c A schematic diagram of the map state obtained after the outer walls of multiple sub-regions are aligned.

[0016] Figure 3 A schematic flowchart of a method for controlling an autonomous mobile device provided as an exemplary embodiment of this application;

[0017] Figure 4 A schematic diagram of an environmental map construction apparatus provided for an exemplary embodiment of this application;

[0018] Figure 5 A schematic diagram of the structure of a control device provided for an exemplary embodiment of this application;

[0019] Figure 6a A schematic diagram of the structure of an autonomous mobile device provided as an exemplary embodiment of this application;

[0020] Figure 6b A schematic diagram of the structure of an autonomous mobile robot provided as an exemplary embodiment of this application;

[0021] Figure 7 This application provides a schematic diagram of the structure of a terminal device. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Before introducing the embodiments of this application, the autonomous mobile device in these embodiments will be described. The autonomous mobile device in these embodiments can be any mechanical device capable of highly autonomous spatial movement within an environment, such as a robot, air purifier, or driverless transport vehicle. The robot can be a household robot, such as a robotic vacuum cleaner or a home care robot; or a commercial robot, such as a patrol robot, guidance robot, or service robot assisting users in various tasks in shopping malls, supermarkets, banks, hospitals, airports, or train stations, as well as a transport robot used in various warehouses. In these embodiments, the autonomous mobile device has a locomotion device, enabling it to move autonomously within its current environment. Additionally, the autonomous mobile device also has various sensors, such as visual sensors and laser sensors. The visual sensor can be a monocular camera, a binocular camera, or a depth camera (Red, Green, Blue-Depth map, RGB-D); the laser sensor includes, but is not limited to, line laser sensors and area laser sensors. These sensors can collect environmental information as autonomous mobile devices move. For example, visual sensors can capture images of the surrounding environment, while laser sensors can collect information about obstacles. This environmental information provides data for the autonomous mobile device's localization, navigation, and obstacle avoidance. In addition to the real-time environmental information collected by these sensors, localization, navigation, and obstacle avoidance also rely on environmental maps of the environment in which the autonomous mobile device operates.

[0024] In the following embodiments of this application, a novel map construction method is provided, which can construct an environmental map similar to a floor plan. This environmental map can not only be used for positioning, navigation, or obstacle avoidance of autonomous mobile devices, but also, because it is closer to the user's intuitive experience and easier for the user to understand, the user can also interactively control the autonomous mobile device based on this environmental map. Compared with traditional environmental maps that record obstacle information and environmental maps that record the movement trajectory of the device, using the environmental map provided in the embodiments of this application to interactively control the autonomous mobile device is beneficial to improving the quality of human-computer interaction based on the environmental map.

[0025] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart illustrating an environmental map construction method provided for an exemplary embodiment of this application. Figure 1 As shown, the method includes:

[0027] 11. Obtain the first environment map corresponding to the traversed environment areas. The first environment map includes unoccupied areas.

[0028] 12. Based on the location of spatial dividers in the environmental area, identify the area dividing lines in the unoccupied areas, which divide the unoccupied areas into multiple sub-areas.

[0029] 13. Add exterior walls to multiple sub-regions to obtain a second environment map, which includes multiple sub-regions and their exterior walls within the environment map.

[0030] In this embodiment, when the autonomous mobile device first enters an unknown environment, it traverses the unknown environment along a certain path to construct an environmental map; alternatively, it can also traverse the unknown environment while performing tasks during the initial task execution to construct an environmental map. It should be noted that during a single traversal, the autonomous mobile robot may traverse the entire environment, or it may only traverse a portion of the environment due to various reasons, such as inaccessibility of certain areas or insufficient battery power.

[0031] In either case, for the traversed environmental area, an environmental map corresponding to that area can be constructed by combining the walking trajectory of the autonomous mobile device during the traversal. For ease of distinction and description, this environmental map is referred to as the first environmental map. Specifically, during the traversal, sensors on the autonomous mobile device can be used to collect surrounding environmental information, and the first environmental map can be constructed based on the collected environmental information. In one optional embodiment, the autonomous mobile device is equipped with a laser sensor, so laser SLAM technology can be used to construct the first environmental map. In another optional embodiment, the autonomous mobile device is equipped with a visual sensor, such as a monocular camera or a binocular camera, so visual SLAM technology can be used to construct the first environmental map. Optionally, the first environmental map can be a raster map, but it is not limited to this. It should be noted that the first environmental map corresponding to the traversed environmental area can be obtained through the method described above; other methods can also be used. For example, if a first environmental map has already been constructed, it can be obtained directly.

[0032] In this embodiment, for a traversed environmental area, a corresponding first environmental map can be obtained. The first environmental map includes unoccupied areas within that environmental area. Unoccupied areas refer to areas formed by connecting locations not occupied by obstacles. Optionally, unoccupied areas can be areas that the autonomous mobile device can pass through, such as connected areas with a width greater than the maximum width and height of the autonomous mobile device, like a living room area, balcony area, or bedroom area; alternatively, unoccupied areas may also be areas that the autonomous mobile device cannot pass through, such as areas under sofas or beds, and gaps between adjacent pieces of furniture. In one optional embodiment, such as... Figure 2a As shown, the first environmental map already contains the boundary outlines of unoccupied areas. In another optional embodiment, the first environmental map includes occupied and unoccupied location information within the environmental area. The area formed by connecting the unoccupied location information is the unoccupied area, but the boundary outlines of the unoccupied areas are not marked on the first environmental map. In this optional embodiment, based on the occupied and unoccupied location information recorded in the first environmental map, the unoccupied areas within the environmental area can be marked on the first environmental map. One form of the unoccupied area presented on the first environmental map is as follows: Figure 2a As shown. In this embodiment, when the first environment map includes occupied and unoccupied location information in the environment area, the specific implementation of marking unoccupied areas on the first environment map is not limited. Examples are given below:

[0033] In an alternative embodiment, a boundary contour can be added between the occupied and unoccupied location information. Figure 2a The curve in the diagram represents the boundary contour between occupied and unoccupied location information; the area enclosed by this boundary contour (i.e., the area within the boundary contour) is the unoccupied area. Figure 2a The image only shows the boundary outline and the enclosed internal area; the area outside the boundary outline is occupied by obstacles. Figure 2a The diagram is not shown. Further, in one method of adding boundary contours between occupied and unoccupied location information, map points on the first environment map can be traversed. For each traversed map point, its adjacent map points are searched for those with different occupation states. If a match is found, a line segment is added between the two map points; if no match is found, it means that the map point and its adjacent map points have the same occupation state, or are all occupied, or are all unoccupied. After all map points have been traversed, the line segments between adjacent map points with different occupation states are connected to obtain the boundary contour. Figure 2aThe boundary contour shown is illustrated. Here, a map point is the smallest map unit on the first environment map; for example, in a raster map, one grid cell represents one map point. Alternatively, in another way of adding a boundary contour between occupied and unoccupied location information, the first environment map can be displayed, presenting the occupied and unoccupied location information to the user. The user can use their finger or stylus to slide between the occupied and unoccupied location information to initiate the operation of adding a boundary contour. Responding to the user's sliding operation, a curve is added along the user's sliding trajectory, resulting in a contour as shown below. Figure 2a The boundary contour shown.

[0034] In another optional embodiment, unoccupied positions can be filled with the same color, and the colors of all unoccupied positions can be connected to form a colored area, which is the unoccupied area. Optionally, map points on the first environment map can be traversed. For each traversed map point, its occupancy status can be determined. If it is occupied, no action is taken, and the traversal continues to its adjacent map points; if it is unoccupied, it is filled with color, until all map points have been traversed to obtain the unoccupied area.

[0035] In another optional embodiment, the occupied positions can be filled with the same color, and the colors of all occupied positions can be connected to each other to obtain a region filled with color. This region is the unoccupied region; correspondingly, the region on the first environment map that is not filled with color is the unoccupied region. Optionally, the map points on the first environment map can be traversed. For each traversed map point, its occupation status can be determined. If it is unoccupied, no processing is done, and the traversal continues to its adjacent map points; if it is occupied, it is filled with color, until all map points have been traversed, thus obtaining the non-communication area and the unoccupied area.

[0036] In this embodiment, for unoccupied areas included in the first environmental map, the area dividing lines in the unoccupied areas can be identified by combining the positions of spatial dividers within the environmental area. Here, a spatial divider refers to an object existing within the environmental area that can divide the environmental area into different regions, such as interior walls, partitions, various doors, shelves, workstations, etc. The implementation of spatial dividers will vary depending on the application scenario. For example, in a home scenario, the environmental area can be a home area, and the spatial dividers within the home can be at least one of the following: entrance door, room doors, interior walls, and partitions. The area dividing lines are the mappings of the spatial dividers existing in the environmental area onto the first environmental map, and the area dividing lines can divide the unoccupied areas into multiple sub-regions. Figure 2aTaking the unoccupied area shown as an example, the area dividing line of that unoccupied area can be obtained, such as... Figure 2b As shown. In Figure 2b In this context, if the spatial divider is the room door, then the sub-regions divided by the area dividing lines are individual rooms.

[0037] In an optional embodiment, the spatial dividers and their locations in the environmental area can be known in advance. For example, the spatial dividers in the environmental area can be determined by the floor plan corresponding to the environmental area, and their locations can be determined. Then, according to the coordinate transformation relationship between the floor plan and the first environmental map, the location of the spatial dividers in the floor plan can be transformed into the first environmental map, completing the mapping of the spatial dividers to the first environmental map. The mapping result is the area dividing line in the first environmental map.

[0038] In another optional embodiment, the autonomous mobile device is equipped with a visual sensor, which can collect environmental images of the area as the autonomous mobile device traverses the environment. Based on this, the environmental images collected by the visual sensor can be acquired; spatial segments in the environmental images and their position coordinates in the environmental images can be identified; then, the position coordinates of the identified spatial segments in the environmental images can be mapped onto a first environmental map to obtain the area division lines of the unoccupied areas. The method for identifying spatial segments in the environmental images can be any of the following:

[0039] Semantic-based recognition methods:After acquiring the environmental image, semantic segmentation can be performed to obtain semantic labels for at least one object in the environmental image. Based on the semantic labels of at least one object and the semantic labels of known spatial segmenters, the spatial segmenters present in the environmental image are identified. Semantic segmentation, when processing the environmental image, can be specific at the pixel level; that is, each pixel in the environmental image is assigned to a specific object category. Optionally, a semantic segmentation model can be pre-trained. The environmental image is input into the semantic segmentation model to obtain the objects contained in the environmental image and their semantic labels. The semantic segmentation model can be a convolutional neural network model, a fully convolutional neural network (FCN) model, or a fully convolutional DenseNet model, etc. In this embodiment, the semantic labels of known spatial segmenters are pre-maintained. After the semantic segmentation model outputs each object and its semantic label in the environmental image, the semantic labels of each object can be matched with the semantic labels of the known spatial segmenters. If a match is found, the object corresponding to the matched semantic label is determined to be the spatial segmenter. For example, in a home environment, known spatial segmentation objects include room doors, partitions, and interior walls. Suppose that objects such as room doors, tables, chairs, televisions, wardrobes, sofas, and coffee tables are identified from the environmental image. By matching the semantic information of each object with the semantic information of the known spatial segmentation objects, and finding that the room door is matched, it can be determined that the room door in the environmental image is a specific spatial segmentation object.

[0040] Model-based recognition methods: For specific application scenarios, objects that can serve as spatial segmenters can be collected. These objects and their images are then used as training samples to train the model, resulting in a segmenter recognition model. Based on this, after acquiring an environmental image, it can be input into the segmenter recognition model for identification. This allows the model to determine whether spatial segmenters exist in the environmental image. Furthermore, if spatial segmenters are present in the environmental image, the model can output what those segmenters are.

[0041] Geometric feature-based recognition methods:After acquiring the environmental image, features can be extracted from at least one object contained in the environmental image to obtain the geometric features of at least one object. In this embodiment, the geometric features of known spatial segments are pre-maintained. After obtaining the geometric features of each object in the environmental image, the geometric features of each object can be matched with the geometric features of the known spatial segments. If a match is found, the object corresponding to the matched geometric features can be identified as a spatial segment. For example, in a home environment, known spatial segments include room doors, and the length, width, and height of the room doors are known. Suppose that objects such as room doors, tables, chairs, televisions, wardrobes, sofas, and coffee tables, along with their geometric dimensions, are identified from the environmental image. The geometric dimensions of each object are matched with the geometric dimensions of the known spatial segments. If the geometric dimensions of the room door are found to match, then the room door in the environmental image can be identified as a specific spatial segment.

[0042] Furthermore, in the three methods described above, the environmental image containing the spatial segment can be one or multiple images. When there are multiple environmental images containing the spatial segment, the position coordinates of the spatial segment in each environmental image can be mapped onto the first environmental map, thus obtaining multiple sets of position coordinates. Then, the average position coordinate of these multiple sets is calculated, and the line segment corresponding to the average position coordinate in the first environmental map is used as the region segmentation line in the unoccupied area. This method of using multiple environmental images helps improve the accuracy of the calculated region segmentation line. Additionally, the number of region segmentation lines existing in the unoccupied area can be one or multiple. These region segmentation lines can divide the unoccupied area into multiple sub-regions, such as... Figure 2b As shown. In Figure 2b In the example shown, the entire unoccupied area is divided into 9 sub-areas, and marked with the numbers 0-8.

[0043] For users, Figure 2b The environmental map shown is not intuitive, and it is unclear which part of the environment each sub-region corresponds to. In this embodiment, after obtaining multiple sub-regions, external walls are added to these sub-regions to obtain a second environmental map, as shown below. Figure 2cAs shown. It should be noted that the environmental area in this embodiment comes from a building, such as a shopping mall, shop, home, supermarket, stadium, airport, or train station. The walls are a crucial part of the building to which the environmental area belongs. Walls are divided into exterior and interior walls. Interior walls largely determine the building's internal structure, while exterior walls determine the building's external outline. In this embodiment, adding exterior walls to sub-areas yields a second environmental map containing multiple sub-areas and their exterior walls. This environmental map resembles a floor plan; users can intuitively understand which area in the actual environment each sub-area corresponds to in the second environmental map by combining the exterior walls of each sub-area. This allows for more convenient, intuitive, and accurate interactive control of autonomous mobile devices through the environmental map, improving the quality of human-computer interaction based on the environmental map.

[0044] In this embodiment, the implementation of adding exterior walls to multiple sub-regions is not limited to this method. In one optional embodiment, for each sub-region, feature information corresponding to that sub-region can be extracted from the environmental image collected by the visual sensor, and this feature information can be added to the corresponding position in the first environmental map; a regular outline is added around each sub-region as an exterior wall to obtain a second environmental map; wherein each sub-region and its corresponding feature information all fall within its corresponding outline, and the outlines of different sub-regions do not overlap. In another optional embodiment, the user can understand the feature information of each sub-region on-site, and input the feature information of each sub-region into the autonomous mobile device through the interactive interface provided by the autonomous mobile device. The autonomous mobile device stores the feature information of each sub-region locally; in this way, the autonomous mobile device can directly obtain the feature information of each sub-region from the local device and add it to the corresponding position in the first environmental map; a regular outline is added around each sub-region as an exterior wall to obtain a second environmental map; wherein each sub-region and its corresponding feature information all fall within its corresponding outline, and the outlines of different sub-regions do not overlap. The regular outline can be a regular polygon such as a rectangle, square, triangle, pentagon, or hexagon. The feature information extracted from environmental images includes, but is not limited to: geometrically based feature points, semantically based contour points, and line features. Geometrically based feature points, such as corner points, can be extracted using algorithms such as ORB (Oriented Fast and Rotated Brief) and Scale-invariant Feature Transform (SIFT). Semantically based contour points can be the contour features of various objects in the environmental image, such as tables, chairs, sofas, and beds. Line features refer to locally salient features in the image, such as features with large gray-level gradient changes, which often appear in indoor environmental images at corners, object edges, or corners of adjacent walls.

[0045] In this application embodiment, the implementation of adding a regular outline to each sub-region is not limited to this method. In one optional embodiment, for each sub-region, when adding a regular outline to the sub-region, the boundary outline of the sub-region can be obtained, and the general shape of the boundary outline can be determined; for example, the boundary outline of the sub-region is generally a rectangle or a square; then, using the general shape of the boundary outline as a reference, a regular outline is added to the periphery of the sub-region. In another optional embodiment, for each sub-region, when adding a regular outline to the sub-region, the sub-region and its boundary outline can be displayed separately on the display screen; after the user sees the sub-region and its boundary outline, they can determine the general shape of the boundary outline of the sub-region, and then use the general shape of the boundary outline as a reference to manually draw a regular polygon on the periphery of the sub-region; the autonomous mobile device can respond to the user's operation of drawing a regular polygon on the display screen, and add a regular polygon as an outline to the periphery of the sub-region along the movement trajectory on the display screen. In another alternative embodiment, after dividing the unoccupied area into multiple sub-regions, a first environmental map is displayed on the screen. The first environmental map includes multiple sub-regions. After seeing the first environmental map, the user can determine the approximate shape of each sub-region and its boundary contour. Then, using the approximate shape of the boundary contour of each sub-region as a reference, regular polygons are manually drawn around each sub-region in sequence. The autonomous mobile device can respond to the user's operation of drawing regular polygons on the screen and add regular polygons around each sub-region as its outline along the movement trajectory on the screen in sequence.

[0046] Further, optionally, after adding a regular outline to the perimeter of each sub-region, the gaps between the outlines of multiple sub-regions can be filled to align with the exterior walls of the multiple sub-regions. For example, in the case of... Figure 2c After filling the gaps in the second environment map shown, we can obtain... Figure 2d The second environment map shown is more aesthetically pleasing and better suited to the user's intuitive experience.

[0047] It should be noted here that, Figure 2c and Figure 2d In this design, different sub-regions correspond to different color depths to facilitate differentiation between them. In practical applications, sub-regions may or may not have color fills; there is no restriction on this.

[0048] Further optional, in Figure 2c or Figure 2d The second environmental map shown includes multiple sub-regions and the exterior walls of each sub-region. In addition, other objects contained in each sub-region and on the exterior walls can be identified from the environmental images collected by the visual sensor, such as windows or hanging murals.

[0049] Further optional, in Figure 2c or Figure 2d The second environmental map shown can also display the area of ​​each sub-region, making it more like a floor plan and closer to the user's intuitive understanding. In one optional embodiment, the floor area of ​​each sub-region can be obtained from the actual floor plan or architectural drawings corresponding to the environmental region and provided to the autonomous mobile device, which then marks the floor area of ​​each sub-region on the second environmental map. In another optional embodiment, after adding exterior walls to each sub-region, the floor area of ​​each sub-region can be estimated based on the shape and size of the exterior walls; then, the floor area of ​​each sub-region is marked on the second environmental map.

[0050] Alternatively, in order to more accurately estimate the unit area of ​​each sub-region based on the shape and size of the exterior walls, when adding exterior walls to each sub-region, the thickness of the exterior walls in the actual building scene can be considered. According to the scale of the second environment map, exterior walls that can cover each sub-region and its corresponding feature information can be reasonably added around each sub-region.

[0051] Further optional, in Figure 2c or Figure 2d In the second environmental map shown, each sub-region represents an unoccupied portion of the environmental area. In addition, it may include information on occupied locations within that environmental area. Thus, autonomous mobile devices can directly perform positioning, navigation, or obstacle avoidance based on the occupied and unoccupied location information included in the second environmental map.

[0052] In this embodiment, after obtaining the second environment map, the autonomous mobile device can output the second environment map so that the user can interact with the autonomous mobile device through the second environment map. The methods for outputting the second environment map include, but are not limited to, the following:

[0053] Output method 1: If the mobile device is equipped with a display screen, the second environmental map can be displayed on the screen for user use.

[0054] Output method 2: The autonomous mobile device is equipped with a projection model. The projection module can be used to project the second environment map onto a screen or wall for display by the user.

[0055] Output Method 3: The autonomous mobile device can be bound to the user's terminal device. The user's terminal device has an application (APP) installed to control the autonomous mobile device. The autonomous mobile device can send the second environment map to the user's terminal device, and the terminal device will display the second environment map to the user for use.

[0056] For users, interactive control of autonomous mobile devices can be achieved through a second environmental map. Specifically, users can interact with and control their autonomous mobile devices using the second environmental map displayed on the terminal device; alternatively, they can interact with and control the autonomous mobile device using a second environmental map projected onto a screen or wall; or they can interact with and control the autonomous mobile device using a second environmental map displayed on the autonomous mobile device itself. These will be explained in detail below:

[0057] Interactive control method 1 based on the second environment map: After obtaining the second environmental map, the autonomous mobile device saves it locally. Furthermore, when a user wishes to interactively control the autonomous mobile device using the second environmental map, they can issue a command to display it. For example, the user can issue this command via voice, through an app on their terminal device, or via a physical or virtual button provided by the autonomous mobile device. Regardless of the method, upon receiving the user's command, the autonomous mobile device can display the second environmental map on its screen. After viewing the second environmental map, the user can determine the approximate or precise location of the object to be controlled in the actual environment based on the sub-regions and their exterior walls displayed on the map, and the relative positions of objects to the exterior walls in the actual environment. Then, the user can issue a trigger operation on the second environmental map, such as clicking on a map location to instruct the autonomous mobile device to perform a specified operation on the target object in the actual environment corresponding to that map location. For autonomous mobile devices, they can respond to triggered operations on a second environment map and perform specified operations on target objects in the actual environment that correspond to the triggered map location.

[0058] Depending on the application scenario, the triggered map location, the corresponding target object, and the specified operation will vary. The following example uses a home scenario, which includes the entrance door, room doors, interior walls, exterior walls, and partitions. These entrance doors, room doors, interior walls, exterior walls, and partitions divide the entire home environment into multiple sub-areas. These sub-areas include, but are not limited to, one or more of the following: rooms (such as master bedroom, secondary bedroom, children's room), kitchen, bathroom, living room, balcony, study, or gym. Additionally, windows are typically located on the exterior walls of a home environment. Using a home robotic vacuum cleaner as an example, the process of a user interacting with and controlling the robotic vacuum cleaner through a second environmental map similar to a floor plan is illustrated.

[0059] To free up users' hands and improve cleaning efficiency and quality, users purchase a robotic vacuum cleaner equipped with laser and vision sensors. Upon initial power-on, the robot traverses the entire home environment, using the laser sensor to collect obstacle information in the traversed areas and the vision sensor to capture environmental images of those areas. After traversing the area, the robot uses its built-in SLAM system to construct an indoor grid map, i.e., a first environmental map. This first environmental map can be constructed using either laser SLAM or visual SLAM technology. Based on the first environmental map, a second environmental map, similar to a floor plan map, is constructed using the method described in the aforementioned embodiment. Users can interact with the robot through the second environmental map, instructing it to perform cleaning and / or monitoring tasks.

[0060] Scenario Example A1:

[0061] The robotic vacuum cleaner is equipped with a vision sensor, which can provide users with various monitoring services. In this embodiment, the user wants the robotic vacuum cleaner to assist in monitoring the status of the front door to avoid home security problems caused by forgetting to close the door or by door malfunctions. Based on this, the user can enable the monitoring service of the robotic vacuum cleaner. Specifically, the user instructs the robotic vacuum cleaner to display a second environmental map via voice or touch. Since the second environmental map is similar to the home floor plan, the user can easily and intuitively determine the location of the front door on the second environmental map; then, the user can initiate a trigger operation on that map location, such as clicking, double-clicking, long-pressing, or touching the map location.

[0062] In response to a trigger operation on the map location, the robot vacuum cleaner identifies the service associated with the trigger operation as a monitoring service and determines that the target object corresponding to the map location in the actual environment is the front door. Then, it outputs a confirmation message to the user via voice, such as "Please confirm whether to monitor the front door." Upon receiving the user's confirmation voice signal, the robot vacuum cleaner activates its vision sensor and moves from its current position to the target location near the front door. The target location is selected based on the front door falling within the field of view of the vision sensor.

[0063] Then, at the target location, a visual sensor continuously acquires images including the entrance door, and analyzes the acquired images to determine if the entrance door is open or if there is any abnormality. When an open or partially closed entrance door is detected, or if there is an abnormality, a warning message is output to the user via voice to alert them to the situation. The identification of whether the entrance door is closed or abnormal can be performed by the server and the server returning the identification result to the robot vacuum cleaner; alternatively, it can be performed by the robot vacuum cleaner itself. Regardless of who performs the identification operation, optionally, a recognition model can be pre-trained to identify whether the entrance door is closed or abnormal; alternatively, images of the entrance door in a closed or normal state can be pre-saved as reference images. The acquired images including the entrance door state are then compared with the pre-saved reference images to identify whether the entrance door is closed or abnormal. Possible abnormalities of the entrance door include, but are not limited to: the entrance door being violently impacted, someone looking into the house through the entrance door during monitoring, and someone entering the house through the entrance door during monitoring. Alternatively, the robot vacuum cleaner can also pre-scan and save images of family members, and compare the images of people entering the house through the front door or looking into the house during the monitoring period with the pre-saved images of family members; if a stranger is detected, a warning message will be issued in time.

[0064] Scenario Example A2:

[0065] In practical applications, users may not want robot vacuums to enter specific rooms to perform tasks. For example, when a child is sleeping, the user doesn't want the robot vacuum to enter the child's room to perform tasks, so as not to disturb the child's rest. Another example is when ventilating a room by opening a window, it's necessary to close the room door, and the user doesn't want the robot vacuum to enter that room. To address these needs, robot vacuums can provide a room door closing service. This "closing" is a virtual closing method; the robot vacuum will not actually enter the room corresponding to that door.

[0066] When a user does not want the robot vacuum to enter a target room, they can activate the room door closing service provided by the robot vacuum. Specifically, the user can instruct the robot vacuum to display a second environmental map via voice or touch. Since the second environmental map is similar to a floor plan of the home, the user can easily and intuitively determine the sub-area corresponding to the target room on the second environmental map, and can also determine the map position of the target room's door on the second environmental map; then, the user can initiate a trigger operation on that map position, such as clicking, double-clicking, long-pressing, or touching the map position.

[0067] In response to a trigger operation on the map location, the robot vacuum cleaner identifies the service associated with the trigger operation as the room door closing service. Optionally, it can output a confirmation message to the user via voice, such as "Please confirm whether to close this room door." The user confirms via voice. After receiving the user's confirmation voice signal, the robot vacuum cleaner sets up a virtual wall at the location corresponding to the room door on the second environmental map. In this way, when the robot vacuum cleaner moves to the room door, it will adjust its movement direction because it cannot pass through the door, thus preventing it from entering the target room and achieving the effect of closing the room door.

[0068] Optionally, the user can trigger the map location again, instructing the door to be released from its closed position. In response to this second trigger, the robot vacuum removes the virtual wall at the location corresponding to the door on the second environment map, thus releasing the door from its closed position. After the door is released, the robot vacuum can enter the target room through the door to perform cleaning or monitoring tasks.

[0069] Scenario Example A3:

[0070] Similar to scenario A1 above, the robotic vacuum cleaner is equipped with a visual sensor, which can provide users with various monitoring services. In this embodiment, the user wants the robotic vacuum cleaner to help monitor the status of windows to avoid home security problems caused by forgetting to close windows or windows malfunctioning. For example, when children are at home, they are often active and climb onto windows to play, which parents cannot track in real time; in this case, the robotic vacuum cleaner's monitoring service can be activated. Another example is when the user is away and wants to open the window for ventilation but is worried about burglaries through the window; in this case, the robotic vacuum cleaner's monitoring service can also be activated.

[0071] Specifically, when users enable the monitoring service of the robot vacuum cleaner, they can instruct the robot vacuum cleaner to display a second environmental map through voice or touch. Since the second environmental map is similar to the home floor plan, users can easily and intuitively determine the map position of the window on the second environmental map; then, they can initiate a trigger operation on that map position, such as clicking, double-clicking, long-pressing, or touching the map position.

[0072] In response to a trigger operation on the map location, the robot vacuum cleaner identifies the service associated with the trigger operation as a monitoring service and determines that the target object corresponding to the map location in the actual environment is a window. Then, it outputs a confirmation message to the user via voice, such as "Please confirm whether to monitor the window." Upon receiving the user's confirmation voice signal, the robot vacuum cleaner activates its vision sensor and moves from its current position to the target location near the window. The target location is selected based on whether the window falls within the field of view of the vision sensor.

[0073] Then, at the target location, a visual sensor continuously acquires images including those of the window, and analyzes the acquired images to determine if the window is open or if there is any abnormality. If an open window, children playing on the window, or someone entering the room from outside is detected, a warning message is output to the user via voice, or a warning message is sent to the user's device via SMS or in-app message to alert the user that the window is open or there is an abnormality, urging the user to pay attention. The identification of whether the window is closed or if there is an abnormality can be performed by the server and the identification result returned to the robot vacuum cleaner; alternatively, it can be performed by the robot vacuum cleaner itself. For specific identification methods, please refer to the description in the aforementioned scenario embodiment A1, which will not be repeated here.

[0074] Scenario Example A4:

[0075] This embodiment of the robotic vacuum cleaner supports an edge-cleaning mode, meaning the robot automatically cleans along the edges of walls. Based on this, users can specify which wall edge the robot should clean along, depending on the situation. For example, if a user wants to clean while tidying up, but items are piled up along the edge of a wall that haven't been cleared yet, the user can choose to have the robot clean along other wall edges, giving them time to clear the items without interrupting the cleaning process. Users can instruct the robot to perform edge-cleaning tasks via voice, display screen, or physical buttons. The robot receives the user's cleaning command and displays a second environmental map, allowing the user to select which wall needs edge-cleaning. Since the second environmental map resembles a floor plan, users can easily and intuitively determine the location of the interior or exterior wall requiring edge-cleaning on the second environmental map. Then, a trigger operation can be initiated for that location, such as clicking, double-clicking, long-pressing, or tapping the map position.

[0076] In response to a trigger operation on the map location, the robotic vacuum cleaner determines whether the target object corresponding to that location in the actual environment is an interior wall or an exterior wall. Optionally, the robotic vacuum cleaner can also output confirmation information to the user via voice, such as "Please confirm whether to perform edge cleaning along the selected wall." The user confirms via voice; after receiving the user's confirmation voice signal, the robotic vacuum cleaner moves from its current position to the interior or exterior wall selected by the user and begins performing the edge cleaning task.

[0077] Scenario Example A5:

[0078] This embodiment of the robotic vacuum cleaner supports a spot cleaning mode, meaning the robot can clean a sub-area of ​​the environment selected by the user. Based on this, the user can instruct the robot to clean a specific sub-area according to the actual situation. For example, the user can instruct the robot to perform cleaning tasks in specific sub-areas such as a children's room, pet room, kitchen, or bathroom. Therefore, the user can instruct the robot to perform spot cleaning tasks via voice, display screen, or physical buttons. At this time, the robot receives the cleaning command from the user and displays a second environmental map, allowing the user to select the sub-area to be cleaned, such as a children's room, pet room, kitchen, or bathroom. Since the second environmental map is similar to a floor plan, the user can easily and intuitively determine the map location of the sub-area to be cleaned on the second environmental map; then, the user can initiate a trigger operation on that map location, such as clicking, double-clicking, long-pressing, or touching the map location.

[0079] In response to a trigger operation on the map location, the robotic vacuum cleaner determines the target object in the actual environment corresponding to that map location as a specific sub-area such as a children's room, pet room, kitchen, or bathroom. Optionally, the robotic vacuum cleaner can also output confirmation information to the user via voice, such as "Please confirm whether to perform spot cleaning on the selected sub-area." The user confirms via voice; after receiving the user's confirmation voice signal, the robotic vacuum cleaner moves from its current location to the selected sub-area and begins performing the spot cleaning task.

[0080] Further optionally, in addition to the scenarios A4 and A5 described above, users can also instruct the robot vacuum cleaner to perform cleaning tasks in other ways. For example, users can directly tell the robot vacuum cleaner to "perform a cleaning task." After receiving the instruction, the robot vacuum cleaner can perform the cleaning task using a random cleaning mode or a bow-shaped cleaning mode. Regardless of the method used by the user to instruct the robot vacuum cleaner to perform the cleaning task, and regardless of the mode used by the robot vacuum cleaner, during the cleaning process, the sub-area corresponding to the current cleaning area on the second environmental map can be obtained and recorded as the target sub-area. Then, based on the exterior walls of the target sub-area and the robot vacuum cleaner's movement trajectory, the area of ​​the apartment that the robot vacuum cleaner has cleaned is calculated; the cleaned area is displayed in the target sub-area of ​​the second environmental map. Furthermore, the robot vacuum cleaner's movement trajectory can also be displayed in the target sub-area of ​​the second environmental map. The area of ​​the apartment that the robot vacuum cleaner has cleaned is greater than or equal to its actual cleaned area. The actual cleaning area is calculated based on the robot vacuum's movement trajectory; it represents the area the robot actually traversed. However, the cleaned area includes not only the actual cleaning area but also areas that the robot cannot access or clean. For example, consider a robot vacuum cleaning a living room. The area occupied by the coffee table, TV cabinet, and sofa is approximately 10 square meters. This 10 square meters is beyond the robot's reach. The remaining area, let's say 20 square meters, is the area the robot can actually clean. Therefore, the actual cleaning area in the living room, calculated from the robot's movement trajectory, is approximately 20 square meters. However, considering the exterior walls of the living room and the robot's movement trajectory, the actual cleaned area is approximately 30 square meters.

[0081] Therefore, based on the second environmental map similar to the above-mentioned floor plan, users can more conveniently, intuitively, and flexibly control the robot vacuum cleaner. At the same time, the second environmental map also enriches the functions of the robot vacuum cleaner, so that the robot vacuum cleaner is no longer limited to providing cleaning services, but can also provide users with various monitoring services.

[0082] Interactive control method 2 based on the second environment map: After obtaining the second environmental map, the autonomous mobile device will both save it locally and send it to the user's terminal device. Furthermore, when the user wishes to interactively control the autonomous mobile device using the second environmental map, they can do so through their terminal device. Based on this, such as... Figure 3 As shown, a method for controlling an autonomous mobile device, described from the perspective of a terminal device, includes:

[0083] 31. In response to a display operation, display a second environment map, which includes multiple passable sub-areas and their outer walls within the corresponding environment area.

[0084] 32. In response to a trigger operation on the second environment map, send a control command to the autonomous mobile device so that the autonomous mobile device performs a specified operation on the target object in the actual environment corresponding to the triggered map location; wherein, the target object corresponds to the triggered map location on the second environment map, and the map location is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0085] Users can initiate the action of displaying a second environmental map on their terminal device. For example, a user can click the app icon on the desktop to open the app controlling the autonomous mobile device, at which point the second environmental map can be displayed. Alternatively, the app's settings page may include a configuration option for users to choose whether to display the second environmental map, such as a toggle button, which the user can use to issue the action of displaying the second environmental map. Users can also instruct the terminal device to display the second environmental map via voice commands. Regardless of the method, after receiving the user's command, the terminal device can display the second environmental map on its screen.

[0086] After viewing the second environment map, the user can determine the approximate or precise location of the object to be controlled in the actual environment within the second environment map based on the sub-regions and their exterior walls displayed on the map, and the relative positions of objects to the exterior walls in the actual environment. Then, a trigger operation can be issued on the second environment map; for example, clicking on a map location instructs the autonomous mobile device to perform a specified operation on the target object in the actual environment corresponding to that map location. The terminal device responds to the trigger operation on the second environment map by sending a control command to the autonomous mobile device, causing it to perform the specified operation on the target object in the actual environment corresponding to the triggered map location.

[0087] Depending on the application scenario, the triggered map location, the corresponding target object, and the specified operation will vary. The following example uses a home scenario, which includes the entrance door, room doors, interior walls, exterior walls, and partitions. These entrance doors, room doors, interior walls, exterior walls, and partitions divide the entire home environment into multiple sub-areas. These sub-areas include, but are not limited to, one or more of the following: rooms (such as master bedroom, secondary bedroom, children's room), kitchen, bathroom, living room, balcony, study, or gym. Additionally, windows are typically located on the exterior walls of a home environment. Using a home robotic vacuum cleaner as an example, the process of a user interacting with and controlling the robotic vacuum cleaner through a second environmental map similar to a floor plan is illustrated.

[0088] To free up users' hands and improve cleaning efficiency and quality, users purchase a robotic vacuum cleaner equipped with laser and vision sensors. Upon initial power-on, the robot traverses the home environment, using the laser sensor to collect obstacle information in the traversed areas and the vision sensor to capture environmental images of those areas. After traversing the area, the robot uses its built-in SLAM system to construct an indoor grid map, i.e., a first environmental map. This first environmental map can be constructed using either laser SLAM or visual SLAM technology. Based on the first environmental map, a second environmental map, similar to a floor plan map, is constructed using the method described in the aforementioned embodiment. Users can interact with the robot through the second environmental map, instructing it to perform cleaning and / or monitoring tasks.

[0089] It should be noted that the robot vacuum cleaner can first traverse a portion of the home area and build a second environmental map corresponding to that portion of the home area; then, in subsequent tasks, if a new home area is traversed, an environmental map can be built for the new home area, and the existing second environmental map can be updated to gradually obtain a second environmental map of the entire home environment.

[0090] After constructing a second environmental map corresponding to the home environment, the robot vacuum cleaner sends this map to the user's terminal device. The user's terminal device has an app installed to control the robot vacuum cleaner. Through this app, the user can view the second environmental map and issue various control commands to the robot vacuum cleaner.

[0091] Scenario Example B1:

[0092] The robotic vacuum cleaner is equipped with a vision sensor, which can provide users with various monitoring services. In this embodiment, the user wants the robotic vacuum cleaner to assist in monitoring the status of the front door to avoid home security problems caused by forgetting to close the door or by door malfunctions. Based on this, the user can enable the robotic vacuum cleaner's monitoring service. Specifically, the user opens the app on the terminal device, and the app displays a second environmental map. Since the second environmental map is similar to the home floor plan, the user can easily and intuitively determine the location of the front door on the second environmental map; then, the user can initiate a trigger operation on that map location, such as clicking, double-clicking, long-pressing, or touching the map location.

[0093] In response to a trigger operation on the map location, the user's terminal device determines the target object in the actual environment corresponding to that map location as the front door. Optionally, above the second environment map, a confirmation message is displayed to the user in the form of a floating layer or a message pop-up, such as "Please confirm whether to monitor the front door," and "Yes" and "No" buttons are shown. In response to the user clicking the "Yes" button, the terminal device sends a control command to the robot vacuum cleaner, instructing it to check if the front door is abnormal or closed.

[0094] After receiving control commands from the terminal device, the robotic vacuum cleaner recognizes the need to check the front door for any abnormalities or ensure it is properly closed. It then activates its vision sensors and moves from its current position to a target location near the front door, selecting a location where the front door falls within the vision sensor's field of view. At this target location, the vision sensor continuously captures images including the front door and analyzes them to determine if the door is open or if there are any abnormalities. If the front door is found to be open, not fully closed, or otherwise abnormal, the robot will issue a warning to the user via voice or send an alarm message to the terminal device via SMS or in-app message, alerting the user to the door's closure or abnormality and requesting their attention.

[0095] Scenario Example B2:

[0096] In practical applications, users may not want robot vacuums to enter specific rooms to perform tasks. For example, when a child is sleeping, the user doesn't want the robot vacuum to enter the child's room to perform tasks, so as not to disturb the child's rest. Another example is when ventilating a room by opening a window, it's necessary to close the room door, and the user doesn't want the robot vacuum to enter that room. To address these needs, robot vacuums can provide a room door closing service. This "closing" is a virtual closing method; the robot vacuum will not actually enter the room corresponding to that door.

[0097] When a user does not want the robot vacuum to enter a target room, they can enable the room door closing service provided by the robot vacuum. Specifically, the user opens the app on their terminal device, and the app displays a second environmental map. Since the second environmental map is similar to a home floor plan, the user can easily and intuitively determine the sub-area corresponding to the target room on the second environmental map, and can also determine the map position of the target room's door on the second environmental map; then, the user can initiate a trigger operation on that map position, such as clicking, double-clicking, long-pressing, or touching the map position.

[0098] In response to a trigger operation on the map location, the user's terminal device may optionally display a confirmation message, such as "Please confirm whether to close the room door," above the second environment map in the form of a floating layer or a message pop-up, and display "Yes" and "No" buttons. In response to the user clicking the "Yes" button, the terminal device sends a control command to the robot vacuum cleaner, instructing it to provide the room door closing service.

[0099] After receiving the control command from the terminal device, the robot vacuum cleaner can identify that a room door needs to be closed. Therefore, it sets a virtual wall at the location corresponding to the room door on the second environmental map. When the robot vacuum cleaner moves towards the room door, it will adjust its direction because it cannot pass through, thus preventing it from entering the target room and achieving the effect of closing the room door.

[0100] Optionally, the user can trigger the map location again, instructing the door to be released from its closed position. In response to this second trigger, the terminal device sends a release command to the robot vacuum. Based on this release command, the robot vacuum removes the virtual wall at the location corresponding to the door on the second environmental map, thus releasing the door from its closed position. After the door is released, the robot vacuum can enter the target room through the door to perform cleaning or monitoring tasks.

[0101] Scenario Example B3:

[0102] Similar to scenario B1 above, the robotic vacuum cleaner is equipped with a visual sensor, which can provide users with various monitoring services. In this embodiment, the user wants the robotic vacuum cleaner to help monitor the status of windows to avoid home security problems caused by forgetting to close windows or windows malfunctioning. For example, when children are at home, they are often active and climb onto windows to play, which parents cannot track in real time; in this case, the robotic vacuum cleaner's monitoring service can be activated. Another example is when the user is away and wants to open the window for ventilation but is worried about burglaries through the window; in this case, the robotic vacuum cleaner's monitoring service can also be activated.

[0103] Specifically, the user opens the app on their terminal device, and the app displays a second environment map. Since the second environment map is similar to a home floor plan, the user can easily and intuitively determine the map position of the window on the second environment map; then, the user can initiate a trigger operation on that map position, such as clicking, double-clicking, long-pressing, or touching the map position.

[0104] In response to a trigger operation on the map location, the user's terminal device determines that the target object corresponding to that map location in the actual environment is a window. Optionally, above the second environment map, a confirmation message is displayed to the user in the form of a floating layer or a message pop-up, such as "Please confirm whether to monitor the window," and "Yes" and "No" buttons are displayed. In response to the user clicking the "Yes" button, the terminal device sends a control command to the robot vacuum cleaner, instructing it to check if the window is abnormal or closed.

[0105] After receiving control commands from the terminal device, the robot vacuum cleaner can identify the need to check for any abnormalities or closure of the window. It then activates its vision sensors and moves from its current position to a target location near the window, selecting a location where the window falls within the field of view of the vision sensors. At this target location, the vision sensors continuously acquire images including the front door and analyze them to determine if the door is open or if there are any abnormalities. If the door is found to be open, not fully closed, or otherwise abnormal, the robot vacuum cleaner will issue a warning to the user via voice or send an alarm message to the terminal device via SMS or in-app message, reminding the user that the door is open or there is an abnormality and requesting their attention.

[0106] Scenario Example B4:

[0107] This embodiment of the robotic vacuum cleaner supports an edge-cleaning mode, meaning the robot will automatically clean along the edges of walls. Based on this, users can specify which wall edge the robot should clean along, depending on the situation. For example, if a user wants to clean while tidying up a room, but items are piled up along the edge of a wall that haven't been cleared yet, the user can choose to have the robot clean along the edges of other walls, giving them time to clear the items without interrupting the cleaning process. To facilitate this, the user opens the app on their device, which displays a second environmental map. Since this second environmental map resembles a floor plan, users can easily and intuitively determine the location of the interior or exterior walls that need edge-cleaning on the second environmental map. Then, the user can initiate a trigger operation on that location, such as clicking, double-clicking, long-pressing, or tapping the map position.

[0108] In response to a trigger operation on the map location, the user's terminal device determines whether the target object corresponding to that map location in the actual environment is an interior wall or an exterior wall. Optionally, the terminal device can also output confirmation information to the user above the second environment map in the form of a floating layer or a message pop-up, such as "Please confirm whether to perform an edge cleaning task along the selected wall," and display "Yes" and "No" buttons. Responding to the user clicking the "Yes" button, the terminal device sends a control command to the robot vacuum cleaner, instructing it to perform an edge cleaning task on the selected interior or exterior wall. After receiving the control command from the terminal device, the robot vacuum cleaner recognizes the need to perform an edge cleaning task on the selected interior or exterior wall, and then moves from its current position to the selected interior or exterior wall to begin performing the edge cleaning task.

[0109] Scenario Example B5:

[0110] This embodiment of the robotic vacuum cleaner supports a spot cleaning mode, meaning the robot can clean a sub-area of ​​the environment selected by the user. Based on this, the user can instruct the robot to clean a specific sub-area according to the actual situation. For example, the user can instruct the robot to perform cleaning tasks in specific sub-areas such as a children's room, pet room, kitchen, or bathroom. Therefore, the user opens the app on their terminal device, which displays a second environmental map. The user can select the sub-area to be cleaned, such as a children's room, pet room, kitchen, or bathroom, through the second environmental map. Since the second environmental map is similar to a floor plan, the user can easily and intuitively determine the map location of the sub-area to be cleaned; then, a trigger operation can be initiated for that map location, such as clicking, double-clicking, long-pressing, or tapping the map location.

[0111] In response to a trigger operation on the map location, the user's terminal device determines that the target object corresponding to that map location in the actual environment is a specific environmental sub-area such as a children's room, pet room, kitchen, or bathroom. Optionally, the terminal device can also output confirmation information to the user above the second environmental map in the form of a floating layer or message pop-up, such as "Please confirm whether to perform a spot cleaning task on the selected environmental sub-area." In response to the user clicking the "Yes" button, the terminal device sends a control command to the robot vacuum cleaner, instructing it to perform a spot cleaning task on the selected environmental sub-area. After receiving the control command from the terminal device, the robot vacuum cleaner recognizes the need to perform a spot cleaning task on the selected environmental sub-area, and then moves from its current location to the selected environmental sub-area to begin performing the spot cleaning task.

[0112] Further optionally, in addition to the scenarios B4 and B5 described above, users can also instruct the robot vacuum cleaner to perform cleaning tasks in other ways. For example, users can directly tell the robot vacuum cleaner to "perform a cleaning task." After receiving the instruction to perform a cleaning task, the robot vacuum cleaner can perform the cleaning task using a random cleaning mode or a bow-shaped cleaning mode. Regardless of the method used by the user to instruct the robot vacuum cleaner to perform a cleaning task, and regardless of the mode used by the robot vacuum cleaner to perform the cleaning task, during the process of the robot vacuum cleaner performing the cleaning task, the sub-area corresponding to the current cleaning area on the second environmental map can be obtained and recorded as the target sub-area. Based on the outer wall of the target sub-area and the movement trajectory of the robot vacuum cleaner, the area of ​​the house that has been cleaned is calculated and returned to the terminal device. The terminal device receives the area of ​​the house that has been cleaned when the robot vacuum cleaner performs the cleaning task and displays the cleaned area in the target sub-area of ​​the second environmental map; wherein, the area of ​​the house that the robot vacuum cleaner has cleaned is greater than or equal to its actual cleaning area. For an explanation of the difference between the cleaned floor area and the actual cleaned area, please refer to the aforementioned embodiments, which will not be repeated here. Furthermore, the robot vacuum can also send its movement trajectory during the cleaning task to the terminal device; the terminal device can also receive the movement trajectory returned by the robot vacuum during the cleaning task and display the robot vacuum's movement trajectory in the target sub-area of ​​the second environmental map.

[0113] Therefore, based on the second environmental map similar to the above-mentioned floor plan, users can more conveniently, intuitively, and flexibly control the robot vacuum cleaner. At the same time, the second environmental map also enriches the functions of the robot vacuum cleaner, so that the robot vacuum cleaner is no longer limited to providing cleaning services, but can also provide users with various monitoring services.

[0114] Interactive control method 3 based on the second environment map:After obtaining the second environmental map, the autonomous mobile device saves it locally. The autonomous mobile device is equipped with a projection module and a projection remote control. When the user wants to interactively control the autonomous mobile device using the second environmental map, they can issue a command to the autonomous mobile device to display the second environmental map via the projection remote control. After receiving the display command from the projection remote control, the autonomous mobile device can project the second environmental map onto a white wall in the home. The location of this white wall can be preset in the autonomous mobile device; based on this, the autonomous mobile device moves to the vicinity of the white wall and projects the map onto it. Alternatively, the autonomous mobile device can also, after receiving the display command, find the nearest white wall in real time and project the map onto that wall. After viewing the second environmental map, the user can determine the approximate or precise location of the object to be controlled in the actual environment based on the sub-regions and their exterior walls displayed on the map, and the relative positions of objects to the exterior walls in the actual environment. Then, a trigger operation can be issued on the second environmental map via a projection remote control. For example, an infrared dot can be projected onto a specific location on the second environmental map to instruct the autonomous mobile device to perform a specified operation on the target object in the actual environment corresponding to that location. For the autonomous mobile device, the projection module or visual camera can capture the infrared dot on the second environmental map, determine the triggered map location, and execute the specified operation on the target object in the actual environment corresponding to that triggered map location.

[0115] Depending on the application scenario, the triggered map location, the corresponding target object, and the specified operation will vary. Taking a home scenario as an example, and assuming the autonomous mobile device is a home robotic vacuum cleaner, the user can instruct the vacuum cleaner to monitor the entrance door or window, provide door closing service, or perform edge cleaning or spot cleaning tasks through the second environmental map. These embodiments differ from the aforementioned scenario embodiments only in the method of user interaction control via the second environmental map; other implementation processes are the same or similar, and therefore can be referred to the aforementioned embodiments, and will not be repeated here.

[0116] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 11 to 14 can be device A; or the execution subject of steps 11-13 can be device A, and the execution subject of step 14 can be device B; and so on.

[0117] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0118] Figure 4 This is a schematic diagram of an environmental map building apparatus provided for an exemplary embodiment of this application. Figure 4 As shown, the device includes: an acquisition module 41, an identification module 43, and an addition module 44.

[0119] Module 41 is used to obtain the first environment map corresponding to the traversed environment area, and the first environment map includes unoccupied areas;

[0120] The identification module 43 is used to identify the area dividing line in the unoccupied area based on the position of the spatial divider in the environmental area. The area dividing line divides the unoccupied area into multiple sub-areas.

[0121] Add module 44 to add exterior walls to multiple sub-regions to obtain a second environment map, which includes multiple sub-regions and their exterior walls.

[0122] Further optional, such as Figure 4 As shown, the device may further include a marking module 42. When the first environmental map includes occupied and unoccupied location information in the environmental area, the marking module 42 is used to mark unoccupied areas on the first environmental map based on the occupied and unoccupied location information, providing a data basis for the identification module 43. Specifically, a boundary contour can be added between the occupied and unoccupied location information, and the area enclosed by the boundary contour is the unoccupied area.

[0123] For a detailed description of the functions implemented by the acquisition module 41, the marking module 42, the identification module 43, and the adding module 44, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0124] Figure 5 This is a schematic diagram of the structure of a control device provided for an exemplary embodiment of this application. (See diagram below.) Figure 5 As shown, the device includes a display module 51 and a transmission module 52.

[0125] Display module 51 is used to respond to display operations and display an environment map, which includes multiple passable sub-areas and their outer walls in the corresponding environment area;

[0126] The sending module 52 is used to send control commands to the autonomous mobile device in response to a trigger operation on the environmental map, so that the autonomous mobile device can perform a specified operation on a target object in the environmental area; wherein, the target object corresponds to the triggered map position on the environmental map, and the map position is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0127] For a detailed description of the functions implemented by the display module 51 and the sending module 52, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0128] Figure 6a This is a schematic diagram of the structure of an autonomous mobile device provided as an exemplary embodiment of this application. Figure 6a As shown, the autonomous mobile device includes: a device body 60, on which a memory 64 and a processor 66 are provided; furthermore, the device body 60 is also provided with a vision sensor 61, as well as some other sensors 62, such as an inertial navigation sensor, a laser sensor, an anti-collision sensor, etc.

[0129] Memory 64 is used to store computer programs and can be configured to store various other data to support operation on the autonomous mobile device. Examples of this data include instructions, messages, images, videos, etc., for any application or method used to operate on the autonomous mobile device.

[0130] The processor 65, coupled to the memory 64, is used to execute a computer program in the memory 64 for: obtaining a first environment map corresponding to the traversed environment area, the first environment map including unoccupied areas; identifying area dividing lines in the unoccupied areas based on the positions of spatial dividers in the environment area, the area dividing lines dividing the unoccupied areas into multiple sub-areas; adding outer walls to the multiple sub-areas to obtain a second environment map, the second environment map including the multiple sub-areas and their outer walls.

[0131] In an optional embodiment, the first environmental map includes occupied and unoccupied location information within the environmental area; based on this, the processor 65 is further configured to: mark unoccupied areas on the first environmental map based on the occupied and unoccupied location information. When the processor 65 marks unoccupied areas on the first environmental map, it is specifically configured to: add a boundary contour between the occupied and unoccupied location information, wherein the area enclosed by the boundary contour is the unoccupied area.

[0132] In an optional embodiment, when the processor 65 identifies the area segmentation line in the unoccupied area, it is specifically used to: acquire environmental images collected by the visual sensor during the autonomous mobile device's traversal of the environmental area; identify spatial segments existing in the environmental images and their position coordinates in the environmental images; and map the position coordinates of the spatial segments in the environmental images to a first environmental map to obtain the area segmentation line in the unoccupied area.

[0133] In an optional embodiment, when the processor 65 identifies spatial segments present in an environmental image, it specifically performs the following: semantic segmentation on the environmental image to obtain semantic labels for at least one object in the environmental image; identifies spatial segments present in the environmental image based on the semantic labels of the at least one object and the semantic labels of known spatial segments; or, inputs the environmental image into a segmentation recognition model for recognition to obtain spatial segments present in the environmental image; or, extracts geometric features of at least one object in the environmental image; and identifies spatial segments present in the environmental image based on the geometric features of the at least one object and the geometric features of known spatial segments.

[0134] In an optional embodiment, the environment images are multiple. When the processor 65 obtains the region segmentation line in the unoccupied area, it specifically performs the following: mapping the position coordinates of the spatial segment in each environment image to the first environment map to obtain multiple sets of position coordinates; calculating the average position coordinate of the multiple sets of position coordinates, and using the line segment corresponding to the average position coordinate as the region segmentation line in the unoccupied area.

[0135] In an optional embodiment, when adding exterior walls to multiple sub-regions, the processor 65 specifically performs the following: for each sub-region, extracts feature information corresponding to the sub-region from the environmental image collected by the visual sensor, and adds the feature information to the corresponding position in the first environmental map; adds a regular outline as an exterior wall around each sub-region to obtain a second environmental map; wherein each sub-region and its corresponding feature information all fall within the corresponding outline, and the outlines of different sub-regions do not overlap.

[0136] In an optional embodiment, after adding a regular outline to the periphery of each sub-region, the processor 65 is further configured to: fill the gaps between the outlines of the multiple sub-regions to align with the exterior walls of the multiple sub-regions.

[0137] In an alternative embodiment, after obtaining the second environment map, the processor 65 is further configured to perform at least one of the following operations:

[0138] Display a second environmental map on the screen of an autonomous mobile device;

[0139] The second environment map is projected onto a screen or wall for display.

[0140] The second environment map is sent to the user's terminal device for display.

[0141] In an optional embodiment, after the processor 65 displays the second environment map on the display screen, it is further configured to: in response to a trigger operation on the second environment map, perform a specified operation on a target object in the environment area, wherein the target object corresponds to the triggered map position on the second environment map, and the map position is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0142] In one optional embodiment, the environmental area corresponding to the first environmental map is a family area, and the spatial divider is at least one of the entrance door, room door, interior wall and partition in the family area, and the exterior wall in the family area is provided with windows.

[0143] In an alternative embodiment, when the processor 65 performs a specified operation on a target object in the environment region, it is specifically configured to perform at least one of the following operations:

[0144] If the target object is the front door, control the autonomous mobile device to check if the front door is abnormal or closed;

[0145] If the target object is a room door, control the autonomous mobile device to set up a virtual wall at the position corresponding to the room door on the second environment map to prevent entry into the room door;

[0146] If the target object is a window, control the autonomous mobile device to check if the window is abnormal or closed;

[0147] If the target object is an interior wall and / or an exterior wall, control the autonomous mobile device to perform an edge cleaning task along the corresponding boundary of the interior wall and / or exterior wall;

[0148] If the target area is a sub-area within a home, the autonomous mobile device will perform spot cleaning tasks within that sub-area. Sub-areas within a home can include rooms, kitchens, living rooms, bathrooms, balconies, studies, children's rooms, or gyms, etc.

[0149] In an optional embodiment, the autonomous mobile device is a robotic vacuum cleaner, which can perform cleaning tasks. In this case, the processor 65 is further configured to: obtain the target sub-region corresponding to the cleaning area on the second environmental map; calculate the cleaned floor area based on the outer wall of the target sub-region and the movement trajectory of the autonomous mobile device; display the cleaned floor area in the target sub-region of the second environmental map; and ensure that the cleaned floor area is greater than or equal to the actual cleaning area.

[0150] Furthermore, such as Figure 6aAs shown, the autonomous mobile device also includes other components such as a communication component 66, a display 67, a power supply component 68, and an audio component 69. Figure 6a The diagram only shows some components and does not mean that autonomous mobile devices only include... Figure 6a The components shown.

[0151] Optionally, the autonomous mobile device in this embodiment can be a robot, a purifier, or the like.

[0152] In one alternative embodiment, the autonomous mobile device is implemented as a robot. For example... Figure 6b As shown, the robot 600 in this embodiment includes: a mechanical body 601, on which one or more processors 602, one or more memories 603 storing computer instructions, and a communication component 604 are provided. The communication component 604 may be a Wi-Fi module, an infrared module, or a Bluetooth module, etc.

[0153] In addition to one or more processors 602, communication components 604, and one or more memories 603, the mechanical body 601 also includes some basic components of the robot 600, such as a vision sensor 606, a power supply component 607, a drive component 608, or an inertial navigation sensor 609. The vision sensor can be a camera, etc. The inertial navigation sensor can be an IMU or an odometer, etc. Optionally, the drive component 608 can include drive wheels, drive motors, omnidirectional wheels, etc. Optionally, if the robot 600 is a self-service robot in a bank, it can also include a keyboard, display screen, document recognition component, document scanning component, touch sensing component, and printing component, etc. If the robot 600 is a handling robot, it can also include a cargo pallet, a mobile chassis, or a robotic arm, etc. If the robot 600 is a sweeping robot, it can also include a cleaning component, which can include a cleaning motor, cleaning brushes, dust brushes, and a vacuum fan, etc. The basic components and their configurations vary among different robots 600; the embodiments described in this application are merely partial examples. It is worth noting that... Figure 6b The components shown in the dashed boxes are optional, not required.

[0154] It is worth noting that one or more processors 602 and one or more memories 603 can be disposed inside the mechanical body 601 or on the surface of the mechanical body 601. The mechanical body 601 is the actuator on which the robot 600 completes its tasks, and can execute the operations specified by the processor 602 in a defined environment. The mechanical body 601, to a certain extent, reflects the appearance of the robot 600. In this embodiment, the appearance of the robot 600 is not limited; for example, it can be humanoid, circular, etc. Regarding the processor 602 executing the computer program stored in the memory 603, it can achieve the same functionality as in the aforementioned embodiments (e.g., ...). Figure 6a The same functions as those in the illustrated embodiment will not be repeated here, but can be found in the descriptions in the foregoing embodiments.

[0155] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by an autonomous mobile device in the above method embodiments.

[0156] Figure 7 This application provides a schematic diagram of the structure of a terminal device. For example... Figure 7 As shown, the terminal device includes: a memory 71, a processor 72, and a display 73.

[0157] Memory 71 is used to store computer programs and can be configured to store various other data to support operation on the autonomous mobile device. Examples of this data include instructions, messages, images, videos, etc., for any application or method used to operate on the autonomous mobile device.

[0158] The processor 72, coupled to the memory 71, is used to execute a computer program in the memory 71 for: displaying an environment map via a display 73 in response to a display operation, the environment map including multiple passable sub-areas and their outer walls in the corresponding environment area; and sending control commands to an autonomous mobile device in response to a trigger operation on the environment map, so that the autonomous mobile device performs a specified operation on a target object in the environment area; wherein the target object corresponds to a triggered map position on the environment map, and the map position is determined by the user based on the relative positional relationship between the target object and the outer wall.

[0159] In an optional embodiment, the environmental area corresponding to the above-mentioned environmental map is a family area, and the target object is at least one of the following in the family area: entrance door, room door, interior wall, partition, exterior wall, and window installed on the exterior wall.

[0160] In one optional embodiment, if the target object is the front door, an operation is triggered to instruct the autonomous mobile device to check if the front door is abnormal or closed; if the target object is a room door, an operation is triggered to instruct the autonomous mobile device to set a virtual wall on the environmental map at the location corresponding to the room door to prevent entry; if the target object is a window, an operation is triggered to instruct the autonomous mobile device to check if the window is abnormal or closed; if the target object is an interior wall and / or an exterior wall, an operation is triggered to instruct the autonomous mobile device to perform an edge cleaning task along the boundary corresponding to the interior wall and / or exterior wall; if the target object is an environmental sub-area in the home, an operation is triggered to instruct the autonomous mobile device to perform a fixed-point cleaning task in the environmental sub-area.

[0161] In an optional embodiment, the autonomous mobile device is a robotic vacuum cleaner, which can perform cleaning tasks and return the area of ​​the apartment it has cleaned to the terminal device. In this case, the processor 72 is further configured to: receive the area of ​​the apartment cleaned when performing the cleaning task from the robotic vacuum cleaner; display the cleaned area of ​​the apartment in a target sub-region of the environmental map; wherein the cleaned area of ​​the apartment is greater than or equal to the actual cleaned area; the target sub-region refers to the sub-region on the second environmental map corresponding to the cleaning area where the cleaning task was performed.

[0162] Further optionally, the processor 72 is also configured to: receive the movement trajectory returned by the autonomous mobile device when performing edge or point cleaning tasks; and display the movement trajectory of the autonomous mobile device in the target sub-region of the environmental map.

[0163] Furthermore, such as Figure 7 As shown, the terminal device also includes other components such as a communication component 74, a power supply component 75, and an audio component 76. Figure 7 The diagram only shows some components and does not mean that the terminal device only includes... Figure 7 The components shown.

[0164] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by a terminal device in the above method embodiments.

[0165] The memory in the above embodiments can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0166] The communication components in the above embodiments are configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may further include a near-field communication (NFC) module, radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, etc.

[0167] The display in the above embodiments includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0168] The power supply component in the above embodiments provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0169] The audio component in the above embodiments can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0170] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0176] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0178] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An environmental map construction method suitable for an autonomous mobile device, characterized by, The method comprises: obtaining a first environment map corresponding to a traversed environment region, the first environment map comprising an unoccupied region; according to the position of a spatial segmentation body in the environment region, identifying a region segmentation line in the unoccupied region, the region segmentation line dividing the unoccupied region into a plurality of sub-regions; adding an outer wall to the plurality of sub-regions to obtain a second environment map, the second environment map comprising the plurality of sub-regions and the outer wall thereof, wherein adding the outer wall to the plurality of sub-regions to obtain the second environment map comprises: adding a regular outer contour as the outer wall to the periphery of each sub-region to obtain the second environment map; the regular outer contour comprises at least one of the following: a rectangle, a square, a triangle, a pentagon, a hexagon.

2. The method of claim 1, wherein, The first environment map comprises occupied and unoccupied position information in the environment region, and the method further comprises: adding a boundary contour between the occupied and unoccupied position information, the area surrounded by the boundary contour being an unoccupied region.

3. The method of claim 1, wherein, According to the position of a spatial segmentation body in the environment region, identifying a region segmentation line in the unoccupied region comprises: obtaining an environment image collected by a visual sensor during traversal of the autonomous mobile device in the environment region; identifying a spatial segmentation body present in the environment image and its position coordinates in the environment image; mapping the position coordinates of the spatial segmentation body in the environment image to the first environment map to obtain the region segmentation line in the unoccupied region.

4. The method of claim 3, wherein, Identifying a spatial segmentation body present in the environment image comprises: performing semantic segmentation on the environment image to obtain a semantic label of at least one object in the environment image; according to the semantic label of the at least one object and the semantic label of a known spatial segmentation body, identifying the spatial segmentation body present in the environment image; or inputting the environment image into a segmentation body identification model for identification to obtain the spatial segmentation body present in the environment image; or extracting geometric features of at least one object in the environment image; according to the geometric features of the at least one object and the geometric features of a known spatial segmentation body, identifying the spatial segmentation body present in the environment image.

5. The method of claim 3, wherein, If the environment image is a plurality of images, mapping the position coordinates of the spatial segmentation body in the environment image to the first environment map to obtain the region segmentation line in the unoccupied region comprises: mapping the position coordinates of the spatial segmentation body in each environment image to the first environment map respectively to obtain a plurality of sets of position coordinates; calculating average position coordinates of the plurality of sets of position coordinates, and taking a line segment corresponding to the average position coordinates as the region segmentation line in the unoccupied region.

6. The method of claim 1, wherein, Adding an outer wall to the plurality of sub-regions to obtain a second environment map comprises: for each sub-region, extracting feature information corresponding to the sub-region from the environment image collected by the visual sensor, and adding the feature information to the corresponding position in the first environment map; adding a regular shape contour as an outer wall body at the periphery of each sub-region to obtain the second environment map; wherein each sub-region and its corresponding feature information are all within the corresponding shape contour, and the shape contours of different sub-regions do not overlap.

7. The method of claim 6, wherein, After adding the regular shape contour at the periphery of each sub-region, the method further comprises: filling the gaps between the shape contours of the plurality of sub-regions to flush the outer wall bodies of the plurality of sub-regions.

8. The method according to any one of claims 1 to 7, characterized in that, After obtaining the second environment map, the method further comprises at least one of the following operations: displaying the second environment map on a display screen of the autonomous mobile device; projecting the second environment map onto a curtain or a wall surface for display; sending the second environment map to a terminal device of a user, so that the terminal device displays the second environment map.

9. The method of claim 8, wherein, After displaying the second environment map on the display screen of the autonomous mobile device, the method further comprises: in response to a triggering operation on the second environment map, performing a specified operation on a target object in the environment region, the target object corresponding to a triggered map position on the second environment map, and the map position being determined by a user based on the relative positional relationship between the target object and the outer wall body.

10. The method of claim 9, wherein, The environment region is a home region, and the space segmentation body is at least one of an entry door, a room door, an inner wall body, and a shutter in the home region, and the outer wall body in the home region is provided with a window.

11. The method of claim 10, wherein, Performing a specified operation on a target object in the environment region comprises at least one of the following: if the target object is an entry door, the autonomous mobile device checks whether the entry door is abnormal or closed; if the target object is a room door, the autonomous mobile device sets a virtual wall on the second environment map at a position corresponding to the room door to close the room door; if the target object is a window, the autonomous mobile device checks whether the window is abnormal or closed; if the target object is an inner wall body and / or an outer wall body, the autonomous mobile device performs a boundary cleaning task along the inner wall body and / or the outer wall body; if the target object is an environmental sub-region in the home, the autonomous mobile device performs a spot cleaning task in the environmental sub-region.

12. The method according to any one of claims 1-7 and 9-11, characterized by, The autonomous mobile device is a sweeping robot, and the method further comprises: in a case where the sweeping robot performs a cleaning task, obtaining a target sub-region corresponding to a cleaning region on the second environment map; calculating a cleaned house type area according to the outer wall body of the target sub-region and the movement trajectory of the sweeping robot; in the target sub-region of the second environment map, displaying the cleaned house type area, which is greater than or equal to the actual cleaning area.

13. A method of controlling an autonomous mobile device, adapted for a terminal device, characterized in that, The method comprises: in response to a display operation, displaying an environment map, the environment map comprising a plurality of passable sub-regions and their outer wall bodies in their corresponding environment regions; wherein the environment map is obtained according to the method of claim 1; in response to a triggering operation on the environment map, sending a control instruction to the autonomous mobile device to make the autonomous mobile device perform a specified operation on a target object in the environment region; The target object corresponds to a triggered map position on the environment map, and the map position is determined by the user based on a relative position relationship between the target object and the outer wall.

14. The method of claim 13, wherein, The environment region is a home region, and the target object is at least one of an entry door, a room door, an inner wall, a partition, an outer wall, and a window arranged on the outer wall in the home region.

15. The method of claim 14, wherein, if the target object is the entry door, the triggering operation instructs the autonomous mobile device to check whether the entry door is abnormal or closed; if the target object is the room door, the triggering operation instructs the autonomous mobile device to set a virtual wall on the environment map at a position corresponding to the room door to prohibit entry into the room door; if the target object is the window, the triggering operation instructs the autonomous mobile device to check whether the window is abnormal or closed; if the target object is the inner wall and / or the outer wall, the triggering operation instructs the autonomous mobile device to perform a boundary cleaning task along the inner wall and / or the outer wall; if the target object is an environment sub-region in the home, the triggering operation instructs the autonomous mobile device to perform a point cleaning task in the environment sub-region.

16. The method according to any one of claims 13-15, characterized in that, The autonomous mobile device is a sweeping robot, and the method further comprises: receiving a cleaned house type area sent by the sweeping robot when performing the cleaning task; displaying the cleaned house type area in the target sub-region of the environment map, the cleaned house type area being greater than or equal to the actual cleaning area; wherein the target sub-region refers to a sub-region corresponding to the cleaning region on the second environment map.

17. The method of claim 16, wherein, Further comprising: receiving a movement trajectory returned by the autonomous mobile device when performing the cleaning task; displaying the movement trajectory of the autonomous mobile device in the target sub-region of the environment map.

18. An autonomous mobile device, comprising: Comprising: a device body, the device body being provided with a memory and a processor; the memory, configured to store a computer program; the processor, coupled to the memory, configured to execute the computer program, so as to: obtain a first environment map corresponding to a traversed environment region, the first environment map comprising an unoccupied region; identify a region division line in the unoccupied region according to a position of a space division body in the environment region, the region division line dividing the unoccupied region into a plurality of sub-regions; add an outer wall to the plurality of sub-regions to obtain a second environment map, the second environment map comprising the plurality of sub-regions and outer walls thereof, wherein adding the outer wall to the plurality of sub-regions to obtain the second environment map comprises: adding a regular outer contour as an outer wall to the periphery of each sub-region to obtain the second environment map; the regular outer contour comprises at least one of a rectangle, a square, a triangle, a pentagon, and a hexagon.

19. A terminal device, comprising: Comprising: a memory, a processor, and a display; the memory, configured to store a computer program; the processor, coupled to the memory, configured to execute the computer program, so as to: In response to a display operation, an environment map is displayed through the display, the environment map including a plurality of passable sub-areas and outer walls in a corresponding environment area thereof; wherein the environment map is constructed according to the method of claim 1; In response to a trigger operation on the environment map, a control instruction is sent to the autonomous mobile device to make the autonomous mobile device perform a specified operation on a target object in the environment area; wherein the target object corresponds to a triggered map position on the environment map, and the map position is determined by a user based on a relative position relationship between the target object and the outer walls.

20. A computer readable storage medium storing a computer program, wherein the computer program comprises instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 19. When the computer program is executed by a processor, the processor is caused to implement the steps in the method of any one of claims 1-17.

Images