Map processing method and device for cleaning equipment, medium and equipment

Abstract

The invention discloses a map processing method and device for cleaning equipment, a medium and equipment. The method comprises the following steps: acquiring an environment map constructed by the cleaning equipment; the environment map is constructed by the cleaning equipment based on environment information collected for the cleaning environment; obtaining an initial three-dimensional environment reconstruction result of the auxiliary equipment for clean environment reconstruction; obtaining registration information, and registering the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the pose of a target environment object in the initial three-dimensional environment reconstruction result and is used for registering the initial three-dimensional environment reconstruction result with an environment map; and correcting the environment map based on the registered three-dimensional environment reconstruction result. According to the scheme, the map accuracy can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent equipment, and specifically relates to a map processing method, device, medium and equipment for cleaning equipment. Background Art

[0002] With the rapid development of science and technology, various intelligent devices have emerged, bringing great convenience to people's lives and work. Cleaning equipment is one of the most important intelligent devices. People only need to run the cleaning equipment, and the cleaning equipment can automatically perform cleaning work. Cleaning equipment often relies on environmental maps to perform cleaning work, so the accuracy of environmental maps is crucial.

[0003] In traditional methods, environmental maps are constructed by cleaning equipment based on information collected by sensors. Since cleaning equipment itself often has many limitations, such as angle and position limitations, the constructed environmental maps are not accurate enough. Therefore, it is urgent to propose a solution to the problem of inaccurate environmental maps of cleaning equipment in traditional methods. Summary of the invention

[0004] Therefore, the technical problem to be solved by the present application is how to improve the accuracy of the environmental map of the cleaning equipment.

[0005] In order to solve the above technical problems, the present application provides a map processing method for a cleaning device, comprising:

[0006] Acquire an environment map constructed by a cleaning device; the environment map is constructed by the cleaning device based on environmental information collected for a cleaning environment;

[0007] Obtaining an initial three-dimensional environment reconstruction result of the cleaning environment reconstructed by the auxiliary device; the initial three-dimensional environment reconstruction result is obtained by reconstructing the cleaning environment based on the collected image after the auxiliary device collects images of the cleaning environment; the collection range of the cleaning environment by the auxiliary device is larger than the collection range of the cleaning device;

[0008] Acquire registration information, and register the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to register the initial three-dimensional environment reconstruction result with the environment map;

[0009] The environment map is corrected based on the registered three-dimensional environment reconstruction result.

[0010] Optionally, the correcting the environment map based on the registered three-dimensional environment reconstruction result includes:

[0011] Obtaining a first position of an environmental object in the environmental map;

[0012] Based on the second pose of the environmental object in the registered three-dimensional environment reconstruction result, the first pose of the environmental object is corrected.

[0013] Optionally, the correcting the first pose of the environment object based on the second pose of the environment object in the registered three-dimensional environment reconstruction result includes:

[0014] Comparing a second posture of the environment object in the three-dimensional environment reconstruction result with the first posture;

[0015] In the case where a posture overlapping object is determined based on the posture comparison result, determining the source of the first posture of the posture overlapping object; the posture overlapping object is an environmental object where the first posture and the second posture overlap;

[0016] In a case where the source representation indicates that the first pose is obtained through custom editing, the first pose of the pose overlapping object is retained in the environment map.

[0017] Optionally, the method further comprises:

[0018] In the case where the source representation indicates that the first posture is obtained by recognition by the cleaning device itself, the first posture of the environmental object in the environmental map is replaced by a second posture of the environmental object in the three-dimensional environment reconstruction result.

[0019] Optionally, the method further comprises:

[0020] In the case where an environmental object to be updated in the three-dimensional environment reconstruction result is determined based on the position and posture comparison result, updating and supplementing the second position and posture of the environmental object to be updated in the three-dimensional environment reconstruction result into the environmental map;

[0021] The to-be-updated environment object is an environment object that exists in the three-dimensional environment reconstruction result but does not exist in the environment map.

[0022] Optionally, the correcting the first pose of the environment object based on the second pose of the environment object in the registered three-dimensional environment reconstruction result includes:

[0023] For each environmental object in the three-dimensional environmental reconstruction result, a first pose of the environmental object in the three-dimensional reconstruction result is fused with a second pose of the environmental object in the environmental map to obtain a final pose of the environmental object.

[0024] Optionally, the three-dimensional environment reconstruction result includes information of a penetrable boundary in the cleaning environment; the penetrable boundary is a boundary environment object that can be penetrated by laser;

[0025] The correcting of the environment map based on the registered three-dimensional environment reconstruction result includes:

[0026] Determining whether the penetrable boundary is a preset degraded boundary based on the information of the penetrable boundary; the degraded boundary is a boundary that can cause map degradation;

[0027] When the penetrable boundary is a degraded boundary, a degraded area corresponding to the degraded boundary in the environment map is removed.

[0028] Optionally, the penetrable boundary includes a window; the information of the penetrable boundary includes a window height of the window;

[0029] The determining, based on the information of the penetrable boundary, whether the penetrable boundary is a preset degraded boundary; the degraded boundary is a boundary that can cause map degradation, includes:

[0030] Determining whether the window is a floor-to-ceiling window that can cause map degradation based on the window height;

[0031] When the penetrable boundary is a degraded boundary, removing the degraded area corresponding to the degraded boundary in the environment map includes:

[0032] In the case where the window is a floor-to-ceiling window, the degraded area caused by the floor-to-ceiling window in the environment map is removed.

[0033] Optionally, the three-dimensional environment reconstruction result includes spatial boundary information for the clean environment;

[0034] The step of correcting the environment map based on the registered three-dimensional environment reconstruction result includes:

[0035] Based on the spatial boundary information, the spatial structure information in the clean environment is supplemented in the environment map.

[0036] Optionally, the space boundary information includes door information for the clean environment; the space dividing line in the environment map includes a door boundary dividing line;

[0037] The method of supplementing the spatial structure information in the clean environment in the environmental map based on the spatial boundary information includes:

[0038] Supplement the door boundary segmentation line in the environment map based on the door information to obtain a candidate segmentation line;

[0039] Performing spatial segmentation on the environment map based on the candidate segmentation lines to obtain an initial spatial set;

[0040] The initial space set is used as the current space set, a space whose area is smaller than a preset area is determined from the current space set as the current space to be merged, and a candidate dividing line between the current space to be merged and an adjacent space is removed to perform space merging processing on the current space to be merged and the adjacent space;

[0041] The spatial set after the spatial merging process is used as the new current spatial set, and the step of iteratively executing the step of determining a space with an area smaller than a preset area from the current spatial set as the current space to be merged is returned until the iteration is stopped when there is no space with an area smaller than the preset area in the current spatial set, thereby obtaining an environment map after spatial segmentation correction.

[0042] Optionally, the three-dimensional environment reconstruction result includes information of objects in the clean environment; and the correcting the environment map based on the registered three-dimensional environment reconstruction result includes:

[0043] identifying objects in an adjacent space from the environment map according to the object category in the object information;

[0044] Based on the position of the object in the adjacent space, a space dividing line for dividing the adjacent space is added to the environment map.

[0045] Optionally, the three-dimensional environment reconstruction result includes information of objects in the clean environment; and the correcting the environment map based on the registered three-dimensional environment reconstruction result includes:

[0046] identifying, according to the object category in the object information, a space type of each space in the environment map for placing the object;

[0047] The identified space types are added to the environment map accordingly.

[0048] Optionally, the acquiring registration information and registering the initial three-dimensional environment reconstruction result with the environment map based on the registration information includes:

[0049] Extracting information of the target environment object from the three-dimensional reconstruction result;

[0050] Converting the extracted information of the target environment object into an object point cloud;

[0051] Converting the object point cloud into different preset poses, and projecting the object point cloud after pose conversion onto the environment map;

[0052] Determine the target pose corresponding to the object point cloud having the highest overlap with the environment map after projection;

[0053] The target environment object in the three-dimensional environment reconstruction result is transformed and adjusted according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

[0054] The present application also provides a map processing device for cleaning equipment, comprising:

[0055] An acquisition module is used to acquire an environment map constructed by a cleaning device; the environment map is constructed by the cleaning device based on the environmental information collected for the cleaning environment; an initial three-dimensional environment reconstruction result reconstructed by an auxiliary device for the cleaning environment; the initial three-dimensional environment reconstruction result is the reconstruction of the cleaning environment based on the collected image after the auxiliary device collects images of the cleaning environment; the collection range of the cleaning environment by the auxiliary device is larger than the collection range of the cleaning device;

[0056] A registration module, used to obtain registration information, and to register the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to register the initial three-dimensional environment reconstruction result with the environment map;

[0057] The correction module is used to correct the environment map based on the registered three-dimensional environment reconstruction result.

[0058] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the map processing method for cleaning equipment provided by the above aspects are implemented.

[0059] The present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps in the map processing method for cleaning equipment provided by the above aspects are implemented.

[0060] The present application also provides a computer device, which may be a cleaning device or a user terminal, including a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps in the map processing method for the cleaning device provided by the above aspects are implemented.

[0061] The technical solution provided by this application has the following advantages:

[0062] The three-dimensional environmental reconstruction results of the auxiliary equipment for the cleaning environment are aligned with the environmental map created by the cleaning equipment itself, and the environmental map is corrected based on the aligned three-dimensional environmental reconstruction results. This can add richer information in the three-dimensional environmental reconstruction results to the environmental map of the cleaning equipment, thereby solving the limitations of the cleaning equipment and improving the accuracy of the environmental map. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 This is an application environment diagram of a map processing method for cleaning equipment in one embodiment of the present application;

[0065] Figure 2 This is a flow chart of a map processing method for a cleaning device in one embodiment of the present application;

[0066] Figure 3 A two-dimensional occupancy probability grid map in one embodiment of the present application;

[0067] Figure 4 A three-dimensional point cloud map in one embodiment of the present application;

[0068] Figure 5 This is a schematic diagram of a three-dimensional environment reconstruction result in one embodiment of the present application;

[0069] Figure 6 A schematic diagram of a customized three-dimensional map in one embodiment of the present application;

[0070] Figure 7 A schematic diagram of a three-dimensional environment reconstruction result after registration in one embodiment of the present application;

[0071] Figure 8 This is a schematic diagram of the map penetration degradation phenomenon in one embodiment of the present application;

[0072] Fig. 9 This is a schematic diagram of map optimization in one embodiment of the present application;

[0073] Fig.10 An environment map after the supplementary information is corrected in one embodiment of the present application;

[0074] Fig.11 A block diagram of a map processing device for a cleaning device in one embodiment of the present application;

[0075] Fig.12 This is a diagram of the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0077] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are only used for distinguishing expressions and are not necessarily used to describe a specific order or sequence.

[0078] The map processing method for cleaning equipment provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the cleaning device 102 communicates with the auxiliary device 104 through the network. The cleaning device 102 is provided with a sensor, and the sensor used by the cleaning device includes at least one of a collision sensor, an IMU, a wheel speed meter, an image acquisition unit, LDS (single-line laser radar) or ToF (area array laser radar). The image acquisition unit is not limited to a camera, but may also include other devices with image acquisition functions such as a video camera or a camera. The sensor can collect environmental information in the cleaning environment around the cleaning device 102. The auxiliary device 104 also has an information collection function for the cleaning environment, and the collection range of the auxiliary device for the cleaning environment is greater than the collection range of the cleaning device.

[0079] Specifically, the cleaning device 102 can obtain the environment map constructed by itself, and obtain the initial three-dimensional environment reconstruction result reconstructed by the auxiliary device 401 for the cleaning environment. The cleaning device 102 can obtain registration information, and align the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to align the initial three-dimensional environment reconstruction result with the environment map. The cleaning device 102 can correct the environment map based on the registered three-dimensional environment reconstruction result.

[0080] It can be understood that the cleaning device 102 is an intelligent device that automatically performs cleaning tasks. The cleaning device 102 can be a sweeping robot or a public cleaning device. That is, the cleaning device is not limited to a sweeping robot of an intelligent home appliance, but can also be a large-scale intelligent device for autonomously performing cleaning tasks in public places. The present application does not limit the specific form of the cleaning device, as long as the device can autonomously perform cleaning tasks, it can fall within the scope of the cleaning device in the present application. The auxiliary device 104 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0081] In some embodiments, Figure 2 As shown, a map processing method for cleaning equipment is provided. The method is applied to a cleaning equipment as an example for explanation. It can be understood that the map processing method for cleaning equipment can also be implemented through the interaction between the cleaning equipment and the auxiliary equipment. The map processing method for cleaning equipment specifically includes the following steps:

[0082] Step 202, obtaining an environment map constructed by the cleaning equipment.

[0083] The cleaning environment is the environment that the cleaning equipment needs to clean. The cleaning environment can be an indoor environment or an outdoor environment. The environmental map is a map constructed by the cleaning equipment based on the environmental information collected by the cleaning equipment itself.

[0084] Specifically, the cleaning device is provided with a sensor, and the cleaning device can use the sensor to collect information about the cleaning environment during movement. After using the sensor to collect environmental information, the cleaning device will also perform map building processing based on the collected environmental information to build an environmental map.

[0085] In some embodiments, when the cleaning device is in a mapping mode, environmental information of the cleaning environment is collected. The mapping mode includes but is not limited to any one of a non-cleaning mapping mode or a cleaning and mapping mode. The non-cleaning mapping mode means that the cleaning device continuously moves in the cleaning environment without performing a cleaning task to collect environmental information to build an environmental map. The cleaning and mapping mode means that the cleaning device collects environmental information to build an environmental map while performing a cleaning task.

[0086] It is understood that the environment map is used to illustrate the location distribution of environmental objects in the cleaning environment. The environment map includes but is not limited to at least one of a two-dimensional collision map based on collision information, a two-dimensional occupancy probability grid map based on LDS, a three-dimensional occupancy probability grid map based on ToF, or a three-dimensional point cloud map based on ToF. Figure 3 FIG. 4 is a two-dimensional occupancy probability grid diagram in one embodiment. Figure 4A three-dimensional point cloud map in one embodiment.

[0087] Step 204: obtaining an initial three-dimensional environment reconstruction result of the auxiliary equipment for the cleaning environment.

[0088] Among them, the auxiliary equipment is a device other than the cleaning equipment itself. The collection range of the auxiliary equipment for the cleaning environment is larger than that of the cleaning equipment, that is, the auxiliary equipment can collect more comprehensive environmental information than the cleaning equipment. For example, when the cleaning equipment collects environmental information based on sensors, it is easily affected by angles, occlusions, etc., resulting in a limited collection range, so that the environmental information collected by the sensor is missing a lot. To address this problem of cleaning equipment, the present application proposes an auxiliary equipment with a collection range larger than that of the cleaning equipment. The use of the auxiliary equipment can collect more comprehensive environmental information, so as to assist in supplementing the environmental information in the environmental map of the cleaning equipment. Therefore, it is called an auxiliary equipment.

[0089] It should be noted that the auxiliary equipment is not limited to a larger collection range than the cleaning equipment. The computing power of the auxiliary equipment can also be greater than that of the cleaning equipment, so as to calculate more comprehensive information to supplement the missing information of the cleaning equipment. The initial three-dimensional environment reconstruction results reconstructed by the auxiliary equipment in this application are used to supplement the information of the environmental map of the cleaning equipment to improve the accuracy and completeness of the environmental map. In some embodiments, the auxiliary device can be a mobile smart terminal with high computing power such as a mobile phone or a smart wearable device.

[0090] The initial three-dimensional environment reconstruction result is the result obtained by the auxiliary device performing three-dimensional reconstruction of the clean environment. The initial three-dimensional environment reconstruction result is obtained after the auxiliary device performs image acquisition of the clean environment and reconstructs the clean environment based on the acquired images. That is, the auxiliary device can acquire images of the clean environment and perform three-dimensional reconstruction of the clean environment based on the acquired images to obtain the initial three-dimensional environment reconstruction result. Figure 5 It is a schematic diagram of a three-dimensional environment reconstruction result in an embodiment.

[0091] In some embodiments, the initial three-dimensional environment reconstruction result may include at least one of spatial boundary information, object information, mesh texture map, point cloud map or customized three-dimensional map in the cleaning environment. Figure 6 A schematic diagram of a customized three-dimensional map in an embodiment.

[0092] It can be understood that the spatial boundary includes but is not limited to at least one of the boundaries of a wall, a window, an open space or a door. Objects in a clean environment include but are not limited to at least one of furniture, home appliances or daily necessities. It should be noted that both the spatial boundary and the objects in a clean environment belong to environmental objects in a clean environment. The information of environmental objects at least includes but is not limited to at least one of the posture or its own attributes. Among them, the attributes include but are not limited to at least one of the length, width, height or category.

[0093] In some embodiments, after the auxiliary device reconstructs the initial three-dimensional environment reconstruction result, the auxiliary device can notify the cleaning device, and the cleaning device obtains the initial three-dimensional environment reconstruction result reconstructed by the auxiliary device. The method for the cleaning device to obtain the three-dimensional reconstruction result includes but is not limited to the auxiliary device directly transmitting the three-dimensional reconstruction result information or the auxiliary device uploading to the server and notifying the cleaning device to download the three-dimensional reconstruction information.

[0094] It is understandable that the notification method for the auxiliary device to notify the cleaning device includes but is not limited to Bluetooth, WiFi, and server-transmitted messages. The notification content includes but is not limited to an indication that the three-dimensional reconstruction has been completed or a specific initial three-dimensional environment reconstruction result. That is, the auxiliary device can upload the initial three-dimensional environment reconstruction result to the server and send a mark to the cleaning device that the three-dimensional reconstruction has been completed. The cleaning device can download the initial three-dimensional environment reconstruction result from the server based on the mark. The auxiliary device can also directly transmit the initial three-dimensional environment reconstruction result to the cleaning device when notifying the cleaning device. It should be noted that the present application does not limit the specific implementation method for the cleaning device to obtain the initial three-dimensional environment reconstruction result of the auxiliary device.

[0095] Step 206, obtaining registration information, and based on the registration information, registering the initial three-dimensional environment reconstruction result with the environment map; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to register the initial three-dimensional environment reconstruction result with the environment map.

[0096] The target environment object is an environment object that exists in the clean environment and is used as a reference for registration. That is, the target environment object is used as a reference to align the initial three-dimensional environment reconstruction result with the environment map. It can be understood that the target environment object is an object that occupies an area in the clean environment that meets the preset conditions and can provide sufficient information for registration.

[0097] In some embodiments, the target environment object may be a main structure object in a clean environment. The main structure object refers to an object that occupies a relatively large area in a clean environment and can provide sufficient information for alignment. For example, the target environment object may be a wall object. A wall object is a wall in a clean environment. It can be understood that for an indoor environment, a wall object is an object that occupies the main structure and has a large area, so it can provide rich information as a basis for alignment. For another example, for an outdoor environment, the target environment object may be a green belt at the edge of a road, etc. The present application does not limit the target environment object to only a wall object, as long as it can play a role in alignment, it is a target environment object.

[0098] It can be understood that the origin of the coordinate system used in the three-dimensional environment reconstruction result reconstructed by the auxiliary device is different from the origin of the coordinate system in the environment map, so the two need to be registered. Transforming and adjusting the position and posture of the target environment object in the three-dimensional environment reconstruction result is essentially to continuously adjust the position and posture of the target environment object in the three-dimensional environment reconstruction result so that the projection of the target environment object in the environment map is located at the position of the target environment object in the environment map. Since the target environment object can serve as a registration reference, the three-dimensional environment reconstruction result can be registered with the environment map.

[0099] Specifically, the posture of the target environment object in the three-dimensional environment reconstruction result can be transformed and adjusted according to multiple preset postures, the projection area of ​​the target environment object adjusted according to each preset posture in the environment map is determined, the overlap between the projection area and the corresponding area of ​​the environment object in the environment map is calculated, and the registration information is determined based on the overlap. For example, the preset posture with the highest overlap can be determined as the target posture that needs to be registered, and then the target environment object can be transformed and adjusted according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

[0100] Figure 7 FIG. 1 is a schematic diagram of a registered three-dimensional environment reconstruction result in one embodiment, that is, some information in the three-dimensional environment reconstruction result is registered with the environment map. Figure 7 It can be seen that there are still some differences between the 3D environment reconstruction results and the environment map of the cleaning equipment itself, for example, Figure 7 The areas indicated by the dashed circles are where there are map differences between the 3D environment reconstruction result and the environment map of the cleaning equipment. In addition, the 3D environment reconstruction result also carries semantic information such as the position (including position and posture), size, and basic attributes of the environment objects, so it can also help the environment map determine the specific information of objects such as objects. For ease of understanding, Figure 7The location of the environmental object is marked in the form of a box to reflect that the information in the three-dimensional environment reconstruction result can help the environment map to supplement the corresponding object semantic information. However, it should be noted that it is not limited to marking the specific location of the environmental object in the environment map. Figure 7 The boxes are only used for illustration. Moreover, the information in the 3D environment reconstruction result is not limited to the specific position and posture, but can also include basic attribute information such as the name and category of the object. Figure 7 Only a table is taken as an example to illustrate the information such as the posture and category that can be used to supplement the environmental objects, but the corresponding category and other information are not supplemented for all the environmental objects marked in the boxes.

[0101] It is understandable that the cleaning device can directly transform and adjust the position and posture of the target environment object in the initial three-dimensional environment reconstruction result to determine the registration information used to align the initial three-dimensional environment reconstruction result with the environmental map. It can also be an auxiliary device to transform and adjust the position and posture of the target environment object in the initial three-dimensional environment reconstruction result to determine the registration information, and synchronize the registration information to the cleaning device. This application does not limit which device determines the registration information.

[0102] Step 208: Correct the environment map based on the registered three-dimensional environment reconstruction result.

[0103] In some embodiments, the fixed areas in the environment map can be marked based on the information of the fixed environment objects in the three-dimensional environment reconstruction result after registration. In this way, during the subsequent operation of the cleaning equipment, the fixed areas are in an occupied state and no longer change, which maintains the stability of the map and improves the robustness of the map. In addition, the credibility of these fixed areas can be improved, and the positioning stability can also be improved when the cleaning equipment is positioned. For example, when the environment map is a two-dimensional occupancy grid probability map, when the cleaning equipment uses the grid in the two-dimensional occupancy grid probability map for positioning, the credibility of the grid in the fixed area in the occupied state can be improved. Fixed environment objects are environment objects that are in a fixed state and will not move. Fixed environment objects can be environment objects in fixed positions such as walls and wardrobes. Specifically, the area where the three-dimensional environment reconstruction result overlaps with the fixed environment objects in the environment map can be determined as a fixed area. For example, the part of the wall that overlaps with the three-dimensional environment reconstruction result in the environment map can be determined as a fixed area. We can consider these fixed areas to be highly credible and basically will not change later, so they are marked as fixed areas, which improves the stability of subsequent maps.

[0104] In some embodiments, the cleaning device can also construct a three-dimensional environment map based on the corrected environment map. Since the environment map is corrected, the constructed three-dimensional environment map is more accurate and closer to the real environment. Further, when constructing the three-dimensional environment map, the cleaning device can also combine the texture map in the three-dimensional environment reconstruction result to construct a three-dimensional environment map with texture information. In this way, the constructed three-dimensional environment map has visual information of the real scene, which can be output to the user. In this way, subsequent users can more intuitively instruct the cleaning device to perform cleaning tasks based on the three-dimensional environment map with texture information, improve the accuracy of subsequent interactions, and thus reduce unnecessary erroneous interactions as much as possible when performing cleaning tasks, reducing the waste of system resources. For example, based on the three-dimensional environment map with texture information, the user can more intuitively know which object is in which position, so that the cleaning command can be issued more accurately.

[0105] It can be understood that since the auxiliary equipment can collect more comprehensive environmental information, after the initial three-dimensional environmental reconstruction results reconstructed by the auxiliary equipment are aligned with the environmental map created by the cleaning equipment, the three-dimensional environmental reconstruction results based on the alignment can assist in supplementing the environmental information in the environmental map of the cleaning equipment. Therefore, by correcting the environmental map, more abundant information can be provided to resource-constrained cleaning equipment, greatly improving the accuracy of the environmental map and enriching the amount of information in the environmental map. Moreover, the cleaning equipment itself only needs to build a simple environmental map, which reduces the requirements for computing power, thereby avoiding the impact of excessive computing power on the operating performance of the cleaning equipment.

[0106] In some embodiments, step 208, obtaining alignment information, and aligning the initial three-dimensional environment reconstruction result with the environment map based on the alignment information includes: extracting information of the target environment object from the three-dimensional reconstruction result; converting the extracted information of the target environment object into an object point cloud; converting the object point cloud into different preset postures, and projecting the object point cloud after the posture conversion to the environment map; determining the target posture corresponding to the object point cloud with the highest overlap with the environment map after projection; transforming and adjusting the target environment object in the three-dimensional environment reconstruction result according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

[0107] Specifically, the cleaning device can extract information of the target environment object from the three-dimensional reconstruction result, and convert the extracted information of the target environment object into an object point cloud according to a fixed resolution. A plurality of preset postures are pre-set, and the cleaning device can perform posture conversion on the object point cloud according to different preset postures, and project the object point cloud after posture conversion to the environment map. For each preset posture, after the object point cloud is transformed and adjusted according to the preset posture, the cleaning device can project the object point cloud to the projection area in the environment map, determine the overlap between the projection area and the non-blank area in the environment map, and determine the preset posture corresponding to the object point cloud with the highest overlap with the environment map as the target posture. It can be understood that the registration information includes the target posture, and then the cleaning device can transform and adjust the target environment object in the three-dimensional environment reconstruction result according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

[0108] It should be noted that the process of determining the target pose or determining the registration information is the process of determining how to register, and the process of transforming and adjusting the target environment object in the three-dimensional environment reconstruction result according to the target pose is the process of registering the three-dimensional environment reconstruction result with the environment map based on the registration information after knowing how to register.

[0109] In some embodiments, the environmental map may be a two-dimensional occupancy probability grid map. After projecting the object point cloud of the target environmental object onto the two-dimensional occupancy probability grid map, the projection result may be scored. If the grid state of the point cloud is occupied, a score is given, otherwise no score is given. In this way, after the object point cloud is converted according to a plurality of preset postures, each preset posture will have a corresponding score, and the cleaning device may select the preset posture with the highest score as the target posture. This can improve the accuracy of the registration.

[0110] In some embodiments, the environment map is corrected based on the registered three-dimensional environment reconstruction result, including: obtaining the first pose of the environment object in the environment map; and correcting the first pose of the environment object based on the second pose of the environment object in the registered three-dimensional environment reconstruction result.

[0111] It can be understood that the three-dimensional environment reconstruction result includes the posture of the environmental objects. In order to distinguish it from the posture of the environmental objects in the environmental map, this application calls the posture in the three-dimensional environment reconstruction result the second posture, and the posture in the environmental map the first posture. It should be noted that "first" and "second" are only used for distinction and are not otherwise limited. It should be noted that in addition to the posture of the environmental objects, the three-dimensional environment reconstruction result may also include other semantic information such as basic identity attributes and size, which is not limited.

[0112] Since the environmental map is created by the cleaning device itself, the first position of the environmental objects in the environmental map is also limited by sensors or computing power and is not accurate enough. Therefore, the cleaning device can use the three-dimensional environmental reconstruction results after registration to correct the position and posture of the environmental objects in the environmental map.

[0113] Specifically, the cleaning device can replace the first pose of the environmental object in the environmental map based on the second pose of the environmental object in the more accurate three-dimensional environmental reconstruction result. The cleaning device can also fuse the first pose and the second pose of the same environmental object to obtain the final pose of the environmental object. Since the second pose of the environmental object in the three-dimensional environmental reconstruction result is more complete, for environmental objects that are included in the three-dimensional environmental reconstruction result but not in the environmental map, the cleaning device can also supplement the second pose of the environmental object in the three-dimensional environmental reconstruction result in the environmental map. The present application does not limit the specific method for correcting the first pose of the environmental object in the environmental map.

[0114] In some embodiments, the environmental object for posture correction may be an object, including but not limited to furniture, household appliances, and daily necessities.

[0115] After the posture is corrected, it can be rendered for visualization. Then, it can be provided to the user, allowing the user to issue more specific cleaning instructions, which improves convenience. In addition, it can also let the cleaning equipment know the specific and accurate posture of the environmental object, which is convenient for the accuracy of subsequent cleaning tasks.

[0116] In some embodiments, based on the second posture of the environmental object in the registered three-dimensional environment reconstruction result, the first posture of the environmental object is corrected, including: performing posture comparison between the second posture of the environmental object in the three-dimensional environment reconstruction result and the first posture; in the case where a posture overlapping object is determined based on the posture comparison result, determining the source of the first posture of the posture overlapping object; the posture overlapping object is an environmental object where the first posture and the second posture overlap; in the case where the source represents that the first posture is obtained through custom editing, retaining the first posture of the posture overlapping object in the environment map.

[0117] Among them, the pose overlapping object is an environmental object whose first pose and second pose overlap. That is, when the second pose of an environmental object in the three-dimensional environment reconstruction result overlaps with the pose in the environment map, the environmental object is a pose overlapping object.

[0118] Specifically, the cleaning device can perform a posture comparison between the second posture of the environmental object in the three-dimensional environment reconstruction result and the first posture to obtain a posture comparison result. Based on the posture comparison result, determine whether there is an object with overlapping postures. In the case where there is an object with overlapping postures, determine the source of the first posture of the object with overlapping postures. It can be understood that the source of the first posture in the three-dimensional environment reconstruction result includes custom editing, and can also include the cleaning device identifying itself based on information collected by the sensor. In the case where the source characterization of the first posture is obtained through custom editing, the first posture of the object with overlapping postures is retained in the environmental map.

[0119] In some embodiments, the cleaning device may pre-store the first pose and corresponding size of each environmental object in the environmental map in a semantic object list. The cleaning device may determine the first pose of each environmental object in the pre-stored semantic object list in combination with the corresponding size to determine the first area corresponding to each environmental object, and determine the second pose of the environmental object in the three-dimensional environmental reconstruction result in combination with the corresponding size to determine the second area corresponding to each environmental object. Furthermore, for each environmental object in the three-dimensional environmental reconstruction result, the cleaning device may compare the second area corresponding to the environmental object with the first area corresponding to each environmental object in the semantic object list to determine whether there are objects with overlapping poses.

[0120] In some embodiments, the three-dimensional environment reconstruction result includes source information of the first pose of each environmental object, and the source of the first pose of the pose overlapping object can be determined based on the source information.

[0121] It can be understood that since the custom edit is edited by the user himself, it can reflect the user's needs to a large extent. Therefore, when the first pose recognized by the auxiliary device conflicts with the second pose edited by the user, the first pose of the object overlapped by the user's custom edited pose can be retained in the environment map.

[0122] In some embodiments, the method further includes: when the source characterization first pose is identified by the cleaning device itself, replacing the first pose of the environmental object in the environmental map with the second pose of the environmental object in the three-dimensional environment reconstruction result.

[0123] It can be understood that since the accuracy and comprehensiveness of the three-dimensional environment reconstruction results are higher than the information recognized by the cleaning equipment itself, when the first pose of the source representation is obtained by recognizing the cleaning equipment itself, the first pose of the environmental object in the environmental map can be replaced by the second pose of the environmental object in the three-dimensional environment reconstruction results to improve the accuracy of the pose in the environmental map.

[0124] In some embodiments, the method also includes: when determining the environmental object to be updated in the three-dimensional environment reconstruction result based on the posture comparison result, updating the second posture of the environmental object to be updated in the three-dimensional environment reconstruction result and adding it to the environment map; wherein the environmental object to be updated is an environmental object that exists in the three-dimensional environment reconstruction result but does not exist in the environment map.

[0125] It can be understood that for the environmental objects to be updated that exist in the three-dimensional environment reconstruction results but do not exist in the environment map, the second posture of the environmental objects to be updated in the three-dimensional environment reconstruction results can be updated and added to the environment map to supplement the semantic information in the environment map and enrich the amount of information.

[0126] In some embodiments, based on the second posture of the environmental object in the registered three-dimensional environmental reconstruction result, the first posture of the environmental object is corrected, including: for each environmental object in the three-dimensional environmental reconstruction result, the first posture of the environmental object in the three-dimensional reconstruction result is fused with the second posture of the environmental object in the environmental map to obtain the final posture of the environmental object.

[0127] It can be understood that in addition to replacing the first pose of the environmental object in the environmental map, the method of correcting the first pose of the environmental object in the environmental map can also fuse the second pose of the same environmental object in the three-dimensional environment reconstruction result with its first pose in the environmental map. Since the more accurate second pose is integrated into the first pose, the pose of the environmental object in the environmental map can be corrected by fusing the pose, thereby improving the pose accuracy.

[0128] In some embodiments, the three-dimensional environment reconstruction result includes information about the permeable boundary in the clean environment. Correcting the environment map based on the registered three-dimensional environment reconstruction result includes: determining whether the permeable boundary is a preset degraded boundary based on the information about the permeable boundary; the degraded boundary is a boundary that can cause map degradation; if the permeable boundary is a degraded boundary, removing the degraded area corresponding to the degraded boundary in the environment map.

[0129] In this embodiment, the environment map is constructed by using information detected by the laser radar. The penetrable boundary is a boundary environment object that can be penetrated by the laser; the penetrable boundary may include at least one of a window or a mirror. It can be understood that since the penetrable boundary is easily penetrated by the laser, in some cases, the area that does not exist on the other side of the penetrable boundary will still have the result of laser detection, that is, there are some unnecessary areas, which are inaccurate degraded areas.

[0130] Specifically, the cleaning device can determine whether the penetrable boundary is a preset degraded boundary based on the information of the penetrable boundary; the degraded boundary is a boundary that can cause map degradation. In the case where the penetrable boundary is a degraded boundary, the cleaning device can determine the position of the cleaning device when collecting information of this part of the area, determine the area on the other side of the degraded boundary opposite to the position as a degraded area, and remove the degraded area.

[0131] Figure 8 FIG. 1 is a schematic diagram of a map penetration degradation phenomenon in an embodiment. Figure 8 It can be seen that there are some degraded areas. For example, the area marked in the dotted box 802 is the degraded area caused by the floor-to-ceiling window being penetrated by the laser. The cleaning equipment will not enter this area for cleaning, so this area is invalid and is a degraded area that needs to be cleared. Fig. 9 FIG. 1 is a schematic diagram of map optimization in an embodiment. Figure 8 After removing the degraded area, the optimized Fig. 9 .

[0132] It can be understood that removing the degraded area can not only beautify the environmental map, but also reduce the noise and interference information in the environmental map, and improve the accuracy of the environmental map, so as to provide more accurate map guidance when the subsequent cleaning equipment performs cleaning tasks.

[0133] In some embodiments, the penetrable boundary includes a window; the information of the penetrable boundary includes the window height of the window. Based on the information of the penetrable boundary, it is determined whether the penetrable boundary is a preset degraded boundary; the degraded boundary is a boundary that can cause map degradation, including: determining whether the window is a floor-to-ceiling window that can cause map degradation based on the window height. In this embodiment, when the penetrable boundary is a degraded boundary, the degraded area corresponding to the degraded boundary in the environment map is removed, including: when the window is a floor-to-ceiling window, the degraded area caused by the floor-to-ceiling window in the environment map is removed.

[0134] It can be understood that the three-dimensional environmental reconstruction results provide very rich spatial boundary information, from which the height of the windows can be obtained, and then the floor-to-ceiling windows in the environmental map can be identified based on the height of the windows. Since the floor-to-ceiling windows are low in position and large in area, they are easily penetrated by the laser radar of the cleaning equipment, causing map degradation. Therefore, after identifying the floor-to-ceiling windows, the degraded areas caused by the floor-to-ceiling windows in the environmental map can be removed to improve the accuracy of the environmental map.

[0135] In some embodiments, the 3D environment reconstruction result includes spatial boundary information for the clean environment. The environment map is modified based on the registered 3D environment reconstruction result, including: supplementing the environment map with spatial structure information of the clean environment based on the spatial boundary information.

[0136] It can be understood that the 3D environment reconstruction result processing can not only supplement the position and posture, but also provide rich spatial boundary information. Based on the spatial boundary information, the spatial structure information in the clean environment can be supplemented in the environment map. The spatial structure information includes at least one of a spatial dividing line or a spatial type.

[0137] In some embodiments, floor-to-ceiling window information included in the three-dimensional environment reconstruction result may be used to provide a dividing line of the space connected to the balcony.

[0138] In some embodiments, the spatial boundary information includes door information for the clean environment; the spatial dividing lines in the environmental map include door boundary dividing lines. Based on the spatial boundary information, the spatial structure information in the clean environment is supplemented in the environmental map, including: supplementing the door boundary dividing lines in the environmental map based on the door information to obtain candidate dividing lines; performing spatial segmentation on the environmental map based on the candidate dividing lines to obtain an initial spatial set; using the initial spatial set as the current spatial set, determining a space with an area smaller than a preset area from the current spatial set as the current space to be merged, removing the candidate dividing lines between the current space to be merged and the adjacent space, so as to perform spatial merging processing on the current space to be merged and the adjacent space; using the spatially merged spatial set as the new current spatial set, returning to iteratively execute the step of determining a space with an area smaller than a preset area from the current spatial set as the current space to be merged, and stopping the iteration when there is no space with an area smaller than the preset area in the current spatial set, so as to obtain an environmental map after spatial segmentation correction.

[0139] It can be understood that since the door boundary segmentation lines in the environment map are not accurate enough, the spatial division based on the door boundary segmentation lines is not accurate enough. Therefore, the door boundary segmentation lines in the environment map can be supplemented based on the door information in the three-dimensional environment reconstruction result, and the supplemented door boundary segmentation lines in the environment map are used as candidate segmentation lines. Then, the optimal segmentation line is determined from these candidate segmentation lines to correct the spatial segmentation of the environment map. It can be understood that the optimal segmentation line is not a single line, but a group. That is, a group of optimal segmentation lines can divide the environment map into as many spaces as possible, and the area of ​​each space is not less than the preset area. In other words, under the premise that the area of ​​the space divided by the environment map is not less than the preset area, the group of spatial segmentation lines that can divide the environment map into the most spaces is the optimal segmentation line.

[0140] Specifically, the cleaning device can perform spatial segmentation on the environmental map based on the candidate dividing line to obtain an initial spatial set. The initial spatial set is used as the current spatial set, and the space with an area smaller than the preset area is determined from the current spatial set as the current space to be merged, and the candidate dividing line between the current space to be merged and the adjacent space is removed to perform spatial merging of the current space to be merged and the adjacent space. It can be understood that after the spatial merging, the spatial set is also updated, and the spatial merging process can be used as the new current spatial set, and the iteration is returned to execute the step of determining the space with an area smaller than the preset area from the current spatial set as the current space to be merged, until there is no space with an area smaller than the preset area in the current spatial set, and the iteration is stopped to obtain the environmental map after the spatial segmentation correction. In this way, the area of ​​the space divided by the environmental map can be not less than the preset area, and the space divided by the environmental map can be maximized, so as to be closer to the real environment and improve the accuracy of the spatial division.

[0141] In some embodiments, the three-dimensional environment reconstruction result includes object information in the clean environment. Modifying the environment map based on the registered three-dimensional environment reconstruction result includes: identifying objects in adjacent spaces from the environment map according to the object category in the object information; and adding space dividing lines for dividing adjacent spaces in the environment map based on the positions of the objects in the adjacent spaces.

[0142] Among them, adjacent spaces refer to two adjacent spaces. For example, the living room and the dining room are adjacent spaces.

[0143] Specifically, the cleaning device can identify objects in adjacent spaces from the environment map according to the object categories in the object information in the three-dimensional environment reconstruction result. Then, the positions of the objects in the adjacent spaces are determined, and based on the positions of the objects in the adjacent spaces, space dividing lines for dividing the adjacent spaces are added to the environment map.

[0144] For example, by using the sofa category and dining table and chair category in the object information, it is possible to determine that the sofa and dining table and chair are two objects located in the living room and dining room respectively. Then, based on the positions of the sofa and dining table and chair in the environment map, the dividing line between the living room and dining room is determined.

[0145] In the above embodiment, the object information in the clean environment is included in the three-dimensional environment reconstruction result, and the space division line for dividing adjacent spaces is added to the environment map, thereby improving the accuracy and information content of the space division.

[0146] In some embodiments, the three-dimensional environment reconstruction result includes object information in the clean environment; correcting the environment map based on the registered three-dimensional environment reconstruction result includes: identifying the space type of each space in the environment map used to place objects according to the object category in the object information; and adding the identified space types to the environment map.

[0147] In some embodiments, the space type includes but is not limited to at least one of a kitchen, a living room, a dining room, a study, a bathroom, a toilet, a balcony, a corridor or a bedroom.

[0148] In some embodiments, a correspondence between object categories and space types is preset, and the cleaning device can identify the space types of various spaces in the environment map for placing objects based on the correspondence and the object categories in the object information. In other embodiments, the cleaning device can also perform associative spatial analysis on the objects based on the object categories to determine the space types of various spaces in the environment map for placing objects. For example, a bathroom can be identified through bathtub information, and a study can be identified through desk information.

[0149] After identifying the space type, the cleaning device may add the identified space type to the environment map accordingly.

[0150] It is understandable that we usually only know which areas or spaces are in the environmental map, but we don’t know what specific space types these areas and spaces are. Therefore, by using the object information in the cleaning environment included in the 3D environmental reconstruction results, we can add space types to the environmental map, which greatly enriches the information volume of the map and enables the cleaning equipment to know the specific space types of each space in the environmental map, thus enhancing the cleaning equipment’s ability to understand the environmental space and facilitating the subsequent cleaning tasks.

[0151] Fig.10 is an environment map after the supplementary information is corrected in one embodiment. Fig.10 It can be seen that compared with Figure 3 It has richer semantic information and more accurate map information.

[0152] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiment of the present application also provides a cleaning equipment map processing device for implementing the above-mentioned cleaning equipment map processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in the embodiments of the map processing device for one or more cleaning equipment provided below can refer to the above-mentioned limitations on the cleaning equipment map processing method, and will not be repeated here.

[0154] In some embodiments, Fig.11 As shown, a map processing device for cleaning equipment is provided, including: an acquisition module 1102, a registration module 1104 and a correction module 1106, wherein:

[0155] The acquisition module 1102 is used to acquire an environment map constructed by the cleaning device; the environment map is constructed by the cleaning device based on the environmental information collected for the cleaning environment; the initial three-dimensional environment reconstruction result reconstructed by the auxiliary device for the cleaning environment; the initial three-dimensional environment reconstruction result is the reconstruction of the cleaning environment based on the collected images after the auxiliary device collects images of the cleaning environment; the collection range of the auxiliary device for the cleaning environment is larger than the collection range of the cleaning device;

[0156] A registration module 1104 is used to obtain registration information and register the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result and is used to register the initial three-dimensional environment reconstruction result with the environment map;

[0157] The correction module 1106 is used to correct the environment map based on the registered three-dimensional environment reconstruction result.

[0158] In some embodiments, the correction module 1106 is further used to obtain a first pose of the environmental object in the environmental map; and correct the first pose of the environmental object based on a second pose of the environmental object in the registered three-dimensional environmental reconstruction result.

[0159] In some embodiments, the correction module 1106 is also used to perform a posture comparison between the second posture and the first posture of the environmental object in the three-dimensional environment reconstruction result; when a posture overlapping object is determined based on the posture comparison result, the source of the first posture of the posture overlapping object is determined; the posture overlapping object is an environmental object where the first posture and the second posture overlap; when the source represents that the first posture is obtained through custom editing, the first posture of the posture overlapping object is retained in the environment map.

[0160] In some embodiments, the correction module 1106 is further configured to replace the first pose of the environmental object in the environmental map with the second pose of the environmental object in the three-dimensional environmental reconstruction result when the source representation first pose is identified by the cleaning device itself.

[0161] In some embodiments, the correction module 1106 is also used to update the second posture of the environmental object to be updated in the three-dimensional environment reconstruction result and add it to the environment map when the environmental object to be updated in the three-dimensional environment reconstruction result is determined based on the posture comparison result; wherein the environmental object to be updated is an environmental object that exists in the three-dimensional environment reconstruction result but does not exist in the environment map.

[0162] In some embodiments, the correction module 1106 is further used to fuse the first pose of each environmental object in the three-dimensional environment reconstruction result with the second pose of the environmental object in the environment map to obtain the final pose of the environmental object.

[0163] In some embodiments, the three-dimensional environment reconstruction result includes information about penetrable boundaries in the clean environment; the penetrable boundary is a boundary environment object that can be penetrated by laser; the correction module 1106 is also used to determine whether the penetrable boundary is a preset degraded boundary based on the information about the penetrable boundary; the degraded boundary is a boundary that can cause map degradation; when the penetrable boundary is a degraded boundary, the degraded area corresponding to the degraded boundary in the environment map is removed.

[0164] In some embodiments, the penetrable boundary includes a window; the information of the penetrable boundary includes the window height of the window; the correction module 1106 is also used to determine whether the window is a floor-to-ceiling window that can cause map degradation based on the window height; when the window is a floor-to-ceiling window, the degraded area caused by the floor-to-ceiling window in the environmental map is removed.

[0165] In some embodiments, the three-dimensional environment reconstruction result includes spatial boundary information for the clean environment; the correction module 1106 is further used to supplement the spatial structure information of the clean environment in the environment map based on the spatial boundary information.

[0166] In some embodiments, the spatial boundary information includes door information for a clean environment; the spatial dividing lines in the environmental map include door boundary dividing lines; the correction module 1106 is also used to supplement the door boundary dividing lines in the environmental map based on the door information to obtain candidate dividing lines; the environmental map is spatially divided based on the candidate dividing lines to obtain an initial spatial set; the initial spatial set is used as the current spatial set, and a space with an area smaller than a preset area is determined from the current spatial set as the current space to be merged, and the candidate dividing lines between the current space to be merged and the adjacent space are removed to perform spatial merging processing on the current space to be merged and the adjacent space; the spatial set after the spatial merging processing is used as the new current spatial set, and the iteration is returned to execute the step of determining a space with an area smaller than a preset area from the current spatial set as the current space to be merged, until there is no space with an area smaller than the preset area in the current spatial set, and the iteration is stopped to obtain the environmental map after the spatial segmentation correction.

[0167] In some embodiments, the three-dimensional environment reconstruction result includes object information in the clean environment; the correction module 1106 is also used to identify objects in adjacent spaces from the environment map based on the object category in the object information; and based on the positions of the objects in the adjacent spaces, supplement the environment map with space dividing lines for dividing adjacent spaces.

[0168] In some embodiments, the three-dimensional environment reconstruction result includes object information in the clean environment; the correction module 1106 is also used to identify the space type of each space used to place objects in the environment map according to the object category in the object information; and add the identified space type to the environment map accordingly.

[0169] In some embodiments, the registration module 1104 is also used to extract information of the target environment object from the three-dimensional reconstruction result; convert the extracted information of the target environment object into an object point cloud; convert the object point cloud into different preset postures, and project the object point cloud after posture conversion to the environment map; determine the target posture corresponding to the object point cloud with the highest overlap with the environment map after projection; transform and adjust the target environment object in the three-dimensional environment reconstruction result according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

[0170] Each module in the map processing device of the cleaning equipment can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the user terminal in the form of hardware, or can be stored in the memory in the user terminal in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0171] In some embodiments, a computer device is provided, which may be a cleaning device or an auxiliary device. It is understood that the auxiliary device may be a terminal. The internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a map processing method of a computer device is implemented.

[0172] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0173] In some embodiments, a cleaning device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the method provided in each embodiment of the present application are implemented.

[0174] In some embodiments, a terminal is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the method provided in each embodiment of the present application are implemented.

[0175] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in each embodiment of the present application are implemented.

[0176] In some embodiments, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in the methods provided in various embodiments of the present application.

[0177] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A map processing method for cleaning equipment, It is characterized in that The method comprises: Acquire an environment map constructed by a cleaning device; the environment map is constructed by the cleaning device based on environmental information collected for a cleaning environment; Obtaining an initial three-dimensional environment reconstruction result of the cleaning environment reconstructed by the auxiliary device; the initial three-dimensional environment reconstruction result is obtained by reconstructing the cleaning environment based on the collected image after the auxiliary device collects images of the cleaning environment; the collection range of the cleaning environment by the auxiliary device is larger than the collection range of the cleaning device; Acquire registration information, and register the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to register the initial three-dimensional environment reconstruction result with the environment map; The environment map is corrected based on the registered three-dimensional environment reconstruction result.

2. The method according to claim 1, It is characterized in that The step of correcting the environment map based on the registered three-dimensional environment reconstruction result includes: Obtaining a first position of an environmental object in the environmental map; Based on the second pose of the environmental object in the registered three-dimensional environment reconstruction result, the first pose of the environmental object is corrected.

3. The method according to claim 2, It is characterized in that The first posture of the environmental object is corrected based on the second posture of the environmental object in the registered three-dimensional environment reconstruction result, including: Comparing a second posture of the environment object in the three-dimensional environment reconstruction result with the first posture; In the case where a posture overlapping object is determined based on the posture comparison result, determining the source of the first posture of the posture overlapping object; the posture overlapping object is an environmental object where the first posture and the second posture overlap; In a case where the source representation indicates that the first pose is obtained through custom editing, the first pose of the pose overlapping object is retained in the environment map.

4. The method according to claim 3, It is characterized in that The method further comprises: In the case where the source representation indicates that the first posture is obtained by recognition by the cleaning device itself, the first posture of the environmental object in the environmental map is replaced by a second posture of the environmental object in the three-dimensional environment reconstruction result.

5. The method according to claim 3, It is characterized in that The method further comprises: In the case where an environmental object to be updated in the three-dimensional environment reconstruction result is determined based on the position and posture comparison result, updating and supplementing the second position and posture of the environmental object to be updated in the three-dimensional environment reconstruction result into the environmental map; The to-be-updated environment object is an environment object that exists in the three-dimensional environment reconstruction result but does not exist in the environment map.

6. The method according to claim 2, It is characterized in that The first posture of the environmental object is corrected based on the second posture of the environmental object in the registered three-dimensional environment reconstruction result, including: For each environmental object in the three-dimensional environmental reconstruction result, a first pose of the environmental object in the three-dimensional reconstruction result is fused with a second pose of the environmental object in the environmental map to obtain a final pose of the environmental object.

7. The method according to claim 1, It is characterized in that The three-dimensional environment reconstruction result includes information about the penetrable boundary in the clean environment; the penetrable boundary is a boundary environment object that can be penetrated by laser; The correcting of the environment map based on the registered three-dimensional environment reconstruction result includes: Determining whether the penetrable boundary is a preset degraded boundary based on the information of the penetrable boundary; the degraded boundary is a boundary that can cause map degradation; In the case where the penetrable boundary is a degraded boundary, a degraded area corresponding to the degraded boundary in the environment map is removed.

8. The method according to claim 7, It is characterized in that The penetrable boundary includes a window; the information of the penetrable boundary includes a window height of the window; determining whether the penetrable boundary is a preset degraded boundary based on the information of the penetrable boundary; The degraded boundary is a boundary that can cause map degradation, including: Determining whether the window is a floor-to-ceiling window that can cause map degradation based on the window height; When the penetrable boundary is a degraded boundary, removing the degraded area corresponding to the degraded boundary in the environment map includes: In the case where the window is a floor-to-ceiling window, the degraded area caused by the floor-to-ceiling window in the environment map is removed.

9. The method according to claim 1, It is characterized in that The three-dimensional environment reconstruction result includes spatial boundary information for the clean environment; The step of correcting the environment map based on the registered three-dimensional environment reconstruction result includes: Based on the spatial boundary information, the spatial structure information in the clean environment is supplemented in the environment map.

10. The method according to claim 9, It is characterized in that The space boundary information includes door information for the clean environment; the space dividing lines in the environment map include door boundary dividing lines; The method of supplementing the spatial structure information in the clean environment in the environmental map based on the spatial boundary information includes: Supplement the door boundary segmentation line in the environment map based on the door information to obtain a candidate segmentation line; Performing spatial segmentation on the environment map based on the candidate segmentation lines to obtain an initial spatial set; The initial space set is used as the current space set, a space whose area is smaller than a preset area is determined from the current space set as the current space to be merged, and a candidate dividing line between the current space to be merged and an adjacent space is removed to perform space merging processing on the current space to be merged and the adjacent space; The spatial set after the spatial merging process is used as the new current spatial set, and the step of iteratively executing the step of determining a space with an area smaller than a preset area from the current spatial set as the current space to be merged is returned until the iteration is stopped when there is no space with an area smaller than the preset area in the current spatial set, thereby obtaining an environment map after spatial segmentation correction.

11. The method according to claim 1, It is characterized in that The three-dimensional environment reconstruction result includes object information in the clean environment; and the correction of the environment map based on the registered three-dimensional environment reconstruction result includes: identifying objects in an adjacent space from the environment map according to the object category in the object information; Based on the position of the object in the adjacent space, a space dividing line for dividing the adjacent space is added to the environment map.

12. The method according to claim 1, It is characterized in that The three-dimensional environment reconstruction result includes object information in the clean environment; and the correction of the environment map based on the registered three-dimensional environment reconstruction result includes: identifying, according to the object category in the object information, a space type of each space in the environment map for placing the object; The identified space types are added to the environment map accordingly.

13. The method according to any one of claims 1 to 12, It is characterized in that The obtaining of registration information and registering the initial three-dimensional environment reconstruction result with the environment map based on the registration information includes: Extracting information of the target environment object from the three-dimensional reconstruction result; Converting the extracted information of the target environment object into an object point cloud; Converting the object point cloud into different preset poses, and projecting the object point cloud after pose conversion onto the environment map; Determine the target pose corresponding to the object point cloud having the highest overlap with the environment map after projection; The target environment object in the three-dimensional environment reconstruction result is transformed and adjusted according to the target posture to align the three-dimensional environment reconstruction result with the environment map.

14. A map processing device for cleaning equipment, It is characterized in that The device comprises: An acquisition module is used to acquire an environment map constructed by a cleaning device; the environment map is constructed by the cleaning device based on the environmental information collected for the cleaning environment; an initial three-dimensional environment reconstruction result reconstructed by an auxiliary device for the cleaning environment; the initial three-dimensional environment reconstruction result is the reconstruction of the cleaning environment based on the collected image after the auxiliary device collects images of the cleaning environment; the collection range of the cleaning environment by the auxiliary device is larger than the collection range of the cleaning device; A registration module, used to obtain registration information, and to register the initial three-dimensional environment reconstruction result with the environment map based on the registration information; the registration information is determined by transforming and adjusting the position and posture of the target environment object in the initial three-dimensional environment reconstruction result, and is used to register the initial three-dimensional environment reconstruction result with the environment map; The correction module is used to correct the environment map based on the registered three-dimensional environment reconstruction result.

15. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer device, which may be a cleaning device or a user terminal, comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

Images