Method and apparatus for processing three-dimensional map, electronic device and medium

Abstract

This invention discloses a method, apparatus, electronic device, and medium for processing three-dimensional maps. The method includes: acquiring a three-dimensional map and determining the ground area to be filled; segmenting the ground area to be filled to obtain multiple sub-ground areas to be filled; determining the missing areas in each sub-ground area and the target color for each missing area; filling each missing area according to the target color to obtain the target ground area; and updating the three-dimensional map according to the target ground area. According to the method, apparatus, electronic device, and medium of this invention, missing parts of the ground in a three-dimensional map can be quickly filled, optimizing the display effect of the three-dimensional map and improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of map processing technology, and in particular to a method, apparatus, electronic device, and medium for processing three-dimensional maps. Background Technology

[0002] With the rapid development of science and technology, artificial intelligence technology is becoming increasingly mature. The application of localization and mapping functions in robots is becoming more widespread and sophisticated, and the accuracy of localization and mapping is also gradually improving. It relies on sensors to perceive the surrounding environment, simultaneously describing both the robot's own position and the environment. By matching the robot's current observations with historical mapping data, the robot's latest position can be determined. Once the latest position is determined, the latest observations can be applied to the current position, updating and supplementing the mapping.

[0003] However, in related technologies, the various implementation schemes in the direction of 3D reconstruction have too high requirements for computing power and time consumption. Most of them are offline solutions or rely on GPUs, making it difficult to achieve real-time operation under the limited computing power and memory conditions of mobile robots. Even if 3D map reconstruction can be performed on mobile robots, the final 3D model will have some missing ground because the mobile robot cannot completely cover the surrounding environment during operation, which affects the aesthetics. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one objective of this invention is to provide a method for processing three-dimensional maps that can quickly fill in missing parts of the ground in a three-dimensional map, optimize the display effect of the three-dimensional map, and improve the user experience.

[0005] The second objective of this invention is to provide a three-dimensional map processing apparatus.

[0006] A third objective of this invention is to provide a computer-readable storage medium.

[0007] The fourth objective of this invention is to provide an electronic device.

[0008] To achieve the above objectives, a first aspect of the present invention provides a method for processing a three-dimensional map. The method includes: acquiring a three-dimensional map and determining a ground region to be filled in the three-dimensional map; segmenting the ground region to be filled to obtain multiple sub-ground regions to be filled; determining a void region in each sub-ground region to be filled and a target color for each void region; filling each void region according to the target color to obtain a target ground region; and updating the three-dimensional map according to the target ground region.

[0009] According to the three-dimensional map processing method of the present invention, the ground area to be filled in the three-dimensional map is divided into multiple sub-ground areas to be filled, and then the missing areas in the sub-ground areas to be filled are filled, thereby quickly filling the missing parts of the ground in the three-dimensional map, optimizing the display effect of the three-dimensional map, and improving the user experience.

[0010] In some embodiments of the present invention, determining the ground region to be filled in the three-dimensional map includes: filtering map points in the three-dimensional map whose z-axis coordinate component is 0, and using them as ground points of the three-dimensional map; determining a first minimum and a first maximum value of the ground points of the three-dimensional map on the x-axis coordinate component, and determining a second minimum and a second maximum value of the ground points of the three-dimensional map on the y-axis coordinate component; determining a plurality of target vertices based on the first minimum, the first maximum, the second minimum, and the second maximum value, and using the ground region enclosed by the plurality of target vertices as the ground region to be filled in the three-dimensional map.

[0011] In some embodiments of the present invention, updating the three-dimensional map according to the target ground area includes: determining outdoor ground points in the target ground area and filtering the outdoor ground points in the target ground area; updating the three-dimensional map according to the filtered target ground area.

[0012] In some embodiments of the present invention, determining the outdoor ground point in the target ground area includes: determining the line vector connecting the nearest wall point to each ground point in the edge region of the target ground area; calculating the angle between each line vector and the normal vector of the corresponding wall point; and determining the ground point corresponding to the line vector as the outdoor ground point in the target ground area based on the angle.

[0013] In some embodiments of the present invention, dividing the ground area to be filled to obtain a plurality of sub-ground areas to be filled includes: dividing the ground area to be filled into a first preset number of parts along the x-axis and dividing the ground area to be filled into a second preset number of parts along the y-axis to obtain the plurality of sub-ground areas to be filled; wherein the first preset number of parts and the second preset number of parts are both positive integers greater than or equal to 2.

[0014] In some embodiments of the present invention, determining the empty area in each sub-ground area to be filled includes: obtaining the number of ground points in each sub-ground area to be filled; if the number of ground points in the sub-ground area to be filled is less than or equal to a first preset number, then the sub-ground area to be filled is taken as the empty area.

[0015] In some embodiments of the present invention, determining the target color of each of the vacant areas includes: calculating the average color of all ground points in each sub-ground area to be filled, and using the average color as the average color of the corresponding sub-ground area to be filled; selecting a second preset number of sub-ground areas to be filled adjacent to the vacant area; wherein each of the selected sub-ground areas to be filled is a non-vacant area; calculating the color-weighted average of the average colors of the second preset number of sub-ground areas to be filled, and using the color-weighted average as the target color of the vacant area.

[0016] In some embodiments of the present invention, the weight of the average color of each adjacent sub-ground region to be filled is negatively correlated with the distance between each adjacent sub-ground region to be filled and the empty region.

[0017] In some embodiments of the present invention, filling each of the vacant areas according to the target color to obtain a target ground area includes: determining the vacant position of each of the vacant areas; and filling the vacant position of each of the vacant areas according to the target color to obtain a target ground area.

[0018] In some embodiments of the present invention, determining the vacancy position of each of the vacancy regions includes: traversing each position of each of the vacancy regions and determining the ground point closest to each position; calculating a first distance between the position and the nearest ground point corresponding to the position; if the first distance is greater than a first threshold, then the position is taken as the vacancy position of the vacancy region.

[0019] In some embodiments of the present invention, after updating the three-dimensional map according to the target ground area, the method further includes: traversing each map point of the updated three-dimensional map and determining the map point closest to each map point; calculating a second distance between the map point and the nearest map point corresponding to the map point, and inserting a new map point between the map point and the nearest map point corresponding to the map point when the second distance is greater than a second threshold; and determining the parameters of the new map point according to the parameters of the map point and the parameters of the nearest map point corresponding to the map point.

[0020] In some embodiments of the present invention, acquiring a three-dimensional map includes: determining an RGB image, a depth image, and an odometry pose at the same time; determining a current local map based on the RGB image, depth image, and odometry pose at the same time; matching the current local map with a global map to correct the odometry pose; updating the current local map based on the corrected odometry pose and a sliding window strategy; and fusing the updated current local map with the RGB image to update the global map and generate the three-dimensional map.

[0021] To achieve the above objectives, a second aspect of the present invention provides a three-dimensional map processing apparatus, the processing apparatus comprising: an acquisition module for acquiring a three-dimensional map and determining ground regions to be filled in the three-dimensional map; a segmentation module for segmenting the ground regions to be filled to obtain multiple sub-ground regions to be filled; a determination module for determining empty regions in each sub-ground region to be filled and a target color for each empty region; and a filling module for filling each empty region according to the target color to obtain a target ground region, and updating the three-dimensional map according to the target ground region.

[0022] According to the 3D map processing apparatus of the present invention, multiple sub-regions to be filled are obtained by dividing the ground area to be filled in the 3D map, and then the missing areas in the sub-regions to be filled are filled, thereby quickly filling the missing parts of the ground in the 3D map, optimizing the display effect of the 3D map, and improving the user experience.

[0023] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a three-dimensional map processing program thereon, which, when executed by a processor, implements the three-dimensional map processing method described in any of the above embodiments.

[0024] According to the computer-readable storage medium of the present invention, by segmenting the ground area to be filled in a 3D map to obtain multiple sub-ground areas to be filled, and then filling the missing areas in the sub-ground areas to be filled, the missing parts of the ground in the 3D map can be quickly filled, the display effect of the 3D map can be optimized, and the user experience can be improved.

[0025] To achieve the above objectives, a fourth aspect of the present invention provides an electronic device, the electronic device including a memory, a processor, and a three-dimensional map processing program stored in the memory and executable on the processor, wherein when the processor executes the three-dimensional map processing program, it implements the three-dimensional map processing method described in any of the above embodiments.

[0026] According to the electronic device of the present invention, by dividing the ground area to be filled in a three-dimensional map into multiple sub-ground areas to be filled, and then filling the missing areas in the sub-ground areas to be filled, the missing parts of the ground in the three-dimensional map can be quickly filled, the display effect of the three-dimensional map can be optimized, and the user experience can be improved.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0029] Figure 1 This is a flowchart illustrating a method for processing a three-dimensional map according to an embodiment of the present invention;

[0030] Figure 2 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of a three-dimensional map processing method according to an embodiment of the present invention;

[0032] Figure 4 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of a three-dimensional map processing method according to another embodiment of the present invention;

[0034] Figure 6 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0035] Figure 7 This is a schematic diagram of a three-dimensional map processing method according to another embodiment of the present invention;

[0036] Figure 8 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0037] Figure 9 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0038] Figure 10 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0039] Figure 11 This is a schematic diagram illustrating the effect of a three-dimensional map processing method according to an embodiment of the present invention;

[0040] Figure 12 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0041] Figure 13 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0042] Figure 14 This is a schematic diagram illustrating the effect of densification processing in a three-dimensional map processing method according to an embodiment of the present invention;

[0043] Figure 15 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0044] Figure 16 This is a schematic diagram of a three-dimensional map processing method according to another embodiment of the present invention;

[0045] Figure 17 This is a schematic diagram illustrating the effect of pose correction in a three-dimensional map processing method according to an embodiment of the present invention.

[0046] Figure 18 This is a flowchart illustrating a method for processing a three-dimensional map according to another embodiment of the present invention;

[0047] Figure 19 This is a structural block diagram of a three-dimensional map processing apparatus according to an embodiment of the present invention;

[0048] Figure 20 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0049] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0050] To clearly illustrate the three-dimensional map processing method, apparatus, electronic device, and medium of the embodiments of the present invention, the following is in conjunction with... Figure 1 The flowchart illustrating the 3D map processing method is described below. For example... Figure 1 As shown, the three-dimensional map processing method of this embodiment of the invention includes the following steps:

[0051] S11: Obtain a 3D map and determine the ground area to be filled in the 3D map;

[0052] S13: Divide the area to be filled into multiple sub-areas;

[0053] S15: Determine the empty areas in each sub-ground area to be filled and the target color for each empty area;

[0054] S17: Fill each empty area according to the target color to obtain the target ground area, and update the 3D map according to the target ground area.

[0055] According to the three-dimensional map processing method of the present invention, the ground area to be filled in the three-dimensional map is divided into multiple sub-ground areas to be filled, and then the missing areas in the sub-ground areas to be filled are filled, thereby quickly filling the missing parts of the ground in the three-dimensional map, optimizing the display effect of the three-dimensional map, and improving the user experience.

[0056] Specifically, the acquired 3D map can be a map generated in real time by the cleaning robot, or a 3D map pre-stored on a cloud server or by the cleaning robot; there are no limitations on this. The acquired 3D map may contain gaps in the ground area, affecting the viewing experience. The forms of the 3D map include, but are not limited to, point cloud maps, uniform-size pixel maps, variable-size pixel maps, and curved surface maps.

[0057] A 3D map can include multiple map points and empty locations. A map point can be understood as a location in the 3D map with known coordinates, color, and normal vector information; that is, the coordinates, color, and normal vector information of each map point are definite and known. An empty location can be understood as a location in the 3D map with unknown coordinates, color, and normal vector information; that is, the coordinates, color, and normal vector information of each empty location are empty and unknown.

[0058] In a 3D map, map points corresponding to the ground in an actual indoor environment are called ground points; that is, ground points represent the ground in an actual indoor environment. Similarly, map points corresponding to walls in an actual indoor environment are called wall points; that is, wall points represent the walls in an actual indoor environment. Finally, map points corresponding to furniture in an actual indoor environment are called furniture points; that is, furniture points represent the furniture in an actual indoor environment.

[0059] The area to be filled can be understood as the region containing all known ground points in the 3D map. It is worth noting that, depending on how the area to be filled is determined, its area can be larger than, smaller than, or equal to the area of ​​the indoor floor; no limitation is made here.

[0060] Depending on the distribution of ground points within the area to be filled and the method of dividing the area, the number of ground points in a certain sub-area of ​​the ground to be filled may be zero, one, or more than one; no limit is imposed here.

[0061] Empty areas can be understood as sub-regions of the ground that are expected to be filled; non-empty areas can be understood as sub-regions of the ground that are not expected to be filled. In other words, for a given sub-region of the ground to be filled, it is either an empty area or a non-empty area. Filling empty areas while leaving non-empty areas unfilled improves the efficiency of ground filling and saves computational resources.

[0062] In some embodiments, after step S17, the method for processing the 3D map further includes displaying the updated 3D map. This allows the updated 3D map to be presented to the user, showcasing indoor environmental details and enhancing the user experience.

[0063] Please see Figure 2 In some embodiments of the present invention, step S11 includes:

[0064] S111: Filter map points in the 3D map whose z-axis coordinate component is 0, and use them as ground points in the 3D map;

[0065] S113: Determine the first minimum and first maximum values ​​of the ground points of the 3D map on the x-axis coordinate component, and determine the second minimum and second maximum values ​​of the ground points of the 3D map on the y-axis coordinate component.

[0066] S115: Determine multiple target vertices based on the first minimum value, the first maximum value, the second minimum value, and the second maximum value, and use the ground area enclosed by the multiple target vertices as the ground area to be filled in the three-dimensional map.

[0067] In this way, a larger area of ​​ground to be filled can be identified, and the missing parts of the ground in the 3D map can be filled to a greater extent.

[0068] Specifically, the ground points of the 3D map can be traversed to find the first minimum value x_min and the first maximum value x_max of the ground points on the x-axis coordinate component, as well as the second minimum value y_min and the second maximum value y_max of the ground points on the y-axis coordinate component. Then, multiple target vertices can be (x_min, y_min, 0), (x_max, y_min, 0), (x_max, y_max, 0), and (x_min, y_max, 0). When the target vertex is a vertex of a rectangle, the ground area corresponding to the rectangle formed by (x_min, y_min, 0), (x_max, y_min, 0), (x_max, y_max, 0), and (x_min, y_max, 0) is the ground area to be filled.

[0069] Please combine Figure 3In one example, the coordinates of ground point A are (6, 2, 0), the coordinates of ground point B are (1, 8, 0), the coordinates of ground point C are (15, 10, 0), and the coordinates of ground point D are (16, 9, 0). By traversing the ground points of the 3D map, we can determine that the first minimum value of the ground points on the x-axis is 1, the first maximum value on the x-axis is 16, the second minimum value on the y-axis is 2, and the second maximum value on the y-axis is 10. Then, we can determine the four rectangular vertices E (1, 2, 0), F (16, 1, 0), G (16, 10, 0), and H (1, 10, 0), and determine the ground area to be filled based on the rectangle formed by vertices E, F, G, and H.

[0070] Please see Figure 4 In some embodiments of the present invention, step S17 includes:

[0071] S171: Identify outdoor ground points in the target ground area and filter the outdoor ground points in the target ground area;

[0072] S173: Update the 3D map based on the filtered target ground area.

[0073] This ensures that the 3D map matches the indoor environment and avoids the appearance of outdoor ground points in the 3D map.

[0074] Specifically, please combine Figure 5 When the area of ​​the ground to be filled is larger than the actual indoor ground area, after filling the empty areas of the ground to be filled, the resulting target ground area may include both indoor and outdoor ground points. However, outdoor ground points are not needed in the 3D map, and outdoor ground points can be regarded as incorrectly filled points. By filtering out the outdoor points in the target ground area, the accuracy of the 3D map can be effectively guaranteed, making the 3D map match the indoor environment.

[0075] Please see Figure 6 In some embodiments of the present invention, step S171 includes:

[0076] S1711: Determine the line vector connecting the nearest wall point to each ground point in the edge region of the target ground area;

[0077] S1713: Calculate the angle between each connecting vector and the normal vector of the corresponding wall point;

[0078] S1715: Determine the ground point corresponding to the connecting vector based on the included angle as the outdoor ground point in the target ground area.

[0079] In this way, outdoor ground points in the target ground area can be accurately and quickly determined. It is understandable that, considering that outdoor ground points are usually located at the edge of the target ground area, calculating the ground points in the edge area of ​​the target ground area while not calculating the ground points in other areas of the target ground area can save computing power, avoid increasing unnecessary calculations, and filter outdoor ground points more quickly.

[0080] Specifically, the edge region of the target ground area can be understood as the region consisting of all ground points whose distance from the boundary of the target ground area is less than or equal to a set threshold.

[0081] Determine the nearest wall point to each ground point in the edge region of the target ground area, and determine the projection point of the nearest wall point in the target ground area. Calculate the line vector connecting each ground point in the edge region of the target ground area to the projection point of the corresponding nearest wall point in the target ground area. The direction of this line vector can be from the ground point to the projection point of the nearest wall point in the target ground area, or it can be from the projection point of the nearest wall point in the target ground area to the ground point. No limitation is made here.

[0082] The normal vector of the wall point points inward.

[0083] Please combine Figure 7 (a) When the direction of the connecting vector is from the ground point to the projection point N of the nearest wall point in the target ground area, if the angle between the connecting vector and the normal vector of the corresponding wall point is greater than or equal to 0 degrees and less than 90 degrees, then the ground point can be determined as an outdoor ground point L; if the angle between the connecting vector and the normal vector of the corresponding wall point is greater than 90 degrees and less than or equal to 180 degrees, then the ground point can be determined as an indoor ground point M.

[0084] Please combine Figure 7 (b) When the direction of the connecting vector is from the projection point of the nearest wall point on the target ground area to the ground point N, if the angle between the connecting vector and the normal vector of the corresponding wall point is greater than or equal to 0 degrees and less than 90 degrees, then the ground point can be determined as an indoor ground point M; if the angle between the connecting vector and the normal vector of the corresponding wall point is greater than 90 degrees and less than or equal to 180 degrees, then the ground point can be determined as an outdoor ground point L.

[0085] In some embodiments of the present invention, step S13 includes:

[0086] S131: Divide the ground area to be filled into a first preset number of parts along the x-axis and divide the ground area to be filled into a second preset number of parts along the y-axis to obtain multiple sub-ground areas to be filled; wherein the first preset number of parts and the second preset number of parts are both positive integers greater than or equal to 2.

[0087] In this way, the area to be filled can be quickly divided into multiple sub-areas to be filled.

[0088] Specifically, the division can be uniform or non-uniform, and is not limited here. In some embodiments, the ground area to be filled is uniformly divided into a first preset number of parts along the x-axis and uniformly divided into a second preset number of parts along the y-axis, thereby obtaining multiple sub-ground areas to be filled with the same area. Sub-ground areas with the same area are easier to calculate and judge.

[0089] It is understandable that the larger the values ​​of the first preset number of parts and the second preset number of parts, the higher the segmentation accuracy and the more accurate the calculation results.

[0090] Please see Figure 8 In some embodiments of the present invention, step S15 includes:

[0091] S151: Get the number of ground points in each sub-ground region to be filled;

[0092] S153: If the number of ground points in the sub-ground area to be filled is less than or equal to the first preset number, then the sub-ground area to be filled will be regarded as a vacant area.

[0093] In this way, the sub-ground area to be filled can be determined.

[0094] In one example, the first preset quantity is 10. If the number of ground points in a certain sub-ground area to be filled is less than or equal to 10, then the sub-ground area to be filled is considered to be expected to be filled, and the sub-ground area to be filled is regarded as a vacant area.

[0095] Please see Figure 9 In some embodiments of the present invention, step S15 includes:

[0096] S155: Calculate the average color of all ground points in each sub-ground area to be filled, and use the average color as the average color of the corresponding sub-ground area to be filled;

[0097] S157: Select a second preset number of sub-ground regions to be filled that are adjacent to the vacant region; wherein each of the selected sub-ground regions to be filled is a non-vacant region;

[0098] S159: Calculate the color-weighted average of the average colors of the second preset number of sub-ground areas to be filled, and use the color-weighted average as the target color of the missing area.

[0099] In this way, the target color of the missing area is determined based on the non-missing areas adjacent to the missing area, increasing the reliability.

[0100] Specifically, the second preset quantity is greater than or equal to 3. Adjacent should be understood as two regions being relatively close in distance, rather than specifically referring to two regions being zero distance apart and in contact with each other.

[0101] The non-empty regions can be sorted according to the distance between them and the empty regions. A second preset number of non-empty regions can be determined in order from near to far. Then, the color-weighted average of the average color of the second preset number of non-empty regions can be calculated, and the color-weighted average can be used as the target color of the empty region.

[0102] In some embodiments, in step S157, each selected sub-ground region to be filled may simultaneously include non-empty regions and empty regions. That is, the non-empty regions and empty regions adjacent to the empty regions are uniformly sorted according to distance, and a second preset number of sub-ground regions to be filled are sequentially determined in order from nearest to farthest. Then, the color-weighted average of the average colors of the second preset number of sub-ground regions to be filled is calculated, and this color-weighted average is used as the target color for the empty regions. This ensures that the target color better matches the surrounding environmental features, preventing the color of the new ground points generated after filling from differing too much from the colors of known ground points.

[0103] In some embodiments of the present invention, the weight of the average color of each adjacent sub-ground region to be filled is negatively correlated with the distance between each adjacent sub-ground region to be filled and the empty region.

[0104] Specifically, the greater the distance between adjacent sub-ground areas to be filled and the empty area, the smaller the weight of the average color of the adjacent sub-ground areas to be filled when calculating the color-weighted average, that is, the lower the proportion of the average color of the adjacent sub-ground areas to be filled; the smaller the distance between adjacent sub-ground areas to be filled and the empty area, the greater the weight of the average color of the adjacent sub-ground areas to be filled when calculating the color-weighted average, that is, the higher the proportion of the average color of the adjacent sub-ground areas to be filled.

[0105] Please see Figure 10 In some embodiments of the present invention, step S17 includes:

[0106] S175: Determine the location of the vacancy in each vacancy area;

[0107] S177: Fill the empty spaces in each empty area according to the target color to obtain the target ground area.

[0108] In this way, the missing positions in the missing area are filled without changing the parameters of the existing ground points in the missing area.

[0109] Specifically, filling can be understood as determining the coordinates, color, and normal vector information of the missing position, and then generating a new ground point in the missing position.

[0110] The process iterates through each empty position in each empty region and fills each empty position sequentially. This means filling all empty positions within an empty region with the target color of that region, ensuring that each empty position within the region is the target color. It can be understood that the normal vector information of each ground point is the same, all being (0, 0, 1), and the normal vector of the newly generated ground point after filling is also (0, 0, 1). The coordinate information of the newly generated ground point can be determined during the traversal of the empty positions. In one example, the filling effect of the empty positions is shown in the image below. Figure 11 As shown.

[0111] Please see Figure 12 In some embodiments of the present invention, step S175 includes:

[0112] S1751: Traverse every position in each empty region and determine the ground point closest to each position;

[0113] S1753: Calculate the first distance between the location and the nearest ground point corresponding to the location;

[0114] S1755: If the first distance is greater than the first threshold, then the position is taken as the vacant position of the vacant region.

[0115] In this way, the location of the vacancy in each vacancy area can be determined.

[0116] Specifically, the kdtree algorithm can be used to determine the nearest ground point to each location in the missing area, as well as the first distance between each location and its corresponding nearest ground point.

[0117] In one example, the first threshold is set to 1cm. If the first distance between a certain position in the vacant area and the nearest ground point is greater than 1cm, then that position is determined as the vacant position in the vacant area.

[0118] In one example, the first threshold is set to 2cm. If the first distance between a certain position in the vacant area and the nearest ground point is greater than 2cm, then that position is determined as the vacant position in the vacant area.

[0119] Please see Figure 13 In some embodiments of the present invention, after step S17, the method further includes:

[0120] S21: Iterate through each map point in the updated 3D map and determine the map point closest to each map point;

[0121] S23: Calculate the second distance between the map point and the nearest map point corresponding to the map point, and when the second distance is greater than the second threshold, insert a new map point between the map point and the nearest map point corresponding to the map point;

[0122] S25: Determine the parameters of the new map point based on the parameters of the map point and the parameters of the nearest map point corresponding to the map point.

[0123] Thus, the updated 3D map undergoes global densification processing, further optimizing its details and improving its display quality (e.g., ...). Figure 14 (As shown). It is understandable that when different environmental features (such as tables, wardrobes, floors, and walls) are observed at different distances, angles, and frequencies from the sensor during the initial generation of the 3D map, the map points in some areas of the 3D map are sparse. Although the map points in these areas are sparse, they retain complete structural information and can represent the corresponding environmental features to a certain extent. For example, a desktop actually includes 2000 points, but there are only 1000 map points corresponding to the desktop in the 3D map. These 1000 sparsely distributed map points retain complete structural information of the desktop, and although they can represent the desktop, the display effect is poor. The method in this embodiment can increase the number of map points corresponding to the desktop in the 3D map, so that the number of map points corresponding to the desktop after processing is greater than 1000, thereby improving the display effect of the desktop in the 3D map.

[0124] Specifically, the kdtree algorithm can be used to determine the nearest map point for each map point in the updated 3D map, as well as the second distance between every two nearest map points.

[0125] In one example, the second threshold is set to 1cm. If the second distance between any two nearest map points is greater than 1cm, a new map point is inserted between these two map points.

[0126] In one example, the second threshold is set to 2cm. If the second distance between any two nearest map points is greater than 2cm, a new map point is inserted between these two map points.

[0127] The parameters of the inserted new map point can include coordinate and color information. The parameters of the inserted new map point can be obtained by linear interpolation based on the parameters of the two map points.

[0128] Please see Figure 15 In some embodiments of the present invention, step S11 includes:

[0129] S31: Determine the RGB image, depth image, and odometry pose at the same time.

[0130] S33: Determine the current local map based on the RGB image, depth map, and odometry pose at the same time.

[0131] S35: Match the current local map with the global map to correct the odometry pose;

[0132] S37: Update the current local map based on the corrected odometry pose and sliding window strategy;

[0133] S39: Merge the updated current local map with the RGB image to update the global map and generate a 3D map.

[0134] In this way, low CPU / memory usage, real-time, and high-precision 3D reconstruction can be achieved on the cleaning robot to obtain a 3D map. It is understandable that due to limitations such as sensor detection distance, robot observation direction, and robot walking path, the generation of a 3D map may result in only partial observation of certain environmental features (such as tables, wardrobes, floors, and walls), leading to missing sections of walls in the 3D map and affecting its aesthetics.

[0135] Specifically, the RGB image can be acquired by an RGB camera, and the depth image can be acquired by a depth camera. When the difference between the timestamps of the RGB image and the depth image is less than a set value, the timestamps of the RGB image and the depth image can be considered to be aligned, that is, the RGB image and the depth image are at the same time.

[0136] Odometry pose refers to the position and orientation of the cleaning robot at the moment its timestamp is aligned with the RGB image. Position can be represented by three-dimensional coordinates (x, y, z). Orientation can be represented by pitch, yaw, and roll angles.

[0137] Odometer pose can be obtained from the pose signal output by the odometer sensor. When the timestamp corresponding to the pose signal cannot be aligned with the timestamp of the RGB image, an alignable pose signal can be obtained by interpolation using a pose signal earlier than the timestamp of the RGB image and a pose signal later than the timestamp of the RGB image. The odometer pose is then determined based on the interpolated pose signal. When the timestamp corresponding to the pose signal is aligned with the timestamp of the RGB image, the odometer pose can be directly determined based on the alignable pose signal. This ensures data accuracy, reduces errors, and avoids errors in the generated 3D map.

[0138] Odometry sensors include, but are not limited to, laser odometry, IMU odometry, and visual odometry. It is worth noting that laser odometry outputs pose signals with no error and is relatively accurate, while IMU odometry and visual odometry output pose signals with errors. The presence or absence of errors in the obtained pose signal can be determined based on the type of odometry sensor.

[0139] When there are errors in the pose signal output by the odometry sensor, a local map can be determined based on the nearest RGB image and nearest depth image of the most recently acquired preset number of frames (e.g., 10 frames). This helps to minimize errors in the generated 3D map. When there are no errors in the pose signal output by the odometry sensor and the cleaning robot's runtime is short, a loop closure detection algorithm can be used to determine RGB images and depth images from historical RGB and historical depth images that have a similar odometry pose to the current RGB image, and these are used as loop closure frames. A local map is then determined based on the nearest RGB image, nearest depth image, and loop closure frame. When there are no errors in the pose signal output by the odometry sensor and the cleaning robot's runtime is long, a breadth-first search algorithm can be used to determine loop closure frames and loop closure frames of loop closure frames. A local map is then determined based on the nearest RGB image, nearest depth image, loop closure frame, and loop closure frame of loop closure frame. This ensures the accuracy of the generated 3D map. In one example, image frames with loop closure relationships are as follows: Figure 16 As shown, when the cleaning robot moves to a new location, new, similar RGB and depth maps are added to the sliding window, and the corresponding map points are removed from the global map and added to the local map. When the differences between the RGB and depth maps gradually increase, they are removed from the sliding window, and the corresponding map points are removed from the local map and added to the global map, thus realizing the flow of the map, which is the sliding window strategy.

[0140] Furthermore, when there are errors in the pose signal output by the odometer sensor, the odometer pose can be corrected based on point-to-surface ICP matching technology. Then, the local map can be corrected based on the corrected odometer pose, with the correction effect as follows: Figure 17 As shown. That is, it iterates through each map point in the local map, finds several map points in the global map that are closest to that map point, constructs a least-squares plane using these points, records the distance from the map point in the local map to the least-squares plane and the normal vector of the least-squares plane, corrects the odometry pose by minimizing the distance from all map points in the local map to the corresponding least-squares plane, and then calculates the relative pose transformation matrix before and after pose correction: T_delta=T_cur*T_pre -1 For each map point P on the local map, the coordinates are multiplied by the transformation matrix T_delta to correct the position of the local map. The corrected coordinates P_new of the local map can be expressed by the following formula: P_new = T_delta * P_old. In some implementations, the global map can be a pre-generated and acquired 3D map that can represent at least some of the indoor environmental features.

[0141] Please see Figure 18 In some embodiments, step S39 includes:

[0142] S391: Segment the RGB image to obtain multiple superpixel regions;

[0143] S393: Based on the odometry pose and camera parameters, project each map point in the updated local map onto the pixel plane of the RGB image to obtain the projection coordinate information corresponding to each map point;

[0144] S395: Based on the projected coordinates, normal vectors, and depth information of each map point, determine several map points that have environmental characteristics consistent with a certain superpixel region;

[0145] S397: Integrate several map points into the corresponding superpixel regions to update the local map again, and update the global map based on the updated local map and the unintegrated superpixel regions to obtain a 3D map.

[0146] Specifically, in step S391, in some embodiments, pixels in the RGB image whose coordinate difference is less than or equal to a preset coordinate threshold, color difference is less than or equal to a preset color threshold, and depth difference is less than or equal to a preset depth threshold are divided into a superpixel region. Thus, by dividing into superpixel regions, a large amount of data can be merged, saving computing power and reducing memory usage. It can be understood that pixels with adjacent coordinates, similar colors, and similar depth values ​​usually correspond to the same environmental feature; therefore, it is unnecessary to use multiple pixels for description, and a single superpixel region can replace multiple pixels. Specifically, the average color of all pixels within the superpixel region is calculated and used as the color of the superpixel region; the average depth value of all pixels within the superpixel region is calculated and used as the depth value of the superpixel region; and the average normal vector of all pixels within the superpixel region is calculated and used as the normal vector of the superpixel region, thereby determining the parameters of the superpixel region. This improves robustness to noise. In some embodiments, the RGB image is evenly divided into several rectangular patches, and each rectangular patch is used as a superpixel region. In this way, superpixel regions can be quickly determined, and when an embedded device runs the method of this embodiment of the invention, it can effectively save computing power and improve efficiency.

[0147] In step S393, based on the odometry pose T and camera parameters K, each map point P in the updated local map is projected onto the pixel plane u of the RGB image: u = K * T * P, so as to obtain the projection coordinate information corresponding to each map point.

[0148] In step S395, when the projected coordinates of a map point match a superpixel region, the normal vector information and depth information of the map point and the superpixel region are further compared. If the difference between the normal vector information and depth information of the two is less than a set value, the map point can be determined to be a map point with environmental characteristics consistent with the superpixel region.

[0149] In step S397, the map point can be weighted and fused with the normal vector information, coordinate information, and color information of the superpixel region to update the local map again. The weight of the map point in the local map is the sum of the weights of the fused historical superpixel regions. The weight of the superpixel region is determined based on its depth information, and the weight is the square of the inverse of the depth. This fusion process saves memory and reduces noise interference. For unfused superpixel regions, they can be identified as newly observed environmental features in the RGB image. Using their color information, normal vector information, and coordinate information, and based on the current odometry pose, they are rotated to the global coordinate system to create new map points, supplementing the global map and thus obtaining a 3D map based on the global map.

[0150] In some implementations, considering that when the odometer sensor is an IMU (Implantable odometer), although enabling pose correction can greatly improve the accuracy of the corrected local map and odometer pose, a small amount of error will still accumulate, and drift will still occur after long-term operation. Therefore, it is necessary to maintain the global map accuracy in real time to ensure accurate matching between the local map and the global map. Furthermore, considering the highly structured operating environment with a large number of vertical and horizontal planes, map points with these features are extracted based on the normal vector information of the map points. These map points are then traversed sequentially, and based on the nearest few map points, a least-squares plane is constructed. These map points are moved along the normal direction of the least-squares plane to the corresponding least-squares plane, achieving convergence of all map points about the feature center plane. In this way, during the next ICP (Internal Mapping) matching, the local map can be effectively prevented from matching the wrong global map position, ensuring the accuracy of pose correction.

[0151] It should be noted that the specific values ​​mentioned above are only for illustrating the implementation of the present invention in detail, and should not be construed as limiting the present invention. In other examples, implementation methods, or embodiments, other values ​​may be selected according to the present invention, and no specific limitations are made here.

[0152] To implement the above embodiments, this invention also proposes a three-dimensional map processing device, which can implement the three-dimensional map processing method of any of the above embodiments. Figure 19 This is a schematic diagram of a three-dimensional map processing apparatus according to an embodiment of the present invention. Figure 19As shown, the 3D map processing apparatus 100 proposed in this invention includes an acquisition module 12, a segmentation module 14, a determination module 16, and a filling module 18. The acquisition module 12 acquires a 3D map and determines the ground area to be filled in the 3D map. The segmentation module 14 segments the ground area to be filled to obtain multiple sub-ground areas to be filled. The determination module 16 determines the empty areas in each sub-ground area to be filled and the target color for each empty area. The filling module 18 fills each empty area according to the target color to obtain the target ground area and updates the 3D map according to the target ground area.

[0153] According to the 3D map processing apparatus 100 of the present invention, multiple sub-regions to be filled are obtained by dividing the ground area to be filled in the 3D map, and then the missing areas in the sub-regions to be filled are filled, thereby quickly filling the missing parts of the ground in the 3D map, optimizing the display effect of the 3D map, and improving the user experience.

[0154] In some embodiments of the present invention, the acquisition module 12 includes a filtering unit, a first determining unit, and a second determining unit. The filtering unit is used to filter map points in the 3D map whose z-axis coordinate component is 0, and use them as ground points of the 3D map. The first determining unit is used to determine a first minimum and a first maximum value of the ground points of the 3D map in the x-axis coordinate component, and to determine a second minimum and a second maximum value of the ground points of the 3D map in the y-axis coordinate component. The second determining unit is used to determine multiple target vertices based on the first minimum, first maximum, second minimum, and second maximum values, and to use the ground area enclosed by the multiple target vertices as the ground area to be filled in the 3D map.

[0155] In some embodiments of the present invention, the filling module 18 includes a filtering unit and a first updating unit. The filtering unit is used to determine outdoor ground points in the target ground area and filter the outdoor ground points in the target ground area. The first updating unit is used to update the three-dimensional map based on the filtered target ground area.

[0156] In some embodiments of the present invention, the filtering unit includes a first determining subunit, a first calculating subunit, and a second determining subunit. The first determining subunit is used to determine the line vector connecting the nearest wall point to each ground point in the edge region of the target ground area. The first calculating subunit is used to calculate the angle between each connecting vector and the normal vector of the corresponding wall point. The second determining subunit is used to determine, based on the angle, that the ground point corresponding to the connecting vector is an outdoor ground point in the target ground area.

[0157] In some embodiments of the present invention, the segmentation module 14 is further configured to divide the ground area to be filled into a first preset number of parts along the x-axis direction and divide the ground area to be filled into a second preset number of parts along the y-axis direction to obtain a plurality of sub-ground areas to be filled; wherein the first preset number of parts and the second preset number of parts are both positive integers greater than or equal to 2.

[0158] In some embodiments of the present invention, the determining module 16 includes an acquisition unit and a third determining unit. The acquisition unit is used to acquire the number of ground points in each sub-ground area to be filled. The third determining unit is used to designate the sub-ground area to be filled as a vacant area if the number of ground points in the sub-ground area to be filled is less than or equal to a first preset number.

[0159] In some embodiments of the present invention, the determining module 16 includes a first calculation unit, a selection unit, and a second calculation unit. The first calculation unit calculates the average color of all ground points within each sub-ground area to be filled, and uses this average value as the average color of the corresponding sub-ground area to be filled. The selection unit selects a second preset number of sub-ground areas to be filled adjacent to the vacant area; wherein each selected sub-ground area to be filled is a non-vacant area. The second calculation unit calculates the color-weighted average of the average colors of the second preset number of sub-ground areas to be filled, and uses this color-weighted average as the target color of the vacant area.

[0160] In some embodiments of the present invention, the weight of the average color of each adjacent sub-ground region to be filled is negatively correlated with the distance between each adjacent sub-ground region to be filled and the empty region.

[0161] In some embodiments of the present invention, the filling module 18 includes a fourth determining unit and a filling unit. The fourth determining unit is used to determine the position of the vacancy in each vacancy area. The filling unit is used to fill the vacancy position in each vacancy area according to the target color to obtain the target ground area.

[0162] In some embodiments of the present invention, the fourth determining unit includes a traversal subunit, a second calculation subunit, and a third determining subunit. The traversal subunit is used to traverse every position in each vacancy region and determine the nearest ground point to each position. The second calculation subunit is used to calculate a first distance between the position and the nearest ground point corresponding to the position. The third determining subunit is used to determine the position as a vacancy position in the vacancy region if the first distance is greater than a first threshold.

[0163] In some embodiments of the present invention, the three-dimensional map processing apparatus 100 further includes a traversal module, an interpolation module, and a parameter determination module. The traversal module is used to traverse each map point of the updated three-dimensional map and determine the nearest map point to each map point. The interpolation module is used to calculate a second distance between a map point and its nearest corresponding map point, and when the second distance is greater than a second threshold, insert a new map point between the map point and its nearest corresponding map point. The parameter determination module is used to determine the parameters of the new map point based on the parameters of the map point and the parameters of its nearest corresponding map point.

[0164] In some embodiments of the present invention, the acquisition module 12 further includes a fifth determining unit, a sixth determining unit, a correction unit, a second updating unit, and a fusion unit. The fifth determining unit is used to determine the RGB image, depth map, and odometry pose at the same time. The sixth determining unit is used to determine the current local map based on the RGB image, depth map, and odometry pose at the same time. The correction unit is used to match the current local map with the global map to correct the odometry pose. The second updating unit is used to update the current local map based on the corrected odometry pose and a sliding window strategy. The fusion unit is used to fuse the updated current local map with the RGB image to update the global map and generate a 3D map.

[0165] It should be noted that the above explanation of the implementation method and beneficial effects of the three-dimensional map processing method also applies to the three-dimensional map processing device 100 of the present invention. To avoid redundancy, it will not be elaborated in detail here.

[0166] To implement the above embodiments, this invention also proposes a computer-readable storage medium storing a three-dimensional map processing program thereon, which, when executed by a processor, implements the three-dimensional map processing method of any of the above embodiments.

[0167] According to the computer-readable storage medium of the present invention, by segmenting the ground area to be filled in a 3D map to obtain multiple sub-ground areas to be filled, and then filling the missing areas in the sub-ground areas to be filled, the missing parts of the ground in the 3D map can be quickly filled, the display effect of the 3D map can be optimized, and the user experience can be improved.

[0168] To implement the above embodiments, this invention also proposes an electronic device that can implement the three-dimensional map processing method of any of the above embodiments. Figure 20 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 20As shown, the electronic device 200 proposed in this invention includes a memory 22, a processor 24, and a three-dimensional map processing program 26 stored in the memory 22 and executable on the processor 24. When the processor 24 executes the three-dimensional map processing program 26, it implements the three-dimensional map processing method of any of the above embodiments.

[0169] According to the electronic device 200 of the present invention, multiple sub-regions to be filled are obtained by dividing the ground area to be filled in a three-dimensional map, and then the missing areas in the sub-regions to be filled are filled, thereby quickly filling the missing parts of the ground in the three-dimensional map, optimizing the display effect of the three-dimensional map, and improving the user experience.

[0170] Specifically, the electronic device 200 can be a device including a color camera, a depth camera, and an odometry sensor, such as cleaning robots (including but not limited to sweeping robots, mopping robots, sweeping and mopping robots, lawnmower robots, etc.), unmanned delivery vehicles, drones, warehouse robots, shopping mall service robots, food delivery robots, handheld 3D reconstruction scanners, fixed 3D reconstruction scanning devices, head-mounted AR devices, etc. In this case, the generation and post-processing of the 3D map can both be performed on the same electronic device. Alternatively, the electronic device 200 can be a server, laptop, personal computer, or other device that does not include a color camera, depth camera, and odometry sensor. In this case, the 3D map can be generated by other devices, and the electronic device 200 can acquire the 3D map generated by other devices and perform post-processing on the acquired 3D map. Other devices include, but are not limited to, cleaning robots (e.g., sweeping robots, mopping robots, sweeping and mopping robots, lawnmower robots, etc.), unmanned delivery vehicles, drones, warehouse robots, shopping mall service robots, food delivery robots, handheld 3D reconstruction scanners, fixed 3D reconstruction scanning devices, head-mounted AR devices, etc.

[0171] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0173] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0174] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0175] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0176] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0177] Furthermore, the terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this invention can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this invention, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly specified in the embodiments.

[0178] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for processing three-dimensional maps, characterized in that, The method is applied to intelligent cleaning devices for indoor cleaning, including: Acquire a 3D map and determine the ground area to be filled in the 3D map; The area to be filled is divided into multiple sub-areas to be filled; Determine the empty areas in each sub-ground area to be filled and the target color of each empty area; determining the target color of each empty area includes: determining the target color of the empty area based on the non-empty areas adjacent to the empty area. Each missing area is filled according to the target color to obtain a target ground area, and the 3D map is updated according to the target ground area; the updated 3D map is then displayed; when the area of ​​the ground area to be filled is larger than the actual indoor ground area, after filling the missing areas of the ground area to be filled to obtain the target ground area, the ground points of the edge area of ​​the target ground area are calculated to filter out outdoor points in the target ground area; the acquisition of the 3D map includes: Determine the RGB image, depth image, and odometry pose at the same time point; The current local map is determined based on the RGB image, depth map, and odometry pose at the same moment; The current local map is matched with the global map to correct the odometry pose; The current local map is updated based on the corrected odometer pose and sliding window strategy; The updated local map is fused with the RGB image to update the global map and generate the 3D map. This includes: sequentially traversing each map point in the current local map, finding several map points in the global map that are closest to the current map point, constructing a least-squares plane using these points, recording the distance from the current map point in the local map to the least-squares plane and the normal vector of the least-squares plane, correcting the odometry pose by minimizing the distance from all map points in the local map to the corresponding least-squares plane, and then obtaining the relative pose transformation matrix before and after pose correction. For the coordinates of each map point in the local map, the transformation matrix is ​​multiplied to correct the position of the local map.

2. The method for processing three-dimensional maps according to claim 1, characterized in that, Determining the ground area to be filled in the 3D map includes: Filter out map points in the 3D map whose z-axis coordinate component is 0, and use them as ground points of the 3D map; Determine the first minimum and first maximum values ​​of the ground points of the 3D map on the x-axis coordinate component, and determine the second minimum and second maximum values ​​of the ground points of the 3D map on the y-axis coordinate component; Multiple target vertices are determined based on the first minimum value, the first maximum value, the second minimum value, and the second maximum value, and the ground area enclosed by the multiple target vertices is used as the ground area to be filled in the three-dimensional map.

3. The method for processing three-dimensional maps according to claim 2, characterized in that, Updating the 3D map based on the target ground area includes: Identify outdoor ground points within the target ground area and filter the outdoor ground points within the target ground area; The 3D map is updated based on the filtered target ground area.

4. The method for processing three-dimensional maps according to claim 3, characterized in that, Determining the outdoor ground points in the target ground area includes: Determine the line vector connecting the nearest wall point to each ground point in the edge region of the target ground area; Calculate the angle between each connecting vector and the normal vector of the corresponding wall point; The ground point corresponding to the connecting vector determined by the included angle is the outdoor ground point in the target ground area.

5. The method for processing three-dimensional maps according to claim 2, characterized in that, The process of dividing the ground area to be filled into multiple sub-ground areas to be filled includes: The ground area to be filled is divided into a first preset number of parts along the x-axis and into a second preset number of parts along the y-axis to obtain the plurality of sub-ground areas to be filled; wherein the first preset number of parts and the second preset number of parts are both positive integers greater than or equal to 2.

6. The method for processing three-dimensional maps according to claim 1, characterized in that, The process of determining the empty areas in each sub-ground region to be filled includes: Obtain the number of ground points in each of the sub-ground regions to be filled; If the number of ground points in the sub-ground area to be filled is less than or equal to the first preset number, then the sub-ground area to be filled is regarded as the vacant area.

7. The method for processing three-dimensional maps according to claim 1, characterized in that, Determining the target color for each of the vacant regions includes: Calculate the average color of all ground points within each sub-ground area to be filled, and use the average color as the average color of the corresponding sub-ground area to be filled; Select a second preset number of sub-ground areas to be filled that are adjacent to the vacant area; wherein each of the selected sub-ground areas to be filled is a non-vacant area; Calculate the color-weighted average of the average colors of the second preset number of sub-ground areas to be filled, and use the color-weighted average as the target color of the missing area.

8. The method for processing three-dimensional maps according to claim 7, characterized in that, The weight of the average color of each adjacent sub-ground region to be filled is negatively correlated with the distance between each adjacent sub-ground region to be filled and the empty region.

9. The method for processing three-dimensional maps according to claim 1, characterized in that, The step of filling each missing area according to the target color to obtain the target ground area includes: Determine the location of the vacancy in each of the aforementioned vacancy regions; The missing positions of each of the missing areas are filled according to the target color to obtain the target ground area.

10. The method for processing three-dimensional maps according to claim 9, characterized in that, Determining the location of the vacancy in each of the vacancy regions includes: Iterate through each location in each of the aforementioned empty regions and determine the ground point closest to each location; Calculate the first distance between the location and the nearest ground point corresponding to the location; If the first distance is greater than the first threshold, then the position is taken as the vacant position of the vacant region.

11. The method for processing a three-dimensional map according to any one of claims 1-10, characterized in that, After updating the 3D map based on the target ground region, the method further includes: Iterate through each map point of the updated 3D map and determine the map point closest to each map point; Calculate the second distance between the map point and the nearest map point corresponding to the map point, and when the second distance is greater than a second threshold, insert a new map point between the map point and the nearest map point corresponding to the map point; The parameters of the new map point are determined based on the parameters of the map point and the parameters of the nearest map point corresponding to the map point.

12. A three-dimensional map processing device, characterized in that, A method for processing a three-dimensional map as described in any one of claims 1-11; the three-dimensional map processing device is applied to an intelligent cleaning device for indoor cleaning, comprising: The acquisition module is used to acquire a 3D map and determine the ground area to be filled in the 3D map; The segmentation module is used to segment the ground area to be filled into multiple sub-ground areas to be filled. A determination module is used to determine the empty areas in each sub-ground area to be filled and the target color of each empty area; the determination of the target color of each empty area includes: determining the target color of the empty area based on the non-empty areas adjacent to the empty area; The filling module is used to fill each of the missing areas according to the target color to obtain a target ground area, and update the 3D map according to the target ground area; and display the updated 3D map; when the area of ​​the ground area to be filled is larger than the actual indoor ground area, after filling the missing areas of the ground area to be filled to obtain the target ground area, the ground points of the edge area of ​​the target ground area are calculated to filter out the outdoor points in the target ground area.

13. A computer-readable storage medium, characterized in that, It stores a three-dimensional map processing program, which, when executed by a processor, implements the three-dimensional map processing method as described in any one of claims 1-11.

14. An electronic device, characterized in that, The system includes a memory, a processor, and a processing program for a three-dimensional map stored in the memory and executable on the processor. When the processor executes the processing program for the three-dimensional map, it implements the three-dimensional map processing method as described in any one of claims 1-11.

Images