Semantic map de-staggering method, device, storage medium and service robot

Abstract

The application provides a semantic map de-overlapping method and device, a readable storage medium and a service robot. The semantic map de-overlapping method comprises the following steps: in a plurality of object bounding boxes of a semantic map, two target bounding boxes are extracted, the object bounding box is a bounding box of an object in the semantic map; a target bounding box in the two target bounding boxes is determined relative to another target bounding box to obtain an overlap rate threshold and an actual overlap rate; and the two target bounding boxes are subjected to de-overlapping processing according to the overlap rate threshold and the actual overlap rate.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of semantic maps, and in particular, relates to a semantic map de-overlapping method and device, a readable storage medium, and a service robot. BACKGROUND

[0002] A service robot can model a home environment by recognizing and measuring objects such as home appliances and furniture, and combining a navigation map to form a semantic map with semantic information value.

[0003] In actual environment perception, due to positioning errors, measurement errors, detection errors, and other errors, the existence of various errors causes the three-dimensional models of real objects and semantic objects to be mismatched, including the problem of abnormal overlap between three-dimensional models of objects, which reduces the accuracy of the semantic map. SUMMARY

[0004] The present application aims to solve one of the technical problems in the prior art or related art.

[0005] To this end, the first aspect of the present application proposes a semantic map de-overlapping method.

[0006] The second aspect of the present application proposes a semantic map de-overlapping device.

[0007] The third aspect of the present application proposes a semantic map de-overlapping device.

[0008] The fourth aspect of the present application proposes a computer program product.

[0009] The fifth aspect of the present application proposes a readable storage medium.

[0010] The sixth aspect of the present application proposes a service robot.

[0011] Therefore, according to the first aspect of the present application, a semantic map de-overlapping method is proposed, which includes: in a plurality of object bounding boxes of a semantic map, extracting two target bounding boxes, the object bounding box being a bounding box of an object in the semantic map; determining an overlap rate threshold and an actual overlap rate of one of the two target bounding boxes relative to the other target bounding box; and performing de-overlapping processing on the two target bounding boxes according to the overlap rate threshold and the actual overlap rate.

[0012] The technical scheme of the present application proposes a semantic map de-overlapping method. Through the method, the de-overlapping processing can be performed between multiple object bounding boxes in the semantic map. In the de-overlapping process, whether there is an overlap between two target bounding boxes is accurately determined according to the actual overlap rate and the overlap rate threshold between the two target bounding boxes. Different overlap rate thresholds correspond to different target objects, which ensures that there is no abnormal overlap between the object bounding boxes in the semantic map after the de-overlapping processing, and the original appearance of the object is preserved to the greatest extent, thereby improving the accuracy of the semantic map and the precision of the service robot based on the semantic map for navigation.

[0013] In the process of constructing the semantic map, multiple object bounding boxes are generated in the semantic map, each object bounding box corresponding to a real object, and the object bounding box being a cuboid. By detecting whether there is an overlap between two object bounding boxes, it can be determined whether the three-dimensional models corresponding to the two objects overlap in the semantic map. In the case where the three-dimensional models corresponding to the two objects have abnormal overlap, the three-dimensional models corresponding to the two objects are de-overlapped.

[0014] After extracting the two target bounding boxes, the actual overlap rate of one target bounding box relative to the other target bounding box is determined, as well as the overlap rate threshold therebetween. The overlap rate threshold is a threshold preset in advance based on the type information of the two target bounding boxes. The overlap rate threshold of the two target bounding boxes includes two, i.e., the overlap rate threshold of bounding box A relative to bounding box B and the overlap rate threshold of bounding box B relative to bounding box A. When calculating the actual overlap rate therebetween, the above two cases are calculated respectively, i.e., the actual overlap rate of bounding box A relative to bounding box B and the actual overlap rate of bounding box B relative to bounding box A.

[0015] By comparing the corresponding overlap rate threshold and the actual overlap rate, it can be determined whether there is an abnormal overlap therebetween, and in the case where there is an abnormal overlap, the de-overlapping processing is performed therebetween, so that the actual overlap rate between the two target bounding boxes meets the preset condition, thereby removing the abnormal overlap between the two target bounding boxes.

[0016] In the technical scheme of the present application, by extracting two target bounding boxes from multiple object bounding boxes and based on the overlap rate threshold and the actual overlap rate between the two target bounding boxes, the two target bounding boxes are detected and de-overlapped, which avoids the abnormal overlap between the two target bounding boxes in the semantic map, preserves the original appearance of the object to the greatest extent, and improves the accuracy of the semantic map.

[0017] In the technical solution, the actual overlap rate of one of the two target bounding boxes relative to the other target bounding box is determined by: obtaining a first projection of a first bounding box on the ground and a second projection of a second bounding box on the ground; and determining the actual overlap rate based on the first projection and the second projection.

[0018] The technical solution provides a way to calculate the actual overlap rate of the two target bounding boxes, and the actual overlap rate between the two target bounding boxes can be calculated.

[0019] Specifically, the two target bounding boxes include a first bounding box and a second bounding box. First, a first projection of the first bounding box on the ground of a semantic map and a second projection of the second bounding box on the ground of the semantic map are obtained. The overlap rate between the two projections can be calculated based on the two projections, and the overlap rate between the two projections is taken as the actual overlap rate.

[0020] It can be understood that the first projection is the projection of the first bounding box on the ground in a direction perpendicular to the ground, and the second projection is the projection of the first bounding box on the ground in a direction perpendicular to the ground. The system can obtain the coordinate information of the first projection and the second projection on the ground, so as to obtain the overlap area between the first bounding box and the second bounding box, and thus calculate the actual overlap rate.

[0021] In the technical solution, the first projection and the second projection of the first bounding box and the second bounding box on the ground are obtained, and the actual overlap rate between the first bounding box and the second bounding box is calculated based on the first projection and the second projection, thereby improving the accuracy of calculating the actual overlap rate of the two target bounding boxes.

[0022] In any of the above technical solutions, the actual overlap rate is determined based on the first projection and the second projection, including: determining the intersection area of the first projection and the second projection; determining the ratio of the intersection area to the projection area of the second projection as the actual overlap rate of the first bounding box relative to the second bounding box; and determining the ratio of the intersection area to the projection area of the first projection as the actual overlap rate of the second bounding box relative to the first bounding box.

[0023] In the technical solution, the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box are calculated based on the intersection area between the first projection and the second projection, and the projection area of the first projection and the projection area of the second projection.

[0024] Specifically, in the calculation of the actual overlap rate of the first bounding box relative to the second bounding box, the intersection area is compared with the projection area of the second projection to obtain the area ratio, which is the actual overlap rate of the first bounding box relative to the second bounding box. In the calculation of the actual overlap rate of the second bounding box relative to the first bounding box, the intersection area is compared with the projection area of the first projection to obtain the area ratio, which is the actual overlap rate of the second bounding box relative to the first bounding box.

[0025] The first coordinate information of the first projection on the ground and the second coordinate information of the second projection on the ground are obtained, and the intersection area can be determined according to the first coordinate information and the second coordinate information.

[0026] In the technical solution of the present application, the actual overlap rates of the two target bounding boxes include the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box. The two actual overlap rates can be obtained by comparing the intersection area with the projection area of the first projection and the projection area of the second projection, respectively, thereby improving the accuracy of the actual overlap rates between the two target bounding boxes calculated.

[0027] In any of the above technical solutions, after the two target bounding boxes are processed by the de-overlapping processing according to the overlap rate threshold and the actual overlap rate, the method further comprises returning to the step of extracting the two target bounding boxes from the plurality of object bounding boxes in the semantic map until the actual overlap rate between any two object bounding boxes in the plurality of object bounding boxes after the processing meets the preset condition.

[0028] In this technical solution, the plurality of object bounding boxes are judged two by two and processed by de-overlapping during the de-overlapping of the entire semantic map, so that there is no abnormal overlap between the plurality of object bounding boxes in the semantic map after the de-overlapping processing.

[0029] Specifically, two target bounding boxes are extracted from the plurality of object bounding boxes, and the two target bounding boxes are the two bounding boxes with the highest confidence in the plurality of object bounding boxes. The two bounding boxes with the highest confidence are detected for abnormal overlap and are preferentially processed by de-overlapping. After the de-overlapping processing of the two bounding boxes, other target bounding boxes are extracted from the semantic map and are detected and processed by de-overlapping with the processed target bounding boxes, so that the plurality of object bounding boxes are detected and processed by de-overlapping two by two.

[0030] In the technical solution of the present application, the target bounding box is extracted by a loop, so that the plurality of object bounding boxes in the semantic map are detected and processed by de-overlapping. The abnormal overlap between the plurality of target bounding boxes in the semantic map is avoided, the original appearance of the object is preserved to the greatest extent, and the accuracy of the semantic map is improved.

[0031] In any of the above technical solutions, in the plurality of object bounding boxes of the semantic map, two target bounding boxes are extracted, including: obtaining a confidence of each object bounding box in the plurality of object bounding boxes; and determining two target bounding boxes in the plurality of object bounding boxes according to the confidence.

[0032] In this technical solution, the plurality of object bounding boxes are sorted according to the confidence of the plurality of object bounding boxes, and in the stage of extracting the target bounding boxes, the object bounding boxes with higher confidence are preferentially extracted as the target bounding boxes, which can improve the accuracy of the semantic map after the de-overlapping processing.

[0033] Specifically, the confidence of each object bounding box is determined in the process of constructing the semantic map, the plurality of object bounding boxes are sorted according to the confidence when the target bounding boxes are extracted, and the target bounding boxes are selected in turn according to the confidence sorting in the process of cyclically extracting two target bounding boxes.

[0034] In the technical solution of the present application, the target bounding boxes are extracted according to the confidence between the plurality of object bounding boxes, which realizes preferential detection and de-overlapping processing of the object bounding boxes with higher confidence, guarantees the high accuracy of the semantic map after the de-overlapping processing, and guarantees the integrity of the object bounding boxes after the de-overlapping processing.

[0035] In any of the above technical solutions, the two target bounding boxes include a third bounding box and a fourth bounding box, the actual overlap rate is an actual overlap rate of the fourth bounding box relative to the third bounding box, and the overlap rate threshold is an overlap rate threshold of the fourth bounding box relative to the third bounding box.

[0036] The de-overlapping processing of the two target bounding boxes according to the overlap rate threshold and the actual overlap rate includes: deleting the fourth bounding box when the actual overlap rate is greater than the overlap rate threshold; and clipping the fourth bounding box based on the third bounding box when the actual overlap rate is less than or equal to the overlap rate threshold.

[0037] In this technical solution, the overlap between the two target bounding boxes can be determined according to the overlap rate threshold, and when the overlap exists between the two target bounding boxes, the numerical relationship between the actual overlap rate and the overlap rate threshold between the two target bounding boxes can be compared to determine whether the abnormal overlap exists between the two target bounding boxes.

[0038] Specifically, the two target bounding boxes include a third bounding box and a fourth bounding box, and the actual overlap rate and the overlap rate threshold are respectively an actual overlap rate of the fourth bounding box relative to the third bounding box and an overlap rate threshold. When the actual overlap rate obtained by comparison is greater than the overlap rate threshold, it is determined that the fourth bounding box cannot maintain integrity after being cropped relative to the third bounding box, and therefore the fourth bounding box is directly deleted. When the actual overlap rate is less than or equal to the overlap rate threshold, it is determined that the fourth bounding box can maintain integrity after being cropped, and therefore the fourth bounding box is cropped.

[0039] In the technical solution of the present application, by comparing the numerical relationship between the overlap rate threshold and the actual overlap rate, it can be determined whether the fourth bounding box is affected in integrity after being cropped. In the case that the integrity of the fourth bounding box is affected, the fourth bounding box is deleted. In the case that the integrity of the fourth bounding box is not affected, the fourth bounding box is cropped. After the fourth bounding box is de-overlapped, abnormal overlap between the third bounding box and the fourth bounding box is avoided, and the integrity of the fourth bounding box is also ensured.

[0040] In any of the above technical solutions, the value range of the overlap rate threshold is 0 to 100%, wherein when the overlap rate threshold is 0, the fourth bounding box cannot overlap relative to the third bounding box, and when the overlap rate threshold is 100%, the fourth bounding box can completely overlap relative to the third bounding box.

[0041] In this technical solution, by limiting the value range of the overlap rate threshold, the accuracy of de-overlapping processing of the two target bounding boxes based on the overlap rate threshold and the actual overlap rate can be ensured.

[0042] Specifically, when the overlap rate threshold is 100%, it is determined that the bounding boxes corresponding to the two objects do not belong to abnormal overlap even if they completely overlap, and therefore de-overlapping processing is not required.

[0043] Exemplarily, the semantic map includes a table bounding box and a chair bounding box, wherein the table bounding box can completely overlap relative to the chair bounding box, that is, the chair is completely inside the table, and therefore the overlap rate threshold of the table bounding box and the chair bounding box can be set to 100%, and therefore de-overlapping processing is not required.

[0044] It should be noted that the actual overlap rate is the ratio of the area of the intersection part to the target bounding box, and therefore cannot exceed 100%, that is, the value range of the actual overlap rate is also 0 to 100%.

[0045] In any of the above embodiments, clipping the fourth bounding box based on the third bounding box comprises: determining a target side wall of the fourth bounding box according to a positional relationship between the third bounding box and the fourth bounding box in the two target bounding boxes; and translating the target side wall in a first direction until the fourth bounding box is separated from the third bounding box, the first direction being a direction from the target side wall to an interior of the fourth bounding box.

[0046] In the above technical solution, the third bounding box is any one of the two target bounding boxes, the fourth bounding box is the other target bounding box except the third bounding box in the two target bounding boxes, and the fourth bounding box is the target bounding box that needs to be clipped.

[0047] In this embodiment, since the object bounding boxes are all cubes, only one side wall of the object bounding box needs to be moved inward during the clipping process, that is, the object bounding box can be clipped. By finding the target side wall of the fourth bounding box that needs to be clipped and moving the target side wall to the interior of the fourth bounding box until the fourth bounding box is separated from the third bounding box, the clipping effect of the fourth bounding box is achieved.

[0048] It should be noted that during the clipping of the fourth bounding box, there can be multiple side walls that can make the fourth bounding box move away from the third bounding box by moving inward, and the side wall that can make the fourth bounding box retain the largest volume after moving is selected as the target side wall.

[0049] In the technical solution of the present application, the target side wall of the fourth bounding box that needs to be clipped is selected, and the target side wall is moved to the interior of the fourth bounding box, so that the clipping of the fourth bounding box is realized, and the clipping process is relatively convenient.

[0050] In any of the above technical solutions, the target side wall of the fourth bounding box is determined according to the positional relationship between the third bounding box and the fourth bounding box, comprising: obtaining an overlapping projection between the third bounding box and the fourth bounding box, the overlapping projection being a projection of an overlapping part of the third bounding box and the fourth bounding box on the ground; determining a distance value between a vertex of the overlapping projection and each side wall of the fourth bounding box; calculating a clipping volume based on the distance value and the area of the side wall; and determining the side wall with the smallest clipping volume as the target side wall.

[0051] In this technical solution, the target side wall is selected by the distance between the vertex of the overlapping projection and each side wall of the fourth bounding box that needs to be clipped, so that the volume of the fourth bounding box can be maximized by moving the target side wall, and the volume that needs to be clipped is reduced, further ensuring the integrity of the fourth bounding box.

[0052] Specifically, the overlapping projection can be a projection of the third bounding box and the fourth bounding box on the ground that intersects a region. The distance value is the distance between the vertex of the intersection region and the respective side wall in the direction perpendicular to the respective side wall. The corresponding cutting volume of each side wall can be calculated by multiplying the area of the respective side wall by the distance value. The side wall corresponding to the minimum cutting volume is selected as the target side wall.

[0053] In the technical solution of the present application, the cutting volume required for each side wall of the fourth bounding box to perform cutting is calculated, and the side wall with the smallest cutting volume is selected as the target side wall, so that the maximum volume can be preserved after the fourth bounding box is cut by translating the target side wall, thereby improving the integrity of the cut fourth bounding box.

[0054] In any of the above technical solutions, determining the overlap rate threshold of one of the two target bounding boxes relative to the other target bounding box includes: obtaining a plurality of category information corresponding to a plurality of object bounding boxes; constructing a preset overlap array based on the plurality of category information and prior information; and finding the overlap rate threshold in the preset overlap array according to the object information of the two target bounding boxes.

[0055] In this technical solution, the preset overlap array is generated based on the category information corresponding to the object bounding box and the corresponding prior information. After the two target bounding boxes are determined, the overlap rate threshold in the preset overlap array is screened to quickly and conveniently find the overlap rate threshold between the two target bounding boxes.

[0056] Specifically, the overlap rate of objects of different categories that can overlap in the actual scene is different, that is, the overlap rate of different objects that can overlap is different, so the possible overlap rate between different objects is preset according to the prior information, the overlap rate threshold between the object bounding boxes of the plurality of objects is obtained, and the overlap rate threshold is stored in the form of a preset overlap array, so that when detecting whether the two target bounding boxes have abnormal overlap, the corresponding overlap rate threshold can be quickly found.

[0057] In some possible technical solutions, when the two target bounding boxes overlap, a fifth bounding box in the two target bounding boxes is selected to perform a de-overlap process according to the overlap rate threshold, and the fifth bounding box corresponds to the overlap rate threshold.

[0058] In the technical solution of the present application, the overlap rate threshold is pre-stored in the form of a preset overlap array, which facilitates the calling during the de-overlap process of the two target bounding boxes, and the overlap rate threshold is set based on the prior information, which can ensure that the de-overlap process conforms to the actual situation of object placement, thereby improving the accuracy of the semantic map after the de-overlap process.

[0059] In any of the above technical solutions, the number of rows of the preset overlap array is equal to the number of columns, and the number of rows of the preset overlap array is equal to the number of category information.

[0060] In this technical solution, by setting the number of category information and the number of rows and columns of the preset overlap array to be equal, it can be ensured that corresponding overlap rate thresholds are set between all pairs of object bounding boxes, improving the integrity of the overlap rate thresholds.

[0061] Specifically, taking category information n as an example, the preset overlap array constructed by the n category information object bounding boxes is an n-row and n-column array.

[0062] According to the second aspect of the present application, a semantic map de-overlap device is provided, comprising: an extraction module for extracting two target bounding boxes from a plurality of object bounding boxes of a semantic map, the object bounding box being a bounding box of an object in the semantic map; a determination module for determining an overlap rate threshold and an actual overlap rate of one of the two target bounding boxes relative to the other target bounding box; a processing module for de-overlapping the two target bounding boxes according to the overlap rate threshold and the actual overlap rate; and the processing module is used to repeat the above steps until the actual overlap rate between any two object bounding boxes in the plurality of processed object bounding boxes meets a preset condition.

[0063] In the technical solution of the present application, a semantic map de-overlap device is provided, which can de-overlap a plurality of object bounding boxes in the semantic map. In the de-overlapping process, whether there is overlap between the two target bounding boxes is accurately determined according to the actual overlap rate and the overlap rate threshold between the two target bounding boxes, and different target objects correspond to different overlap rate thresholds, and the corresponding de-overlapping processing is performed on different target bounding boxes, so that the object bounding boxes in the semantic map after de-overlapping processing will not appear abnormal overlap phenomenon, and the original appearance of the object is preserved to the greatest extent, improving the accuracy of the semantic map and the precision of the service robot based on the semantic map for navigation.

[0064] In the technical solution of the present application, by extracting two target bounding boxes from a plurality of object bounding boxes and detecting and de-overlapping the two target bounding boxes based on the overlap rate threshold and the actual overlap rate between the two target bounding boxes, the plurality of object bounding boxes in the semantic map are detected and de-overlapped by repeating the step of extracting the target bounding box, so that the abnormal overlap phenomenon between the plurality of object bounding boxes in the semantic map is avoided, the original appearance of the object is preserved to the greatest extent, and the accuracy of the semantic map is improved.

[0065] According to a third aspect of the present application, a semantic map de-overlapping apparatus is provided, comprising: a memory, wherein a program or instructions are stored; and a processor, which executes the program or instructions stored in the memory to implement the steps of the semantic map de-overlapping method according to any one of the first aspect, thus having all the beneficial technical effects of the semantic map de-overlapping method according to any one of the first aspect, which will not be repeated here.

[0066] According to a fourth aspect of the present application, a computer program product is provided, which, when executed by a processor, implements the steps of the semantic map de-overlapping method according to any one of the first aspect, thus having all the beneficial technical effects of the semantic map de-overlapping method according to any one of the first aspect, which will not be repeated here.

[0067] According to a fifth aspect of the present application, a readable storage medium is provided, which stores a program or instructions, which, when executed by a processor, implements the steps of the semantic map de-overlapping method according to any one of the first aspect. Thus, it has all the beneficial technical effects of the semantic map de-overlapping method according to any one of the first aspect, which will not be repeated here.

[0068] According to a sixth aspect of the present application, a service robot is provided, comprising: the semantic map de-overlapping apparatus according to the second aspect or the third aspect, and / or the computer program product according to the fourth aspect, and / or the readable storage medium according to the fifth aspect, thus having all the beneficial technical effects of the semantic map de-overlapping apparatus according to the second aspect or the third aspect, and / or the computer program product according to the fourth aspect, and / or the readable storage medium according to the fifth aspect, which will not be repeated here.

[0069] Additional aspects and advantages of the present application will become apparent from the following description, or will be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0070] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0071] Figure 1 One of the schematic flowcharts of the semantic map de-overlapping method provided in some embodiments of the present application is shown;

[0072] Figure 2 Another of the schematic flowcharts of the semantic map de-overlapping method provided in some embodiments of the present application is shown;

[0073] Figure 3Fig. 1 shows a schematic diagram of a first projection and a second projection provided in some embodiments of the present application;

[0074] Figure 4 Fig. 3 shows a schematic flowchart of a method of de-occlusion of a semantic map provided in some embodiments of the present application;

[0075] Figure 5 Fig. 4 shows a schematic flowchart of a method of de-occlusion of a semantic map provided in some embodiments of the present application;

[0076] Figure 6 Fig. 5 shows a schematic diagram of a third bounding box and a fourth bounding box provided in some embodiments of the present application;

[0077] Figure 7 Fig. 6 shows a schematic flowchart of a method of de-occlusion of a semantic map provided in some embodiments of the present application;

[0078] Figure 8 Fig. 7 shows a schematic flowchart of a method of de-occlusion of a semantic map provided in some embodiments of the present application;

[0079] Figure 9 Fig. 8 shows a structural block diagram of a de-occlusion device of a semantic map provided in some embodiments of the present application;

[0080] Figure 10 Fig. 9 shows a structural block diagram of a de-occlusion device of a semantic map provided in some embodiments of the present application;

[0081] Figure 11 Fig. 10 shows a structural block diagram of a service robot provided in some embodiments of the present application. DETAILED DESCRIPTION

[0082] In order to enable every person skilled in the art to better understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the features in the embodiments and examples can be combined with each other as long as they do not conflict with each other.

[0083] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in other manners different from those described herein, and therefore the scope of the present application is not limited by the specific embodiments disclosed below.

[0084] The following description refers to the accompanying drawings that show embodiments of the present application. Figures 1 to 11 Methods, devices, computer program products, readable storage media and service robots of de-occlusion of a semantic map according to some embodiments of the present application are described.

[0085] According to one embodiment of the present application, as Figure 1As shown, a semantic map de-overlapping method is proposed, comprising:

[0086] Step 102, in the plurality of object bounding boxes of the semantic map, two target bounding boxes are extracted, and the object bounding box is the bounding box of the object in the semantic map;

[0087] Step 104, determining the overlap rate threshold and the actual overlap rate of one of the two target bounding boxes relative to the other target bounding box;

[0088] Step 106, de-overlapping processing is performed on the two target bounding boxes according to the overlap rate threshold and the actual overlap rate.

[0089] In an embodiment of the present application, a semantic map de-overlapping method is proposed, which can perform de-overlapping processing between a plurality of object bounding boxes in the semantic map. In the de-overlapping process, whether the two target bounding boxes overlap is accurately determined according to the actual overlap rate and the overlap rate threshold between the two target bounding boxes, and different target objects correspond to different overlap rate thresholds, so that the object bounding boxes in the semantic map after de-overlapping processing will not appear abnormal overlap phenomenon, and the original appearance of the object is preserved to the greatest extent, improving the accuracy of the semantic map and the precision of the service robot based on the semantic map for navigation.

[0090] In the process of constructing the semantic map, a plurality of object bounding boxes are generated in the semantic map, each object bounding box corresponds to a real object, and the object bounding box is a cuboid. By detecting whether there is overlap between two object bounding boxes, it can be determined whether the three-dimensional models corresponding to the two objects overlap in the semantic map. In the case where the three-dimensional models corresponding to the two objects have abnormal overlap, de-overlapping processing is performed on the three-dimensional models corresponding to the two objects.

[0091] Exemplarily, the plurality of object bounding boxes include a table bounding box, a chair bounding box and a bed bounding box. First, the table bounding box and the chair bounding box are selected as target bounding boxes for detection and de-overlapping processing, i.e. whether the chair bounding box is abnormally overlapped relative to the table bounding box is determined, and de-overlapping processing is performed. Then, the table bounding box and the bed bounding box are selected as target bounding boxes for detection and de-overlapping processing, i.e. whether the bed bounding box is abnormally overlapped relative to the table bounding box is determined, and de-overlapping processing is performed.

[0092] After extracting two target bounding boxes, the actual overlap rate of one target bounding box relative to the other, and the overlap rate threshold between them, are determined. The overlap rate threshold is a pre-set threshold based on the type information corresponding to the two target bounding boxes. There are two overlap rate thresholds: one for bounding box A relative to bounding box B, and another for bounding box B relative to bounding box A. When calculating the actual overlap rate between them, both scenarios are calculated separately: the actual overlap rate of bounding box A relative to bounding box B, and the actual overlap rate of bounding box B relative to bounding box A.

[0093] By comparing the corresponding overlap rate threshold and the actual overlap rate, it is possible to determine whether there is abnormal overlap between the two. If there is abnormal overlap, the overlap is removed so that the actual overlap rate between the two target bounding boxes meets the preset conditions, thereby removing the abnormal overlap between the two target bounding boxes.

[0094] In the embodiments of this application, by extracting two target bounding boxes from multiple object bounding boxes, and based on the overlap rate threshold and actual overlap rate between the two target bounding boxes, the two target bounding boxes are detected and de-overlapped, thus avoiding the phenomenon of abnormal overlap between two target bounding boxes in the semantic map, preserving the original appearance of the objects to the greatest extent while improving the accuracy of the semantic map.

[0095] like Figure 2 As shown, in the above embodiment, determining the actual overlap rate of one target bounding box relative to the other target bounding box includes:

[0096] Step 202: Obtain the first projection of the first bounding box on the ground and the second projection of the second bounding box on the ground.

[0097] Step 204: Determine the actual overlap rate based on the first projection and the second projection.

[0098] This embodiment provides a method for calculating the actual overlap rate between two target bounding boxes, which enables the calculation of the actual overlap rate between the two target bounding boxes.

[0099] Specifically, the two target bounding boxes include a first bounding box and a second bounding box. First, the first projection of the first bounding box on the ground of the semantic map and the second projection of the second bounding box on the ground of the semantic map are obtained. Based on the above two projections, the overlap rate between the two projections can be calculated and the overlap rate between the two projections is used as the actual overlap rate.

[0100] It can be understood that the first projection is a projection of the first bounding box on the ground in a direction perpendicular to the ground, and the second projection is a projection of the second bounding box on the ground in a direction perpendicular to the ground. The system can obtain coordinate information of the first projection and the second projection on the ground, so as to obtain an overlapping area between the first bounding box and the second bounding box, and thus calculate the actual overlap rate.

[0101] As shown in FIG. 3, the first projection 302 of the first bounding box overlaps with the second projection 304 of the second bounding box, and the first projection 302 and the second projection 304 are both rectangles. By using coordinate information of the first projection 302 and the second projection 304 on the ground, the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box can be calculated. Figure 3

[0102] In the embodiments of the present application, the first projection and the second projection of the first bounding box and the second bounding box on the ground are obtained, and the actual overlap rate between the first bounding box and the second bounding box is calculated according to the first projection and the second projection, thereby improving the accuracy of calculating the actual overlap rate of the two target bounding boxes.

[0103] As shown in FIG. 3, the first projection 302 of the first bounding box overlaps with the second projection 304 of the second bounding box, and the first projection 302 and the second projection 304 are both rectangles. By using coordinate information of the first projection 302 and the second projection 304 on the ground, the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box can be calculated. Figure 4

[0104] Step 402, determining an intersection area of the first projection and the second projection;

[0105] Step 404, determining, as the actual overlap rate of the first bounding box relative to the second bounding box, a ratio of the intersection area to a projection area of the second projection;

[0106] Step 406, determining, as the actual overlap rate of the second bounding box relative to the first bounding box, a ratio of the intersection area to a projection area of the first projection.

[0107] In this embodiment, the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box are calculated respectively by using the intersection area between the first projection and the second projection, and the projection area of the first projection and the projection area of the second projection.

[0108] Specifically, when calculating the actual overlap rate of the first bounding box relative to the second bounding box, the intersection area is compared with the projection area of the second projection to obtain an area ratio, which is the actual overlap rate of the first bounding box relative to the second bounding box. When calculating the actual overlap rate of the second bounding box relative to the first bounding box, the intersection area is compared with the projection area of the first projection to obtain an area ratio, which is the actual overlap rate of the second bounding box relative to the first bounding box.

[0109] ​​The first coordinate information projected on the ground and the second coordinate information projected on the ground are acquired, and the intersection area can be determined according to the first coordinate information and the second coordinate information.

[0110] As shown in the figure, the projection area of the first projection 302 is S1, the projection area of the second projection 304 is S2, and the intersection area is S2∩S1. Figure 3

[0111] Exemplarily, the actual overlap rate of the first bounding box relative to the second bounding box can be calculated by the following formula:

[0112]

[0113] S1 is the projection area of the first projection, S2 is the projection area of the second projection, and O 12 is the actual overlap rate of the first bounding box relative to the second bounding box.

[0114] Exemplarily, the actual overlap rate of the second bounding box relative to the first bounding box can be calculated by the following formula:

[0115]

[0116] S1 is the projection area of the first projection, S2 is the projection area of the second projection, and O 21 is the actual overlap rate of the second bounding box relative to the first bounding box.

[0117] In the embodiments of the present application, the actual overlap rate of the two target bounding boxes includes the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box, which can be obtained by ratio calculation of the intersection area and the projection area of the first projection and the projection area of the second projection, respectively, thereby improving the accuracy of the actual overlap rate between the two target bounding boxes calculated.

[0118] In any of the above embodiments, after the two target bounding boxes are processed by the de-overlapping processing according to the overlap rate threshold and the actual overlap rate, the method further comprises: returning to the step of extracting the two target bounding boxes from the plurality of object bounding boxes of the semantic map until the actual overlap rate between any two object bounding boxes in the plurality of object bounding boxes after processing meets the preset condition.

[0119] In this embodiment, during the de-overlapping process of the entire semantic map, the plurality of object bounding boxes are judged two by two and de-overlapping processing is performed, so that there is no abnormal overlap between the plurality of object bounding boxes in the semantic map after de-overlapping processing.

[0120] ​Specifically, two target bounding boxes are extracted from the plurality of object bounding boxes, the two target bounding boxes being the two bounding boxes with the highest confidence in the plurality of object bounding boxes, whether the two bounding boxes with the highest confidence are abnormally overlapped is detected, and the two bounding boxes with the highest confidence are preferentially subjected to the de-overlapping processing, after the two bounding boxes with the highest confidence are subjected to the de-overlapping processing, other target bounding boxes in the semantic map are extracted, and the other target bounding boxes are subjected to detection and de-overlapping processing with the processed target bounding boxes, so as to ensure that each two of the plurality of object bounding boxes are subjected to detection and de-overlapping processing.

[0121] Exemplarily, the plurality of object bounding boxes include a table bounding box, a chair bounding box, and a bed bounding box. The table bounding box and the chair bounding box are first selected as the target bounding boxes for detection and de-overlapping processing, that is, whether the chair bounding box is abnormally overlapped relative to the table bounding box is judged, and the de-overlapping processing is performed. Then, the table bounding box and the bed bounding box are selected as the target bounding boxes for detection and de-overlapping processing, that is, whether the bed bounding box is abnormally overlapped relative to the table bounding box is judged, and the de-overlapping processing is performed. Then, the bed bounding box and the chair bounding box are selected as the target bounding boxes for detection and de-overlapping processing, that is, whether the bed bounding box is abnormally overlapped relative to the chair bounding box is judged, and the de-overlapping processing is performed, until each two of the plurality of object bounding boxes are processed, and each two of the two target bounding boxes A and B are subjected to detection and de-overlapping processing twice, that is, A is subjected to detection and de-overlapping processing relative to B, and B is subjected to detection and de-overlapping processing relative to A.

[0122] In the embodiment of the present application, the target bounding box is extracted through the step of circulation, so that each of the plurality of object bounding boxes in the semantic map is subjected to detection and de-overlapping processing, the phenomenon that the plurality of target bounding boxes in the semantic map are abnormally overlapped is avoided, the original appearance of the object is preserved to the greatest extent, and the accuracy of the semantic map is improved.

[0123] As shown in Figure 5 In any of the above embodiments, two target bounding boxes are extracted from the plurality of object bounding boxes in the semantic map, including:

[0124] In step 502, the confidence of each object bounding box in the plurality of object bounding boxes is obtained.

[0125] In step 504, two target bounding boxes in the plurality of object bounding boxes are determined according to the confidence.

[0126] In this embodiment, the plurality of object bounding boxes are sorted according to the confidence of the plurality of object bounding boxes, and in the stage of extracting the target bounding box, the object bounding box with higher confidence is preferentially extracted as the target bounding box, so as to improve the accuracy of the semantic map after the de-overlapping processing.

[0127] Specifically, the confidence of each object bounding box is determined in the process of constructing the semantic map, the plurality of object bounding boxes are sorted according to the confidence when the target bounding box is extracted, and the target bounding box is selected in turn according to the confidence sorting in the process of cyclically extracting two target bounding boxes.

[0128] Exemplarily, the plurality of object bounding boxes include a table bounding box, a chair bounding box and a bed bounding box, and the confidence sorting is a bed bounding box, a chair bounding box and a table bounding box. Then the bed bounding box is taken as the first bounding box of the two target bounding boxes first, and the chair bounding box and the table bounding box are taken as the second bounding box of the two target bounding boxes in turn. Then the chair bounding box is taken as the first bounding box of the two target bounding boxes, and the bed bounding box and the table bounding box are taken as the second bounding box of the two target bounding boxes in turn. Finally, the table bounding box is taken as the first bounding box of the two target bounding boxes, and the chair bounding box and the bed bounding box are taken as the second bounding box of the two target bounding boxes in turn.

[0129] In the embodiment of the application, the target bounding box is extracted according to the confidence between the plurality of object bounding boxes, so that the object bounding box with higher confidence is preferentially detected and de-overlapped, the accuracy of the semantic map after de-overlapping is ensured to be high, and the integrity of the object bounding box after de-overlapping is ensured.

[0130] In any of the above embodiments, the two target bounding boxes include a third bounding box and a fourth bounding box, the actual overlap rate is the actual overlap rate of the fourth bounding box relative to the third bounding box, and the overlap rate threshold is the overlap rate threshold of the fourth bounding box relative to the third bounding box.

[0131] According to the overlap rate threshold and the actual overlap rate, the two target bounding boxes are de-overlapped, including:

[0132] In the case that the actual overlap rate is greater than the overlap rate threshold, the fourth bounding box is deleted.

[0133] In the case that the actual overlap rate is less than or equal to the overlap rate threshold, the fourth bounding box is cropped based on the third bounding box.

[0134] In this embodiment, the overlap between the two target bounding boxes can be determined according to the overlap rate threshold, and in the case that the overlap exists between the two target bounding boxes, the abnormal overlap between the two target bounding boxes can be determined by comparing the numerical relationship between the actual overlap rate and the overlap rate threshold.

[0135] Specifically, the two target bounding boxes include a third bounding box and a fourth bounding box, and the actual overlap rate and the overlap rate threshold are respectively an actual overlap rate of the fourth bounding box relative to the third bounding box and an overlap rate threshold. When the actual overlap rate obtained by comparison is greater than the overlap rate threshold, it is determined that the fourth bounding box cannot maintain integrity after being cropped relative to the third bounding box, and therefore the fourth bounding box is directly deleted. When the actual overlap rate is less than or equal to the overlap rate threshold, it is determined that the fourth bounding box can maintain integrity after being cropped, and therefore the fourth bounding box is cropped.

[0136] For example, the fourth bounding box is bounding box A, the third bounding box is bounding box B, the overlap rate threshold of the bounding box A relative to the bounding box B is 0, that is, the bounding box A cannot overlap the bounding box B, and the actual overlap rate of the bounding box A relative to the bounding box B is 0.2. It is determined that the bounding box A overlaps the bounding box B, and therefore the bounding box B is directly deleted.

[0137] For example, the fourth bounding box is bounding box A, the third bounding box is bounding box B, the overlap rate threshold between the bounding box A and the bounding box B is E, and the actual overlap rate between the bounding box A and the bounding box B is O. In the case of E

[0138] In the embodiments of the present application, by comparing the numerical relationship between the overlap rate threshold and the actual overlap rate, it can be determined whether the fourth bounding box is affected in integrity after being cropped. In the case that the fourth bounding box is affected in integrity, the fourth bounding box is deleted. In the case that the fourth bounding box is not affected in integrity, the fourth bounding box is cropped, so that after the fourth bounding box is de-overlapped, abnormal overlap between the third bounding box and the fourth bounding box is avoided, and the integrity of the fourth bounding box is also ensured.

[0139] In any of the above embodiments, the overlap rate threshold ranges from 0 to 100%, wherein the overlap rate threshold is 0, the fourth bounding box cannot overlap the third bounding box, and the overlap rate threshold is 100%, the fourth bounding box can completely overlap the third bounding box.

[0140] In this embodiment, by limiting the value range of the overlap rate threshold, the accuracy of de-overlapping processing of the two target bounding boxes based on the overlap rate threshold and the actual overlap rate can be ensured.

[0141] Specifically, in the case of the overlap rate threshold being 100%, it is determined that the bounding boxes corresponding to the two objects are not abnormal overlap even if they completely overlap, and therefore de-overlapping processing is not required.

[0142] Exemplarily, the semantic map includes a table bounding box and a chair bounding box, where the table bounding box can completely overlap the chair bounding box, that is, the chair is completely located inside the table, and then the overlap rate threshold of the table bounding box and the chair bounding box can be set as 100%, so that the two bounding boxes do not need to be de-overlapped.

[0143] It should be noted that the actual overlap rate is the ratio of the area of the intersection part to the target bounding box, and therefore cannot exceed 100%, that is, the value range of the actual overlap rate is also 0 to 100%.

[0144] In any of the above embodiments, the clipping of the fourth bounding box based on the third bounding box includes: determining a target side wall of the fourth bounding box according to the positional relationship between the third bounding box and the fourth bounding box in the two target bounding boxes; and translating the target side wall along a first direction until the fourth bounding box is separated from the third bounding box, the first direction being a direction from the target side wall to the inside of the fourth bounding box.

[0145] In the embodiments, the third bounding box is any one of the two target bounding boxes, the fourth bounding box is the other target bounding box except the third bounding box, and the fourth bounding box is the target bounding box that needs to be clipped.

[0146] In the embodiments, since the object bounding boxes are all cubes, only one side wall of the object bounding box needs to be moved inward during the clipping process, that is, the object bounding box can be clipped. By finding the target side wall in the fourth bounding box that needs to be clipped, and moving the target side wall to the inside of the fourth bounding box until the fourth bounding box is separated from the third bounding box, the effect of clipping the fourth bounding box is achieved.

[0147] It should be noted that during the clipping of the fourth bounding box, there can be multiple side walls that can make the fourth bounding box move away from the third bounding box by moving inward, and the side wall that can make the fourth bounding box retain the largest volume after moving is selected as the target side wall.

[0148] As shown in FIG. 6, Figure 6 The third bounding box 602 and the fourth bounding box 604 have an overlapping area, and by moving the target side wall 606 of the fourth bounding box 604 to the inside of the fourth bounding box 604 in the direction shown by the arrow A until the overlapping area of the third bounding box 602 and the fourth bounding box 604 disappears, the clipping operation of the fourth bounding box 604 is completed.

[0149] In the embodiments of the present application, by selecting the target side wall of the fourth bounding box that needs to be clipped, and moving the target side wall to the inside of the fourth bounding box, the clipping of the fourth bounding box is achieved, so that the clipping process is more convenient.

[0150] As shown in FIG. 7, Figure 7As shown in any of the above embodiments, the target side wall of the fourth bounding box is determined according to the positional relationship between the third bounding box and the fourth bounding box, including:

[0151] In step 702, an overlapping projection between the third bounding box and the fourth bounding box is obtained, the overlapping projection being a projection on the ground of an overlapping part of the third bounding box and the fourth bounding box;

[0152] In step 704, a distance value between a vertex of the overlapping projection and each side wall of the fourth bounding box is determined;

[0153] In step 706, a corresponding clipping volume is calculated based on the distance value and an area of the side wall;

[0154] In step 708, a side wall with a minimum clipping volume is determined as the target side wall.

[0155] In this embodiment, the target side wall is screened through the distance between the vertex of the overlapping projection and each side wall of the fourth bounding box to be clipped, so that the volume of the fourth bounding box can be maximized by moving the target side wall, the volume to be clipped is reduced, and the integrity of the fourth bounding box is further ensured.

[0156] Specifically, the overlapping projection can be an intersection area of the projections of the third bounding box and the fourth bounding box on the ground. The distance value is the distance between the vertex of the intersection area and each side wall in a direction perpendicular to the side wall. The corresponding clipping volume of each side wall can be calculated by multiplying the distance value by the area of the side wall, and the side wall corresponding to the minimum clipping volume is selected as the target side wall.

[0157] For example, in the process of clipping the fourth bounding box, the overlapping projection of the overlapping area of the third bounding box and the fourth bounding box on the ground is obtained. The vertex of the overlapping projection is determined, and the maximum vertical distance of the vertex on each side wall of the fourth bounding box is determined in sequence through the coordinate information of the vertex and the coordinate information of each side wall of the fourth bounding box, that is, the distance value. Each side wall of the fourth bounding box and the maximum vertical distance are multiplied to obtain the clipping volume. The corresponding face with the minimum clipping volume is selected, and the corresponding maximum vertical distance is translated inward.

[0158] In the embodiments of the present application, the clipping volume required when each side wall of the fourth bounding box is clipped is calculated, and the side wall with the minimum clipping volume is selected as the target side wall, so that the maximum volume can be retained after the fourth bounding box is clipped by translating the target side wall, thereby improving the integrity of the clipped fourth bounding box.

[0159] As shown in any of the above embodiments, the overlap rate threshold of one of the two target bounding boxes relative to the other target bounding box is determined, including: Figure 8 ​

[0160] At step 802, a plurality of category information corresponding to a plurality of object bounding boxes is acquired.

[0161] At step 804, a preset overlap array is constructed based on the plurality of category information and prior information.

[0162] At step 806, the overlap rate threshold is searched in the preset overlap array according to the object information of the two target bounding boxes.

[0163] In this embodiment, the preset overlap array is generated by the category information corresponding to the object bounding box and the corresponding prior information. After determining the two target bounding boxes, the overlap rate threshold in the preset overlap array is screened to quickly and conveniently find the overlap rate threshold between the two target bounding boxes.

[0164] Specifically, the objects of different category information can have different overlap rates in the actual scene, that is, the overlap rates of different objects may be different. Therefore, the possible overlap rates of different objects are preset according to the prior information, the overlap rate threshold between the object bounding boxes of a plurality of objects can be obtained, and the above overlap rate threshold is stored in the form of a preset overlap array, so that when detecting whether the two target bounding boxes have abnormal overlap, the corresponding overlap rate threshold can be quickly found.

[0165] Exemplarily, the preset overlap array is generated by taking the placement relationship of objects in daily life as prior information. Taking a wardrobe, a table and a chair as examples, the corresponding prior information is as follows: (1) the wardrobe cannot overlap with the table and the chair; (2) the table cannot overlap with the wardrobe, but can overlap with the chair; (3) the chair cannot overlap with the wardrobe, but can overlap with the table; (4) when the chair is completely located in the table, the chair is deleted.

[0166] The preset overlap array generated in combination with the prior knowledge (1), (2) and (3) is shown in Table 1 as follows:

[0167] Table 1

[0168] Wardrobe Table Chair Wardrobe 0 0 0 Table 0 0 1 Chair 0 1 0

[0169] The preset overlap array generated in combination with the prior knowledge (1), (2), (3) and (4) is shown in Table 2 as follows:

[0170] Table 2

[0171] Wardrobe Table Chair Wardrobe 0 0 0 Table 0 0 0.8 Chair 0 1 0

[0172] Exemplarily, taking a bed, a bedside table, a television and a television cabinet as examples, the preset overlap array is shown in Table 3 as follows:

[0173] Table 3

[0174] Bed Bedside table Television Television cabinet Bed 0 1 0 0 Bedside table 0.5 0 0 0 Television 0 0 0 1 Television cabinet 0 0 1 0

[0175] In some possible embodiments, when the two target bounding boxes overlap, the fifth bounding box in the two target bounding boxes is selected to perform the de-overlapping processing according to the overlap rate threshold value corresponding to the fifth bounding box.

[0176] For example, when the bed and the bedside table overlap, since the overlap rate threshold value of the bed relative to the bedside table is 1 and the overlap rate threshold value of the bedside table relative to the bed is 0.5, when the overlap occurs, the target bounding box corresponding to the bed is preferentially cropped or deleted.

[0177] In the embodiments of the present application, the overlap rate threshold values are pre-stored in a preset overlap array, which facilitates the calling in the de-overlapping processing of the two target bounding boxes, and the overlap rate threshold values are set based on prior information, which can ensure that the de-overlapping processing conforms to the actual situation of object placement and improves the accuracy of the semantic map after the de-overlapping processing.

[0178] In any of the above embodiments, the number of rows of the preset overlap array is equal to the number of columns, and the number of rows of the preset overlap array is equal to the number of category information.

[0179] In this embodiment, by setting the number of category information and the number of rows and columns of the preset overlap array to be equal, the corresponding overlap rate threshold values can be set between all pairs of object bounding boxes, thereby improving the integrity of the overlap rate threshold values.

[0180] Specifically, taking n category information as an example, the preset overlap array constructed by the n category information object bounding boxes is an n-row and n-column array.

[0181] In an embodiment according to the present application, as shown in FIG. 9, Figure 9 a de-overlapping device 900 for a semantic map is provided, which comprises:

[0182] An extraction module 902 is configured to extract two target bounding boxes from a plurality of object bounding boxes of a semantic map, the object bounding boxes being the bounding boxes of objects in the semantic map.

[0183] A determination module 904 is configured to determine an overlap rate threshold value and an actual overlap rate of one of the two target bounding boxes relative to the other target bounding box.

[0184] A processing module 906 is configured to perform de-overlapping processing on the two target bounding boxes according to the overlap rate threshold value and the actual overlap rate.

[0185] The embodiment of the present application provides a semantic map de-overlapping device, through the device, de-overlapping processing can be performed on a plurality of object bounding boxes in the semantic map, in the de-overlapping process, whether there is overlap between two target bounding boxes is accurately judged according to the actual overlap rate and the overlap rate threshold between the two target bounding boxes, and different overlap rate thresholds correspond to different two target objects, so that the abnormal overlap phenomenon between the object bounding boxes in the semantic map after the de-overlapping processing is avoided, the original appearance of the object is maximally reserved, the accuracy of the semantic map is improved, and the precision of the service robot based on the semantic map for navigation is improved.

[0186] In the embodiment of the present application, two target bounding boxes are extracted from a plurality of object bounding boxes, and the two target bounding boxes are detected and de-overlapped based on the overlap rate threshold and the actual overlap rate between the two target bounding boxes, so that the abnormal overlap phenomenon between the two target bounding boxes in the semantic map is avoided, the original appearance of the object is maximally reserved, and the accuracy of the semantic map is improved.

[0187] In the above embodiment, the semantic map de-overlapping device 900 further comprises:

[0188] The acquisition module is used for acquiring a first projection of a first bounding box on the ground and a second projection of a second bounding box on the ground.

[0189] The determination module 904 is used for determining the actual overlap rate based on the first projection and the second projection.

[0190] In the embodiment, a method for calculating the actual overlap rate of the two target bounding boxes is provided, and the actual overlap rate between the two target bounding boxes can be calculated through the method.

[0191] In the embodiment of the present application, the first projection and the second projection of the first bounding box and the second bounding box on the ground are acquired respectively, and the actual overlap rate between the first bounding box and the second bounding box is calculated according to the first projection and the second projection, so that the accuracy of calculating the actual overlap rate of the two target bounding boxes is improved.

[0192] In any of the above embodiments, the determination module 904 is used for determining the intersection area of the first projection and the second projection.

[0193] The determination module 904 is used for determining the ratio of the intersection area to the projection area of the second projection as the actual overlap rate of the first bounding box relative to the second bounding box.

[0194] The determination module 904 is used for determining the ratio of the intersection area to the projection area of the first projection as the actual overlap rate of the second bounding box relative to the first bounding box.

[0195] The actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box are respectively calculated by an intersection area between the first projection and the second projection and a projection area of the first projection and a projection area of the second projection.

[0196] In the embodiments of the present application, the actual overlap rates of the two target bounding boxes include the actual overlap rate of the first bounding box relative to the second bounding box and the actual overlap rate of the second bounding box relative to the first bounding box, which are respectively calculated by the ratio of the intersection area to the projection area of the first projection and the projection area of the second projection, so that the accuracy of the actual overlap rate between the two target bounding boxes is improved.

[0197] In any of the above embodiments, the processing module 906 is further configured to return to the step of extracting the two target bounding boxes from the plurality of object bounding boxes of the semantic map after the de-overlapping processing of the two target bounding boxes, until the actual overlap rate between any two object bounding boxes of the plurality of object bounding boxes after the processing meets the preset condition.

[0198] In the embodiments of the present application, the step of extracting the target bounding box is repeated, so that the plurality of object bounding boxes in the semantic map are detected and de-overlapped, which avoids the abnormal overlap between the plurality of target bounding boxes in the semantic map, maximizes the original appearance of the object, and improves the accuracy of the semantic map.

[0199] In any of the above embodiments, the obtaining module is configured to obtain the confidence of each object bounding box in the plurality of object bounding boxes.

[0200] The determining module 904 is configured to determine the two target bounding boxes in the plurality of object bounding boxes according to the confidence.

[0201] In this embodiment, the plurality of object bounding boxes are sorted according to the confidence, and in the step of extracting the target bounding box, the object bounding box with a higher confidence is preferentially extracted as the target bounding box, which can improve the accuracy of the semantic map after the de-overlapping processing.

[0202] In the embodiments of the present application, the target bounding box is extracted according to the confidence between the plurality of object bounding boxes, so that the object bounding box with a higher confidence is preferentially detected and de-overlapped, which ensures that the accuracy of the semantic map after the de-overlapping processing is high and the integrity of the object bounding box after the de-overlapping processing is ensured.

[0203] In any of the above embodiments, the two target bounding boxes include a third bounding box and a fourth bounding box, the actual overlap rate is an actual overlap rate of the fourth bounding box relative to the third bounding box, and the overlap rate threshold is an overlap rate threshold of the fourth bounding box relative to the third bounding box.

[0204] The processing module 906 is configured to delete the fourth bounding box when the actual overlap rate is greater than the overlap rate threshold.

[0205] The processing module 906 is configured to clip the fourth bounding box based on the third bounding box when the actual overlap rate is less than or equal to the overlap rate threshold.

[0206] In this embodiment, the overlap rate threshold can be used to determine whether the two target bounding boxes can overlap. When the two target bounding boxes can overlap, the actual overlap rate between the two target bounding boxes is compared with the overlap rate threshold to determine whether the two target bounding boxes overlap abnormally.

[0207] In the embodiments of the present application, the actual overlap rate is compared with the overlap rate threshold to determine whether the fourth bounding box is complete after being clipped. When the fourth bounding box is not complete, the fourth bounding box is deleted. When the fourth bounding box is complete, the fourth bounding box is clipped. Thus, the abnormal overlap between the third bounding box and the fourth bounding box is avoided after the de-overlapping processing of the fourth bounding box, and the completeness of the target bounding box is ensured.

[0208] In any of the above embodiments, the overlap rate threshold ranges from 0 to 100%. When the overlap rate threshold is 0, the fourth bounding box cannot overlap the third bounding box. When the overlap rate threshold is 100%, the fourth bounding box can completely overlap the third bounding box.

[0209] In this embodiment, the range of the overlap rate threshold is limited, which ensures the accuracy of the de-overlapping processing of the two target bounding boxes based on the overlap rate threshold and the actual overlap rate.

[0210] In any of the above embodiments, the determining module 904 is configured to determine a target side wall of the fourth bounding box according to a positional relationship between the third bounding box and the fourth bounding box in the two target bounding boxes.

[0211] The processing module 906 is configured to translate the target side wall in a first direction until the fourth bounding box is separated from the third bounding box, and the first direction is a direction from the target side wall to the inside of the fourth bounding box.

[0212] The third bounding box is any one of the two target bounding boxes, the fourth bounding box is the other target bounding box except the third bounding box, and the fourth bounding box is the target bounding box that needs to be cropped.

[0213] In this embodiment, since the object bounding boxes are all cubes, only one side wall of the object bounding box needs to be moved inward in the cropping process, that is, the object bounding box can be cropped. By finding the target side wall in the fourth bounding box that needs to be cropped, and moving the target side wall to the inside of the fourth bounding box until the fourth bounding box is separated from the third bounding box, the effect of cropping the fourth bounding box is achieved.

[0214] In the embodiments of the present application, the target side wall of the fourth bounding box that needs to be cropped is selected, and the target side wall is moved to the inside of the fourth bounding box to achieve the cropping of the fourth bounding box, so that the cropping process is more convenient.

[0215] In any of the above embodiments, the acquisition module is configured to acquire an overlapping projection between the third bounding box and the fourth bounding box, and the overlapping projection is a projection of an overlapping part of the third bounding box and the fourth bounding box on the ground.

[0216] The determination module 904 is configured to determine a distance value between a vertex of the overlapping projection and each side wall of the fourth bounding box.

[0217] The processing module 906 is configured to calculate a corresponding cropping volume based on the distance value and the area of the side wall.

[0218] The determination module 904 is configured to determine the side wall with the smallest cropping volume as the target side wall.

[0219] In this embodiment, the target side wall is selected by the distance between the vertex of the overlapping projection and each side wall of the fourth bounding box that needs to be cropped, so that the volume of the fourth bounding box can be maximized by moving the target side wall, and the volume that needs to be cropped is reduced, further ensuring the integrity of the fourth bounding box.

[0220] In the embodiments of the present application, the cropping volume of each side wall of the fourth bounding box is calculated, and the side wall with the smallest cropping volume is selected as the target side wall, so that the largest volume can be retained after the fourth bounding box is cropped by moving the target side wall, thereby improving the integrity of the fourth bounding box after cropping.

[0221] In any of the above embodiments, the acquisition module is configured to acquire a plurality of category information corresponding to a plurality of object bounding boxes.

[0222] The processing module 906 is configured to construct a preset overlapping array based on the plurality of category information and prior information.

[0223] The processing module 906 is configured to find the overlap rate threshold in the preset overlap array according to the object information of the two target bounding boxes.

[0224] In this embodiment, the preset overlap array is generated by the category information corresponding to the object bounding box and the corresponding prior information. After the two target bounding boxes are determined, the overlap rate threshold in the preset overlap array is screened to quickly and conveniently find the overlap rate threshold between the two target bounding boxes.

[0225] In the embodiment of the present application, the overlap rate threshold is pre-stored in the preset overlap array, which facilitates the calling in the de-overlapping process of the two target bounding boxes. Moreover, the overlap rate threshold is set based on the prior information, which can ensure that the de-overlapping process conforms to the actual situation of object placement and improve the accuracy of the semantic map after the de-overlapping process.

[0226] In any of the above embodiments, the number of rows of the preset overlap array is equal to the number of columns, and the number of rows of the preset overlap array is equal to the number of category information.

[0227] In this embodiment, by setting the number of category information to be equal to the number of rows and columns of the preset overlap array, it can be ensured that the corresponding overlap rate threshold is set between all pairs of object bounding boxes, thereby improving the completeness of the overlap rate threshold.

[0228] In an embodiment according to the present application, as shown in Figure 10 a semantic map de-overlapping device 1000 is provided, which comprises a processor 1002 and a memory 1004, the memory 1004 stores programs or instructions; the processor 1002 executes the programs or instructions stored in the memory 1004 to implement the steps of the semantic map de-overlapping method in any of the above embodiments, thus having all the beneficial technical effects of the semantic map de-overlapping method in any of the above embodiments, which will not be described in detail here.

[0229] In an embodiment according to the present application, a computer program product is provided, which, when executed by a processor, implements the steps of the semantic map de-overlapping method in any of the above embodiments, thus having all the beneficial technical effects of the semantic map de-overlapping method in any of the above embodiments, which will not be described in detail here.

[0230] In an embodiment according to the present application, a readable storage medium is provided, which stores programs or instructions, and the programs or instructions, when executed by a processor, implement the steps of the semantic map de-overlapping method in any of the above embodiments. Thus, all the beneficial technical effects of the semantic map de-overlapping method in any of the above embodiments are achieved, which will not be described in detail here.

[0231] In accordance with one embodiment of the present application, as shown in Figure 11 a service robot 1100 is provided, comprising the de-occlusion device 900 of the semantic map of any of the above embodiments, and / or the computer program product 1102 of any of the above embodiments, and / or the readable storage medium 1104 as defined in any of the above embodiments, thus having all the beneficial technical effects of the de-occlusion device 900 of the semantic map of any of the above embodiments, and / or the computer program product 1102 of any of the above embodiments, and / or the readable storage medium 1104 of any of the above embodiments, which will not be repeated here in more detail.

[0232] It should be noted that in the claims, the specification and the drawings of the present application, the terms "multiple" means two or more, unless otherwise specifically defined, the terms "upper", "lower" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and making the description process more simple, and is not intended to indicate or imply that the device or element must have the described specific orientation, be constructed and operated in a specific orientation, therefore these descriptions cannot be understood as a limitation on the present application; the terms "connection", "installation", "fixation" and the like should be understood in a broad sense, for example, "connection" can be a fixed connection between multiple objects, or a detachable connection between multiple objects, or an integral connection; can be a direct connection between multiple objects, or an indirect connection between multiple objects through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances of the above data.

[0233] In the claims, the specification and the drawings of the present application, the description of the terms "one embodiment", "some embodiments", "a specific embodiment" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in the embodiments or examples of the present application. In the claims, the specification and the drawings of the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0234] The above is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for de-duplication of semantic maps, characterized in that, The method comprises the following steps: In a plurality of object bounding boxes of the semantic map, two target bounding boxes are extracted, the object bounding boxes being bounding boxes of objects in the semantic map; A threshold overlap rate and an actual overlap rate of one of the two target bounding boxes relative to the other target bounding box are determined; According to the threshold overlap rate and the actual overlap rate, a de-overlap processing is performed on the two target bounding boxes; The determination of the threshold overlap rate of one of the two target bounding boxes relative to the other target bounding box comprises the following steps: Obtaining a plurality of category information corresponding to the plurality of object bounding boxes; Based on the plurality of category information and prior information, a preset overlap array is constructed; According to the object information of the two target bounding boxes, the threshold overlap rate is found in the preset overlap array.

2. The method of de-staggering a semantic map of claim 1, wherein, The determination of the threshold overlap rate and the actual overlap rate of one of the two target bounding boxes relative to the other target bounding box comprises the following steps: Obtaining a first projection of a first bounding box on the ground and a second projection of a second bounding box on the ground in the two target bounding boxes; Based on the first projection and the second projection, the actual overlap rate is determined.

3. The method of de-warping a semantic map according to claim 2, wherein, The determination of the actual overlap rate based on the first projection and the second projection comprises the following steps: Determining the intersection area of the first projection and the second projection; The ratio of the intersection area to the projection area of the second projection is determined as the actual overlap rate of the first bounding box relative to the second bounding box; The ratio of the intersection area to the projection area of the first projection is determined as the actual overlap rate of the second bounding box relative to the first bounding box.

4. The method of de-staggering a semantic map according to any one of claims 1 to 3, wherein, After the de-overlap processing of the two target bounding boxes according to the threshold overlap rate and the actual overlap rate, the method further comprises the following steps: Return to the step of extracting two target bounding boxes from a plurality of object bounding boxes of the semantic map until the actual overlap rate between any two object bounding boxes in the plurality of processed object bounding boxes meets a preset condition.

5. The method of de-staggering a semantic map according to any one of claims 1 to 3, wherein, The extraction of two target bounding boxes from a plurality of object bounding boxes of the semantic map comprises the following steps: Obtaining the confidence of each object bounding box in the plurality of object bounding boxes; According to the confidence, the two target bounding boxes in the plurality of object bounding boxes are determined.

6. The method of de-staggering a semantic map according to any one of claims 1 to 3, wherein, The two target bounding boxes comprise a third bounding box and a fourth bounding box, the actual overlap rate is the actual overlap rate of the fourth bounding box relative to the third bounding box, and the threshold overlap rate is the threshold overlap rate of the fourth bounding box relative to the third bounding box; The de-overlap processing of the two target bounding boxes according to the threshold overlap rate and the actual overlap rate comprises the following steps: In the case that the actual overlap rate is greater than the threshold overlap rate, the fourth bounding box is deleted; In the case that the actual overlap rate is less than or equal to the threshold overlap rate, the fourth bounding box is cropped based on the third bounding box.

7. The method of de-staggering a semantic map according to claim 6, wherein, The overlap rate threshold ranges from 0 to 100%, wherein the fourth bounding box cannot overlap the third bounding box when the overlap rate threshold is 0, and the fourth bounding box can completely overlap the third bounding box when the overlap rate threshold is 100%.

8. The method of de-warping a semantic map according to claim 6, wherein, The clipping of the fourth bounding box based on the third bounding box comprises: determining a target side wall of the fourth bounding box according to a positional relationship between the third bounding box and the fourth bounding box in the two target bounding boxes; translating the target side wall in a first direction until the fourth bounding box is separated from the third bounding box, the first direction being a direction from the target side wall to the inside of the fourth bounding box.

9. The method of de-warping a semantic map according to claim 8, wherein, The determining of the target side wall of the fourth bounding box according to the positional relationship between the third bounding box and the fourth bounding box in the two target bounding boxes comprises: obtaining an overlap projection between the third bounding box and the fourth bounding box, the overlap projection being a projection of an overlapping part of the third bounding box and the fourth bounding box on the ground; determining a distance value between a vertex of the overlap projection and each side wall of the fourth bounding box; calculating a clipping volume based on the distance value and the area of the side wall; determining the side wall with the smallest clipping volume as the target side wall.

10. The method of deswamping a semantic map of claim 1, wherein, The number of rows of the preset overlap array is equal to the number of columns, and the number of rows of the preset overlap array is equal to the number of category information.

11. A de-staggering device for a semantic map, characterized in that comprises: a memory having a program or instructions stored thereon; a processor configured to execute the program or instructions to implement the steps of the method for de-overlapping of the semantic map according to any one of claims 1 to 10.

12. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or instructions are executed by the processor to implement the steps of the method for de-overlapping of the semantic map according to any one of claims 1 to 10.

13. A service robot, characterized in that comprises: the de-overlapping device of the semantic map according to claim 11; and / or the readable storage medium according to claim 12.

Images