Semantic map correction method and device, service robot and readable storage medium

Abstract

The application provides a semantic map correction method and device, a service robot and a readable storage medium. The semantic map correction method comprises the following steps: identifying a target object in a semantic map; determining a target moving mode corresponding to the target object; and moving the target object relative to a reference line segment according to the target moving mode, so as to correct the semantic map, wherein the reference line segment is matched with the target object.

Description

Technical Field

[0001] This application relates to the field of semantic map technology, and more specifically, to a semantic map correction method, apparatus, service robot, and readable storage medium. Background Technology

[0002] Service robots can perceive and model the home environment, identifying and measuring objects such as home appliances and furniture, and combining this with navigation maps to form semantic maps with semantic information value.

[0003] All applications of service robots are based on semantic maps, and the accuracy of the semantic map affects the effectiveness of the server robot in performing subsequent tasks. Because object detection algorithms typically have some error, the positional relationship between the 3D model of an object and the wall may be abnormal, leading to lower accuracy of the semantic map and affecting the accuracy of the service robot's navigation using the semantic map. Summary of the Invention

[0004] This application aims to solve one of the technical problems existing in the prior art or related technologies.

[0005] Therefore, the first aspect of this application proposes a method for revising semantic maps.

[0006] The second aspect of this application proposes a semantic map correction device.

[0007] A third aspect of this application proposes a semantic map correction device.

[0008] The fourth aspect of this application proposes a computer program product.

[0009] The fifth aspect of this application proposes a readable storage medium.

[0010] The sixth aspect of this application proposes a service robot.

[0011] In view of this, according to the first aspect of this application, a method for correcting a semantic map is proposed, comprising: identifying a target object in the semantic map; determining the target movement mode corresponding to the target object; moving the target object relative to a reference line segment according to the target movement mode, wherein the reference line segment matches the target object, so as to correct the semantic map.

[0012] The technical solution of this application proposes a semantic map correction method to adjust the positional relationship between the target object and the reference line segment in the semantic map, so as to ensure that the positional relationship between the target object and the reference line segment conforms to the actual situation, thereby improving the accuracy of the semantic map and further ensuring the accuracy of robot navigation through the semantic map.

[0013] The target object can be an object model in the semantic map, or the bounding box corresponding to the object model in the semantic map. The reference line segment can be a line segment in a reference object in the semantic map, which is a line segment in another model or other bounding box in the semantic map that is not the target object.

[0014] In this technical solution, the semantic map is constructed by the robot through the collection of image data of the indoor scene. The semantic map includes multiple target objects, each corresponding to an object in the indoor scene. After extracting the target object, the target movement mode matched to that target object can be determined, with different categories of target objects corresponding to different target movement modes.

[0015] It should be noted that the position of the object relative to the wall may be different for different target objects. Therefore, there is a corresponding target movement method for each target object to ensure that after the target object is moved by the target movement method, the positional relationship between the target object and the reference line segment in the semantic map conforms to the relationship between the object and the wall in the real scene.

[0016] In this technical solution, after determining the target's movement method, reference line segments are used as reference points for the moving target object. In the semantic map, each reference line segment corresponds to a wall in the indoor scene; therefore, the semantic map includes multiple reference line segments. By selecting the target object, a reference line segment that matches the target object can be chosen. The reference line segment is one of the multiple reference line segments in the semantic map.

[0017] In the technical solution of this application, the positional relationship between the target object and the reference line segment in the semantic map is adjusted by moving the target object extracted from the semantic map along the reference line segment according to the corresponding target movement method. This realizes the correction of the positional relationship between the target object and the reference line segment in the semantic map, improves the accuracy of the semantic map, and ensures the accuracy of the service robot's navigation through the semantic map.

[0018] In the above technical solution, determining the target movement mode corresponding to the target object includes: determining the category information corresponding to the target object; and filtering at least one target movement mode from a preset movement mode set based on the category information.

[0019] This technical solution provides a process for determining the movement mode of a target based on the category information of the target object.

[0020] During the movement of a target object, it can be moved using a single target movement method or multiple target movement methods. When there are multiple target movement methods, the target positions of these methods do not conflict.

[0021] Specifically, the target movement method is matched with the actual positional relationship between the target object and the wall. Prior information is used to determine the actual positional relationship between different categories of objects and the wall, and corresponding preset movement methods are generated based on this relationship. After extracting the target object from the semantic map, the corresponding target movement method is determined based on the category information corresponding to the target object.

[0022] In the technical solution of this application, the target movement method that matches the category information is found among multiple preset movement methods by using the category information of the target object, so as to ensure the accuracy of the positional relationship between the reference line segment and the target object in the semantic map after the target object is moved by the target movement method.

[0023] In any of the above technical solutions, the reference line segment includes a first reference line segment. Based on category information, filtering at least one target movement mode in a preset movement mode set includes: determining the first preset movement mode in the preset movement mode set as the target movement mode according to the category information.

[0024] In this technical solution, the first preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a first positioning category, which includes: the orientation of the target object is perpendicular to the first reference line segment, and the distance between the target object and the first reference line segment is less than a first distance; and the first projection point of the first feature point of the target object in the direction perpendicular to the first reference line segment is located in a first plane, where the first plane is a plane in the first reference bounding box corresponding to the first reference line segment.

[0025] The first preset movement method includes: identifying a first feature point in the target object, the first feature point being located on the periphery of the target object; moving the target object relative to a first reference line segment until the first distance between the first feature point and the first reference line segment is less than a first threshold.

[0026] The technical solution proposes a first preset movement method, which moves the target object to a position that aligns with the first reference line segment.

[0027] Specifically, the bounding box of the target object is cubic in shape, comprising a bottom face, a top face, and side walls. The bottom face is the surface of the bounding box that meets the ground, the top face is opposite the bottom face, and the side walls are the surfaces located between the top and bottom faces. The first feature point is located on the side wall of the bounding box corresponding to the target object. During the semantic map construction process, a corresponding bounding box needs to be created for each target object, and the shape of the bounding box is either cuboid or cube.

[0028] In this technical solution, the first preset movement method involves moving the first feature point of the target object to a distance less than a first threshold from the first reference line segment, ensuring that the target object fits snugly against the first reference line segment in the semantic map. The first distance is the distance of the target object along the vertical direction of the first reference line segment.

[0029] It should be noted that the first preset movement method is applicable to target objects whose category information is bed, cabinet, wardrobe, etc.

[0030] In the technical solution of this application, the target object is moved toward the direction of the first reference line segment until the first distance between the two is less than the first threshold, so as to ensure that the target object is in contact with the wall.

[0031] In any of the above technical solutions, the reference line segment includes a second reference line segment; filtering at least one target movement mode in the preset movement mode set based on category information includes: determining the second preset movement mode in the preset movement mode set as the target movement mode according to the category information;

[0032] In this technical solution, the second preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a second positioning category, which includes: the orientation of the target object is perpendicular to the second reference line segment, and the distance between the target object and the second reference line segment is less than a second distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, which is a plane in the second reference bounding box corresponding to the second reference line segment.

[0033] The second preset movement method includes: identifying a second feature point in the target object, the second feature point being located inside the target object; moving the target object relative to the second reference line segment until the second feature point coincides with the second reference line segment.

[0034] The technical solution proposes a second preset movement method, which moves the target object to a position that coincides with the second reference line segment, and the target object is attached to the middle of the second reference line segment, that is, by embedding the target object into the middle of the second reference line segment.

[0035] In this technical solution, the second preset movement method is to move the second feature point of the target object to a position that coincides with the second reference line segment, so as to ensure that the target object coincides with the second reference line segment in the semantic map.

[0036] It should be noted that the second preset movement method is applicable to target objects whose category information is windows, doors, etc.

[0037] In the technical solution of this application, by determining the second feature point inside the target object and moving the target object through the second preset movement method, the second feature point in the target object coincides with the second reference line segment, thereby realizing the movement of the target object that may be embedded in the wall into the wall, and further improving the accuracy of the semantic map.

[0038] In any of the above technical solutions, the reference line segment includes a third reference line segment and a fourth reference line segment; based on category information, filtering at least one target movement mode in the preset movement mode set includes: determining the third preset movement mode in the preset movement mode set as the target movement mode according to the category information;

[0039] In this technical solution, the third preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a third positioning category, which includes: the orientation of the target object is perpendicular to the third reference line segment, and the distance between the target object and the third reference line segment is less than a third distance; the distance between the third feature point in the target object and the target endpoint of the third reference line segment is less than a fourth distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, which is the plane in the second reference bounding box corresponding to the second reference line segment.

[0040] The third preset movement method includes: identifying a third reference line segment and a fourth reference line segment in the semantic map, wherein the third reference line segment is perpendicular to and intersects the fourth reference line segment; moving the target object along the parallel direction of the third reference line segment until the second distance between the target object and the fourth reference line segment is less than a second threshold.

[0041] The technical solution proposes a third preset movement method, which moves the bounding box of the target object to a position aligned with the third reference line segment and the fourth reference line segment. That is, the two side walls of the bounding box of the target object are aligned with the third reference line segment and the fourth reference line segment, respectively.

[0042] The second distance is the distance between the target object and the third reference line segment in the direction perpendicular to the fourth reference line segment. After the target object is moved by the third preset movement method, the second distance is less than the second threshold, so that the target object is aligned with the third reference line segment in the left and right directions.

[0043] In this technical solution, the third preset movement method is to move the target object to a position aligned with the adjacent third reference line segment and fourth reference line segment.

[0044] It should be noted that the third preset movement method is applicable to target objects whose category information is doors, wardrobes, etc.

[0045] In the technical solution of this application, by determining the fourth reference line segment adjacent to the third reference line segment and moving the target object through the third preset movement method, the moved target object is aligned with the reference line segment on the left and right. This realizes the alignment setting of the target object that may be aligned with two adjacent walls with the third and fourth reference line segments, further improving the accuracy of the semantic map.

[0046] In any of the above technical solutions, the reference line segment includes a fifth reference line segment; based on category information, filtering at least one target movement mode in the preset movement mode set includes: determining the fourth preset movement mode in the preset movement mode set as the target movement mode according to the category information;

[0047] In this technical solution, the fourth preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a fourth positioning category, which includes: the orientation of the target object is perpendicular to the fourth reference line segment, and the distance between the target object and the fourth reference line segment is less than a fifth distance; the distance between the fourth feature point in the target object and the midpoint of the third reference line segment is less than a sixth distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, which is the plane in the second reference bounding box corresponding to the second reference line segment.

[0048] The fourth preset movement method includes: moving the target object along the direction parallel to the fifth reference line segment until the third distance between the center point of the target plane of the target object and the center point of the fifth reference line segment is less than the third threshold, and the target plane is the plane of the target object close to the fifth reference line segment.

[0049] The technical solution proposes a fourth preset movement method, which moves the target object to a position aligned with the center of the fifth reference line segment, that is, the position where the center point of the target plane in the target bounding box of the target object corresponds to the center point of the fifth reference line segment.

[0050] Here, the target plane is the plane within the target object that is closest to the fifth reference line segment. This plane is the sidewall of the target object closest to the fifth reference line segment. The third distance is the distance between the center point of the target plane and the center point of the fifth reference line segment. When the third distance is less than a third threshold, it can be determined that the target object is centered and aligned with the fifth reference line segment.

[0051] In this technical solution, the fourth preset movement method is to move the target object to the middle position of the fifth reference line segment so that the two are centered and aligned.

[0052] It should be noted that the fourth preset movement method is applicable to target objects whose category information is windows, televisions, etc.

[0053] In the technical solution of this application, by determining the center point on the target plane of the target object and the center point on the fifth reference line segment, as well as the third distance between the two center points, the target object is moved through the fourth preset method, so that the moved target object is centered and aligned with the fifth reference line segment. This achieves the center alignment of the target object that may be placed in the middle of the wall with the fifth reference line segment, further improving the accuracy of the semantic map.

[0054] In any of the above technical solutions, moving the target object relative to the reference line segment according to the target movement method includes: if at least two target movement methods are obtained through screening, determining the execution order of the at least two target movement methods based on category information; and moving the target object sequentially according to the execution order and the at least two target movement methods.

[0055] This technical solution provides a process for moving a target object according to multiple target movement methods.

[0056] Specifically, different target movement methods change the positional relationship between the target object and the reference line segment. Moving the target object relative to the reference line segment using multiple target movement methods can further improve the accuracy of the positional relationship between the target object and the reference line segment. Furthermore, based on the category information corresponding to the target object, the execution order of multiple target movement methods can be determined, and the multiple target movement methods can be executed sequentially according to this execution order.

[0057] In the technical solution of this application, when the extracted target object corresponds to multiple target movement methods, the target object is moved relative to the reference line segment in sequence according to the multiple target movement methods, thereby ensuring that the moved target object matches the actual placement of the object and improving the accuracy of the corrected semantic map.

[0058] In any of the above technical solutions, the category information includes the object type; when at least two target movement methods are obtained through filtering, the execution order of the at least two target movement methods is determined based on the category information, including: determining the execution order according to the target mapping relationship and the object type, wherein the target mapping relationship is the mapping relationship between the object type and the execution order.

[0059] In this technical solution, the execution order of multiple target movement methods is found by identifying the object type corresponding to the target object.

[0060] Specifically, when multiple target movement methods are found based on category information, the object type in the category information is obtained, and the execution order can be found based on the mapping relationship between object type and target.

[0061] In the technical solution of this application, the execution order corresponding to different target objects can be accurately determined by the object type in the category information. When it is necessary to move the target object using multiple target movement methods, the movement can be carried out in the straight-line order, which can improve the accuracy of the movement.

[0062] In any of the above technical solutions, identifying target objects in a semantic map includes: acquiring image data; and constructing target objects in a semantic map based on the image data.

[0063] This technical solution provides a method for adjusting the positional relationship between the target object and the reference line segment during the construction of a semantic map.

[0064] Specifically, during the construction of the semantic map, multiple bounding boxes corresponding to target objects are generated in the semantic map based on the collected image data. Each target object corresponds to a real-world object, and the shape of the bounding box corresponding to the target object is a cube. Furthermore, the category information corresponding to the constructed target object can be determined, thereby determining the target object's movement method, and moving the target object relative to the reference line segment according to the target movement method.

[0065] It should be noted that before adjusting the position of the target object relative to the reference line segments during the construction of the semantic map, multiple reference line segments in the semantic map need to be constructed first. After ensuring the accuracy of the multiple reference line segments, the target object is then moved based on the reference line segments.

[0066] In the technical solution of this application, the accuracy of the completed semantic map is ensured by adjusting and correcting the generated target objects in real time during the construction of the semantic map, without the need for subsequent adjustments to the semantic map.

[0067] In any of the above technical solutions, identifying target objects in a semantic map includes: acquiring a semantic map; and extracting target objects from the semantic map.

[0068] This technical solution provides a method for adjusting the positional relationship between the target object and the reference line segment after the target object has been acquired and constructed; that is, in the process of correcting the constructed target object. Specifically, after acquiring the constructed semantic map, the target object is extracted from the semantic map.

[0069] In the technical solution of this application, the construction of the semantic map is achieved by extracting the target objects in the completed semantic map and moving the target objects relative to the reference line segments.

[0070] In any of the above technical solutions, after identifying the target object in the semantic map, the method further includes: obtaining a set of reference line segments in the semantic map; and filtering reference line segments in the set of reference line segments based on the target object, wherein the reference line segments are the reference line segments in the set of reference line segments that are closest to the target object.

[0071] This technical solution provides a detailed process for filtering reference line segments in a semantic map by selecting target objects.

[0072] Specifically, after extracting the target object from the semantic map, multiple reference line segments, i.e., a set of reference line segments, are also extracted from the semantic map. The reference line segment closest to the target object is selected as the final reference line segment. When moving the target object relative to a reference line segment, the target object is most likely to have a positional relationship with the closest reference line segment. Therefore, selecting the closest reference line segment among the multiple reference line segments improves the accuracy of moving the target object relative to the reference line segment.

[0073] In the technical solution of this application, the reference line segment closest to the target object in the set of reference line segments is used as the reference line segment, and the target object is moved based on the reference line segment, thus ensuring the accuracy of the movement of the target object.

[0074] In any of the above technical solutions, the semantic map correction method further includes: determining the target bounding box corresponding to the target object in the semantic map; moving the target object based on the feature points in the target bounding box until the feature points and the reference line segment satisfy a preset relationship;

[0075] The feature points include at least one of a first feature point and a second feature point, wherein the first feature point is located on the peripheral sidewall of the target bounding box and the second feature point is located inside the target bounding box.

[0076] The reference line segment includes at least one of the first reference line segment, the second reference line segment, the third reference line segment, the fourth reference line segment, and the fifth reference line segment. The third reference line segment and the fourth reference line segment are obtained based on semantic map recognition. The third reference line segment and the fourth reference line segment are perpendicular and intersect.

[0077] In this technical solution, by identifying the target bounding box corresponding to the target object in the semantic map, and based on the feature points and reference line segments in the target bounding box, the target object is moved in the semantic map so that the positional relationship between the target object and the reference line segments meets the preset relationship, thereby adjusting the positional relationship between the target object and the reference line segments in the semantic map and realizing the correction of the positional relationship between the target object and the reference line segments in the semantic map.

[0078] In the technical solution of this application, the target bounding box is an object bounding box in the semantic map that matches the target object. The constructed semantic bounding box includes multiple object bounding boxes, and the target bounding box that matches the target object is searched among these multiple object bounding boxes.

[0079] Wherein, the first feature point and the second feature point are the same feature points as those in any of the above technical solutions. When moving the target object based on feature points, it is necessary to find the corresponding first feature point and second feature point based on the target bounding box. The first feature point can be a head feature point or a tail feature point of the target bounding box. The second feature point can also be a head feature point or a tail feature point of the target bounding box.

[0080] The first, second, third, and fourth reference line segments are the same as those in any of the aforementioned technical solutions. These segments are line segments within the bounding boxes of the objects corresponding to the reference objects matched with the target object in the semantic map. The third and fourth reference line segments are perpendicular and intersecting. By identifying the line segments within the bounding boxes of the reference objects in the semantic map, two perpendicular and intersecting line segments are selected and used as the third and fourth reference line segments. These two line segments can be within the same bounding box or different bounding boxes.

[0081] In the technical solution of this application, a target object is extracted from a semantic map, and a corresponding target bounding box is found based on the target object. The position of the target object is adjusted based on at least one of a first feature point and a second feature point within the target bounding box, and at least one of a corresponding first reference line segment, a second reference line segment, a third reference line segment, and a fourth reference line segment. This adjusts the positional relationship between the target object and the reference line segments in the semantic map, improving the accuracy of the semantic map and ensuring the precision of the service robot's navigation using the semantic map.

[0082] According to a second aspect of this application, a semantic map correction device is proposed, comprising: an identification module for identifying target objects in the semantic map; a determination module for determining the target movement mode corresponding to the target object; and a movement module for moving the target object relative to a reference line segment according to the target movement mode, wherein the reference line segment matches the target object, so as to correct the semantic map.

[0083] The technical solution of this application proposes a semantic map correction device, which is used to adjust the positional relationship between the target object and the reference line segment in the semantic map, so as to ensure that the positional relationship between the target object and the reference line segment conforms to the actual situation, thereby improving the accuracy of the semantic map and further ensuring the accuracy of robot navigation through the semantic map.

[0084] In this technical solution, the semantic map is constructed by the robot through the collection of image data of the indoor scene. The semantic map includes multiple target objects, each corresponding to an object in the indoor scene. After extracting the target object, the target movement mode matched to that target object can be determined, with different categories of target objects corresponding to different target movement modes.

[0085] It should be noted that the position of the object relative to the wall may be different for different target objects. Therefore, there is a corresponding target movement method for each target object to ensure that after the target object is moved by the target movement method, the positional relationship between the target object and the reference line segment in the semantic map conforms to the relationship between the object and the wall in the real scene.

[0086] In this technical solution, after determining the target's movement method, reference line segments are used as reference points for the moving target object. In the semantic map, each reference line segment corresponds to a wall in the indoor scene; therefore, the semantic map includes multiple reference line segments. By selecting the target object, a reference line segment that matches the target object can be chosen. The reference line segment is one of the multiple reference line segments in the semantic map.

[0087] In the technical solution of this application, the positional relationship between the target object and the reference line segment in the semantic map is adjusted by moving the target object extracted from the semantic map along the reference line segment according to the corresponding target movement method. This realizes the correction of the positional relationship between the target object and the reference line segment in the semantic map, improves the accuracy of the semantic map, and ensures the accuracy of the service robot's navigation through the semantic map.

[0088] According to a third aspect of this application, a semantic map correction apparatus is proposed, comprising: a memory storing a program or instructions; and a processor executing the program or instructions stored in the memory to implement the steps of the semantic map correction method as described in any of the technical solutions in the first aspect, thus possessing all the beneficial technical effects of the semantic map correction method in any of the technical solutions in the first aspect, which will not be elaborated further here.

[0089] According to the fourth aspect of this application, a computer program product is proposed. When the computer program product is executed by a processor, it implements the steps of the semantic map correction method as described in any of the technical solutions in the first aspect, and thus has all the beneficial technical effects of the semantic map correction method in any of the technical solutions in the first aspect, which will not be elaborated further here.

[0090] According to the fifth aspect of this application, a readable storage medium is provided, on which a program or instructions are stored. When the program or instructions are executed by a processor, they implement the steps of the semantic map correction method as described in any of the technical solutions in the first aspect above. Therefore, it possesses all the beneficial technical effects of the semantic map correction method in any of the technical solutions in the first aspect above, which will not be elaborated further here.

[0091] According to the sixth aspect of this application, a service robot is proposed, comprising: a semantic map correction device as defined in the second or third aspect above, and / or a computer program product as defined in the fourth aspect above, and / or a readable storage medium as defined in the fifth aspect above, thus having all the beneficial technical effects of the semantic map correction device as defined in the second or third aspect above, and / or the computer program product as defined in the fourth aspect above, and / or the readable storage medium as defined in the fifth aspect above, which will not be elaborated further here.

[0092] Additional aspects and advantages of this application will become apparent in the following description or may be learned by practice of this application. Attached Figure Description

[0093] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0094] Figure 1 One of the schematic flowcharts of the semantic map correction method provided in some embodiments of this application is shown;

[0095] Figure 2 The second schematic flowchart illustrates a semantic map correction method provided in some embodiments of this application;

[0096] Figure 3 The third schematic flowchart illustrates a semantic map correction method provided in some embodiments of this application;

[0097] Figure 4 A schematic diagram of a first preset movement mode provided in some embodiments of this application is shown;

[0098] Figure 5 The fourth schematic flowchart illustrates a semantic map correction method provided in some embodiments of this application;

[0099] Figure 6 A schematic diagram of a second preset movement mode provided in some embodiments of this application is shown;

[0100] Figure 7 Fifth of the schematic flowcharts illustrating the semantic map correction method provided in some embodiments of this application is shown;

[0101] Figure 8 A schematic diagram of a third preset movement method provided in some embodiments of this application is shown;

[0102] Figure 9 A schematic diagram of a fourth preset movement method provided in some embodiments of this application is shown;

[0103] Figure 10 A schematic flowchart of a semantic map correction method provided in some embodiments of this application is shown as Flowchart 6;

[0104] Figure 11 This is illustrated in the seventh schematic flowchart of a semantic map correction method provided in some embodiments of this application;

[0105] Figure 12 This is shown as a schematic flowchart of a semantic map correction method provided in some embodiments of this application (number eight).

[0106] Figure 13 The diagram shows a structural block diagram of a semantic map correction apparatus provided in some embodiments of this application;

[0107] Figure 14 The diagram shows a structural block diagram of a semantic map correction apparatus provided in some embodiments of this application;

[0108] Figure 15 The following are structural block diagrams of the service robot provided in some embodiments of this application;

[0109] Figure 16 A schematic diagram of the target object provided in some embodiments of this application is shown. Detailed Implementation

[0110] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, these embodiments and the features described herein can be combined with each other.

[0111] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0112] The following reference Figures 1 to 16 This application describes methods, apparatus, computer program products, readable storage media, and service robots for modifying semantic maps according to some embodiments.

[0113] According to one embodiment of this application, such as Figure 1 As shown, a semantic map correction method is proposed, including:

[0114] Step 102: Identify the target objects in the semantic map;

[0115] Step 104: Determine the target movement method corresponding to the target object;

[0116] Step 106: Move the target object relative to the reference line segment according to the target movement method. The reference line segment matches the target object to correct the semantic map.

[0117] The embodiments of this application propose a semantic map correction method to adjust the positional relationship between the target object and the reference line segment in the semantic map, ensuring that the positional relationship between the target object and the reference line segment conforms to the actual situation, thereby improving the accuracy of the semantic map and further ensuring the accuracy of robot navigation through the semantic map.

[0118] The target object can be an object model in the semantic map, or the bounding box corresponding to the object model in the semantic map. The reference line segment can be a line segment in a reference object in the semantic map, which is a line segment in another model or other bounding box in the semantic map that is not the target object.

[0119] In this embodiment, the semantic map is constructed by the robot using image data of the indoor scene. The semantic map includes multiple target objects, each corresponding to an object in the indoor scene. After extracting the target object, the target movement mode matched to that target object can be determined, with different categories of target objects corresponding to different target movement modes.

[0120] In this embodiment, the reference line segment is a line segment within the bounding box of the reference object.

[0121] For example, the reference object can be a wall, that is, the reference line segment is a wall line segment.

[0122] For example, the reference object can be a refrigerator, that is, the reference line segment is the line segment in the refrigerator enclosure.

[0123] It should be noted that the positions of different target objects relative to reference objects may vary. Therefore, each target object has a corresponding target movement method to ensure that after the target object is moved using the target movement method, the positional relationship between the target object and the reference line segment in the semantic map conforms to the relationship between objects and reference objects in the real scene.

[0124] For example, the target object is a window model, and the reference line segment is a wall line segment. The window model can be embedded in the middle of the reference line segment. The target movement method includes: moving the target object to the middle of the reference line segment, and embedding the target object within the reference line segment, thereby ensuring that the positional relationship between the target object and the reference line segment conforms to the actual placement position of the object.

[0125] For example, the target object is a wardrobe model, the reference line segments are wall line segments, and the wardrobe enclosure can be aligned with two adjacent reference line segments. The target movement method includes moving the target object to a position that is close to the two adjacent reference line segments, thereby ensuring that the positional relationship between the target object and the reference line segments matches the actual placement of the object.

[0126] In this embodiment, after determining the target's movement method, a reference line segment is used as a reference for the moving target object. In the semantic map, each reference line segment corresponds to a wall in the indoor scene; therefore, the semantic map includes multiple reference line segments. The target object allows selection of a reference line segment that matches it. The reference line segment is one of the multiple reference line segments in the semantic map.

[0127] For example, the reference line segment is the reference line segment closest to the target object among multiple reference line segments in the semantic map. When moving the target object relative to the reference line segment, the target object is most likely to have a positional relationship with the closest reference line segment. Therefore, using the closest reference line segment among multiple reference line segments as the reference line segment improves the accuracy of moving the target object relative to the reference line segment.

[0128] In the embodiments of this application, by moving the target object extracted from the semantic map along the reference line segment according to the corresponding target movement method, the positional relationship between the target object and the reference line segment in the semantic map is adjusted, thereby correcting the positional relationship between the target object and the reference line segment in the semantic map, improving the accuracy of the semantic map, and ensuring the accuracy of the service robot's navigation through the semantic map.

[0129] like Figure 2 As shown, in the above embodiments, determining the target movement mode corresponding to the target object includes:

[0130] Step 202: Determine the category information corresponding to the target object;

[0131] Step 204: Based on category information, filter at least one target movement method from the preset movement method set.

[0132] This embodiment provides a process for determining the movement mode of a target based on the category information of the target object.

[0133] During the movement of a target object, it can be moved using a single target movement method or multiple target movement methods. When there are multiple target movement methods, the target positions of these methods do not conflict.

[0134] For example, multiple target movement methods include: centering the target object with a reference line segment, and aligning the target object with the reference line segment.

[0135] Specifically, the target movement method is matched with the actual positional relationship between the target object and the wall. Prior information is used to determine the actual positional relationship between different categories of objects and the wall, and corresponding preset movement methods are generated based on this relationship. After extracting the target object from the semantic map, the corresponding target movement method is determined based on the category information corresponding to the target object.

[0136] In the embodiments of this application, the target movement method that matches the category information is found among multiple preset movement methods by using the category information of the target object, so as to ensure the accuracy of the positional relationship between the reference line segment and the target object in the semantic map after the target object is moved by the target movement method.

[0137] like Figure 3 As shown, in any of the above embodiments, the reference line segment includes a first reference line segment, and based on category information, at least one target movement mode is selected from a preset movement mode set, including:

[0138] Based on the category information, the first preset movement method in the preset movement method set is determined as the target movement method;

[0139] In this embodiment, the first preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a first positioning category, which includes: the orientation of the target object is perpendicular to the first reference line segment, and the distance between the target object and the first reference line segment is less than a first distance; and the first projection point of the first feature point of the target object in the direction perpendicular to the first reference line segment is located in a first plane, where the first plane is a plane in the first reference bounding box corresponding to the first reference line segment.

[0140] For example, the first feature point can be the head feature point of the bounding box of the target object. That is, the first feature point is the midpoint of the top line segment of the back face of the target object, where the back face is the first preset plane in the process of constructing the target object.

[0141] For example, the first feature point can be the tail feature point of the bounding box of the target object. That is, the first feature point is the midpoint of the top line segment of the front of the target object, where the front is the second preset plane during the construction of the target object, and the second preset plane is set opposite to the first preset plane.

[0142] The first preset movement method includes:

[0143] Step 302: Identify the first feature point in the target object, the first feature point being located on the peripheral sidewall of the target object;

[0144] Step 304: Move the target object relative to the first reference line segment until the first distance between the first feature point and the first reference line segment is less than the first threshold.

[0145] In this embodiment, a first preset movement method is proposed, which moves the target object to a position that is in contact with the first reference line segment.

[0146] Specifically, the bounding box corresponding to the target object is cubic in shape, and includes a bottom face, a top face, and side walls. The bottom face is the surface of the target object that is in contact with the ground, the top face is opposite to the bottom face, and the side walls are the surfaces located between the top and bottom faces. The first feature point is located on the side wall of the bounding box corresponding to the target object. During the construction of the semantic map, a corresponding bounding box needs to be created for each target object, and the shape of the bounding box is either cuboid or cube.

[0147] like Figure 16 As shown, the head of the target object 1602 is located near the first reference line segment 1604, and the first feature point 1606 is located at the head of the bounding box corresponding to the target object 1602. The direction of arrow A is from the head to the tail of the bounding box corresponding to the target object 1602.

[0148] In this embodiment, the first preset movement method involves moving the first feature point of the target object to a distance less than a first threshold from the first reference line segment, ensuring that the target object aligns with the first reference line segment in the semantic map. The first distance is the distance of the target object along the vertical direction of the first reference line segment.

[0149] It should be noted that the first preset movement method is applicable to target objects whose category information is bed, cabinet, wardrobe, etc.

[0150] like Figure 4As shown, before the movement, the first distance L1 between the target object 402 and the first reference line segment 404 is greater than the first threshold. After the movement in the first preset movement mode, the first distance L1 between the target object 402 and the first reference line segment 404 is less than the first threshold.

[0151] In the embodiments of this application, the target object is moved toward the direction of the first reference line segment until the first distance between the two is less than the first threshold, ensuring that the target object is in contact with the wall.

[0152] like Figure 5 As shown, in any of the above embodiments, the reference line segment includes a second reference line segment; filtering at least one target movement mode in the preset movement mode set based on category information includes: determining the second preset movement mode in the preset movement mode set as the target movement mode according to the category information;

[0153] In this embodiment, the second preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a second positioning category, which includes: the orientation of the target object is perpendicular to the second reference line segment, and the distance between the target object and the second reference line segment is less than a second distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, which is a plane in the second reference bounding box corresponding to the second reference line segment.

[0154] For example, the second feature point can be the head feature point or the tail feature point of the bounding box of the target object.

[0155] The second preset movement method includes:

[0156] Step 502: Identify a second feature point in the target object, the second feature point being located inside the target object;

[0157] Step 504: Move the target object relative to the second reference line segment until the second feature point coincides with the second reference line segment.

[0158] In this embodiment, a second preset movement method is proposed, which moves the target object to a position that coincides with the second reference line segment, and the target object is attracted to the middle of the second reference line segment, that is, by embedding the target object into the middle of the second reference line segment.

[0159] For example, the second feature point can be the middle feature point of the object bounding box corresponding to the target object, that is, the second feature point is located inside the target object and at the physical center of the target object.

[0160] In this embodiment, the second preset movement method is to move the second feature point of the target object to a position that coincides with the second reference line segment, so as to ensure that the target object coincides with the second reference line segment in the semantic map.

[0161] It should be noted that the second preset movement method is applicable to target objects whose category information is windows, doors, etc.

[0162] like Figure 6 As shown, before the movement, there is a distance between the target object 602 and the second reference line segment 604. After the movement in the second preset movement mode, the target object 602 is embedded in the middle of the second reference line segment 604. At this time, the second feature point 606 coincides with the second reference line segment 604.

[0163] In the embodiments of this application, by determining the second feature point inside the target object and moving the target object using the second preset movement method, the second feature point in the target object coincides with the second reference line segment, thereby moving the target object that may be embedded in the wall into the wall and further improving the accuracy of the semantic map.

[0164] like Figure 7 As shown, in any of the above embodiments, the reference line segment includes a third reference line segment and a fourth reference line segment; filtering at least one target movement mode in the preset movement mode set based on category information includes: determining the third preset movement mode in the preset movement mode set as the target movement mode according to the category information;

[0165] In this embodiment, the third preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a third positioning category, which includes: the orientation of the target object is perpendicular to the third reference line segment, and the distance between the target object and the third reference line segment is less than a third distance; the distance between the third feature point in the target object and the target endpoint of the third reference line segment is less than a fourth distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, where the second plane is the plane in the second reference bounding box corresponding to the second reference line segment.

[0166] For example, the third feature point is the center point of the target object.

[0167] The third preset movement method includes:

[0168] Step 702: Identify the third and fourth reference line segments in the semantic map. The third reference line segment is perpendicular to and intersects the fourth reference line segment.

[0169] Step 704: Move the target object along the direction parallel to the third reference line segment until the second distance between the target object and the fourth reference line segment is less than the second threshold.

[0170] In this embodiment, a third preset movement method is proposed, which moves the bounding box of the target object to a position aligned with the third reference line segment and the fourth reference line segment. That is, the two side walls of the bounding box of the target object are aligned with the third reference line segment and the fourth reference line segment, respectively.

[0171] The second distance is the distance between the target object and the third reference line segment in the direction perpendicular to the fourth reference line segment. After the target object is moved by the third preset movement method, the second distance is less than the second threshold, so that the target object is aligned with the third reference line segment in the left and right directions.

[0172] For example, the third reference line segment is the first wall line segment, and the fourth reference line segment is the second wall line segment. The second wall line segment is adjacent to the first wall line segment, and the second wall line segment and the first wall line segment are perpendicular to each other; that is, the first wall line segment and the second wall line segment correspond to two adjacent walls in the semantic map.

[0173] In this embodiment, the third preset movement method is to move the target object to a position aligned with the adjacent third reference line segment and fourth reference line segment.

[0174] It should be noted that the third preset movement method is applicable to target objects whose category information is doors, wardrobes, etc.

[0175] like Figure 8 As shown, before the movement, the target object 802 is in contact with the third reference line segment 804, and the second distance L2 between the target object 802 and the fourth reference line segment 806 is greater than the second threshold. After the movement in the third preset movement mode, the second distance L2 between the target object 802 and the fourth reference line segment 806 is less than the second threshold.

[0176] In the embodiments of this application, by determining the fourth reference line segment adjacent to the third reference line segment and moving the target object through the third preset movement method, the moved target object is aligned with the reference line segment on the left and right. This realizes the alignment setting of the target object that may be aligned with two adjacent walls with the third and fourth reference line segments, further improving the accuracy of the semantic map.

[0177] In any of the above embodiments, the reference line segment includes a fifth reference line segment; based on category information, filtering at least one target movement mode from a preset movement mode set includes:

[0178] Based on the category information, the fourth preset movement method in the preset movement method set is determined as the target movement method;

[0179] In this embodiment, the fourth preset movement mode can be used as the target movement mode by using the category information of the target object. The category information includes: a fourth positioning category, which includes: the orientation of the target object is perpendicular to the fourth reference line segment, and the distance between the target object and the fourth reference line segment is less than a fifth distance; the distance between the fourth feature point in the target object and the midpoint of the third reference line segment is less than a sixth distance; and the second projection point of the second feature point of the target object in the direction perpendicular to the second reference line segment is located in a second plane, where the second plane is the plane in the second reference bounding box corresponding to the second reference line segment.

[0180] For example, the fourth feature point is the center point of the target object.

[0181] The fourth preset movement method includes: moving the target object along the direction parallel to the fifth reference line segment until the third distance between the center point of the target plane of the target object and the center point of the fifth reference line segment is less than the third threshold, and the target plane is the plane of the target object close to the fifth reference line segment.

[0182] In this embodiment, a fourth preset movement method is proposed, which moves the target object to a position that is centered and aligned with the fifth reference line segment, that is, the position where the center point of the target plane in the target bounding box of the target object corresponds to the center point of the fifth reference line segment.

[0183] Here, the target plane is the plane within the target object that is closest to the fifth reference line segment. This plane is the sidewall of the target object closest to the fifth reference line segment. The third distance is the distance between the center point of the target plane and the center point of the fifth reference line segment. When the third distance is less than a third threshold, it can be determined that the target object is centered and aligned with the fifth reference line segment.

[0184] In this embodiment, the fourth preset movement method is to move the target object to the middle position of the fifth reference line segment so that the two are centered and aligned.

[0185] It should be noted that the fourth preset movement method is applicable to target objects whose category information is windows, televisions, etc.

[0186] like Figure 9As shown, before the movement, the third distance between the center point O1 of the target plane 904 of the bounding box corresponding to the target object 902 and the center point O2 of the fifth reference line segment 906 is L3. The third distance L3 is greater than the third threshold. After the movement in the fourth preset movement mode, the third distance L3 between the center point O1 of the target plane 904 and the center point O2 of the fifth reference line segment 906 is less than the third threshold, so that the target object 902 and the fifth reference line segment 906 are centered and aligned.

[0187] In the embodiments of this application, by determining the center point on the target plane of the target object and the center point on the fifth reference line segment, as well as the third distance between the two center points, the target object is moved in a fourth preset manner, so that the moved target object is centered and aligned with the fifth reference line segment. This achieves the center alignment of the target object that may be placed in the middle of the wall with the fifth reference line segment, further improving the accuracy of the semantic map.

[0188] In any of the above embodiments, moving the target object relative to the reference line segment according to the target movement method includes: if at least two target movement methods are obtained through filtering, determining the execution order of the at least two target movement methods based on category information; and moving the target object sequentially according to the execution order and the at least two target movement methods.

[0189] This embodiment provides a process for moving a target object according to multiple target movement methods.

[0190] Specifically, different target movement methods change the positional relationship between the target object and the reference line segment. Moving the target object relative to the reference line segment using multiple target movement methods can further improve the accuracy of the positional relationship between the target object and the reference line segment. Furthermore, based on the category information corresponding to the target object, the execution order of multiple target movement methods can be determined, and the multiple target movement methods can be executed sequentially according to this execution order.

[0191] For example, the target object is a window enclosure, typically located at the center of a wall, and in a central position within the wall. Therefore, multiple target movement methods can include a second preset movement method and a fourth preset movement method. The straight-line sequence is to execute the first preset movement method first, followed by the second preset movement method.

[0192] In the embodiments of this application, when the extracted target object corresponds to multiple target movement methods, the target object is moved relative to the reference line segment in sequence according to the multiple target movement methods, thereby ensuring that the moved target object matches the actual placement of the object and improving the accuracy of the corrected semantic map.

[0193] In any of the above embodiments, the category information includes the object type; when at least two target movement methods are obtained through filtering, the execution order of the at least two target movement methods is determined based on the category information, including: determining the execution order according to the target mapping relationship and the object type, wherein the target mapping relationship is the mapping relationship between the object type and the execution order.

[0194] In this embodiment, a scheme for the execution order corresponding to multiple target movement methods is found based on the object type corresponding to the target object.

[0195] Specifically, when multiple target movement methods are found based on category information, the object type in the category information is obtained, and the execution order can be found based on the mapping relationship between object type and target.

[0196] For example, the execution order includes the execution order of multiple preset movement methods, and the target movement method includes at least some of the multiple preset movement methods. For instance, when the object type is a table, the execution order corresponding to the table is the first preset movement method followed by the fourth preset movement method. When the target movement method includes both the first and fourth preset movement methods, the first preset movement method is executed first, followed by the fourth preset movement method. When the target movement method includes only the first or fourth preset movement method, only the first or fourth preset movement method needs to be executed.

[0197] In the embodiments of this application, the execution order corresponding to different target objects can be accurately determined by the object type in the category information. When it is necessary to move the target object using multiple target movement methods, the movement can be carried out in the straight-line order, which can improve the accuracy of the movement.

[0198] like Figure 10 As shown, in any of the above embodiments, identifying target objects in the semantic map includes:

[0199] Step 1002: Acquire image data;

[0200] Step 1004: Construct the target objects in the semantic map based on the image data.

[0201] This embodiment provides a scheme for adjusting the positional relationship between the target object and the reference line segment during the construction of a semantic map.

[0202] Specifically, during the construction of the semantic map, multiple bounding boxes corresponding to target objects are generated in the semantic map based on the collected image data. Each target object corresponds to a real-world object, and the shape of the bounding box corresponding to the target object is a cube. Furthermore, the category information corresponding to the constructed target object can be determined, thereby determining the target object's movement method, and moving the target object relative to the reference line segment according to the target movement method.

[0203] It should be noted that before adjusting the position of the target object relative to the reference line segments during the construction of the semantic map, multiple reference line segments in the semantic map need to be constructed first. After ensuring the accuracy of the multiple reference line segments, the target object is then moved based on the reference line segments.

[0204] In the embodiments of this application, the accuracy of the completed semantic map is ensured by adjusting and correcting the generated target objects in real time during the construction of the semantic map, without the need for subsequent adjustments to the semantic map.

[0205] like Figure 11 As shown, in any of the above embodiments, identifying target objects in the semantic map includes:

[0206] Step 1102, obtain the semantic map;

[0207] Step 1104: Extract the target objects from the semantic map.

[0208] This embodiment provides a scheme for adjusting the positional relationship between the target object and the reference line segment after the target object has been acquired and constructed; that is, in the process of correcting the constructed target object. Specifically, after acquiring the constructed semantic map, the target object is extracted from the semantic map.

[0209] In the embodiments of this application, the construction of the semantic map is achieved by extracting the target objects in the constructed semantic map and moving the target objects relative to the reference line segments.

[0210] like Figure 12 As shown, in any of the above embodiments, after identifying the target object in the semantic map, the method further includes:

[0211] Step 1202: Obtain the set of reference line segments in the semantic map;

[0212] Step 1204: Based on the target object, filter the reference line segments in the reference line segment set;

[0213] Among them, the reference line segment is the reference line segment in the set of reference line segments that is closest to the target object.

[0214] This embodiment provides a specific process for filtering reference line segments in a semantic map using target objects.

[0215] Specifically, after extracting the target object from the semantic map, multiple reference line segments, i.e., a set of reference line segments, are also extracted from the semantic map. The reference line segment closest to the target object is selected as the final reference line segment. When moving the target object relative to a reference line segment, the target object is most likely to have a positional relationship with the closest reference line segment. Therefore, selecting the closest reference line segment among the multiple reference line segments improves the accuracy of moving the target object relative to the reference line segment.

[0216] In the embodiments of this application, the accuracy of moving the target object is ensured by using the reference line segment closest to the target object in the set of reference line segments as the reference line segment and moving the target object based on the reference line segment.

[0217] In any of the above embodiments, the semantic map correction method further includes: determining the target bounding box corresponding to the target object in the semantic map; moving the target object based on the feature points in the target bounding box until the feature points and the reference line segment satisfy a preset relationship;

[0218] The feature points include at least one of a first feature point and a second feature point, wherein the first feature point is located on the peripheral sidewall of the target bounding box and the second feature point is located inside the target bounding box.

[0219] The reference line segment includes at least one of the first reference line segment, the second reference line segment, the third reference line segment, the fourth reference line segment, and the fifth reference line segment. The third reference line segment and the fourth reference line segment are obtained based on semantic map recognition. The third reference line segment and the fourth reference line segment are perpendicular and intersect.

[0220] In this embodiment, by identifying the target bounding box corresponding to the target object in the semantic map, and based on the feature points and reference line segments in the target bounding box, the target object is moved in the semantic map so that the positional relationship between the target object and the reference line segments meets the preset relationship, thereby adjusting the positional relationship between the target object and the reference line segments in the semantic map and realizing the correction of the positional relationship between the target object and the reference line segments in the semantic map.

[0221] In the embodiments of this application, the target bounding box is an object bounding box in the semantic map that matches the target object. The constructed semantic bounding box includes multiple object bounding boxes, and the target bounding box that matches the target object is searched among the multiple object bounding boxes.

[0222] The first and second feature points are the same as those in any of the above embodiments. When moving the target object based on feature points, it is necessary to find the corresponding first and second feature points based on the target bounding box. The first feature point can be a head feature point or a tail feature point of the target bounding box. The second feature point can also be a head feature point or a tail feature point of the target bounding box.

[0223] The first, second, third, and fourth reference line segments are the same as those in any of the above embodiments. These segments are line segments within the bounding boxes of the objects corresponding to the reference objects matched with the target object in the semantic map. The third and fourth reference line segments are perpendicular and intersecting. By identifying the line segments within the bounding boxes corresponding to the reference objects in the semantic map, two perpendicular and intersecting line segments are selected and used as the third and fourth reference line segments. These two line segments can be within the same or different bounding boxes.

[0224] In the embodiments of this application, a target object is extracted from the semantic map, and a corresponding target bounding box is found based on the target object. The position of the target object is adjusted based on at least one of the first and second feature points in the target bounding box, and at least one of the corresponding first, second, third, and fourth reference line segments. This adjusts the positional relationship between the target object and the reference line segments in the semantic map, improving the accuracy of the semantic map and ensuring the precision of the service robot's navigation using the semantic map.

[0225] In one embodiment according to this application, such as Figure 13 As shown, a semantic map correction device 1300 is proposed, comprising:

[0226] The recognition module 1302 is used to identify target objects in the semantic map;

[0227] The determination module 1304 is used to determine the target movement mode corresponding to the target object;

[0228] The moving module 1306 is used to move the target object relative to the reference line segment according to the target moving method. The reference line segment is matched with the target object to correct the semantic map.

[0229] The embodiments of this application propose a semantic map correction device for adjusting the positional relationship between a target object and a reference line segment in the semantic map, ensuring that the positional relationship between the target object and the reference line segment conforms to the actual situation, thereby improving the accuracy of the semantic map and further ensuring the accuracy of robot navigation through the semantic map.

[0230] In this embodiment, the semantic map is constructed by the robot using image data of the indoor scene. The semantic map includes multiple target objects, each corresponding to an object in the indoor scene. After extracting the target object, the target movement mode matched to that target object can be determined, with different categories of target objects corresponding to different target movement modes.

[0231] It should be noted that the position of the object relative to the wall may be different for different target objects. Therefore, there is a corresponding target movement method for each target object to ensure that after the target object is moved by the target movement method, the positional relationship between the target object and the reference line segment in the semantic map conforms to the relationship between the object and the wall in the real scene.

[0232] In this embodiment, after determining the target's movement method, a reference line segment is used as a reference for the moving target object. In the semantic map, each reference line segment corresponds to a wall in the indoor scene; therefore, the semantic map includes multiple reference line segments. The target object allows selection of a reference line segment that matches it. The reference line segment is one of the multiple reference line segments in the semantic map.

[0233] In the embodiments of this application, by moving the target object extracted from the semantic map along the reference line segment according to the corresponding target movement method, the positional relationship between the target object and the reference line segment in the semantic map is adjusted, thereby correcting the positional relationship between the target object and the reference line segment in the semantic map, improving the accuracy of the semantic map, and ensuring the accuracy of the service robot's navigation through the semantic map.

[0234] In the above embodiments, the determining module 1304 is used to determine the category information corresponding to the target object;

[0235] Semantic map correction device 1300, including:

[0236] The filtering module is used to filter at least one target movement method from a preset set of movement methods based on category information.

[0237] This embodiment provides a process for determining the movement mode of a target based on the category information of the target object.

[0238] During the movement of a target object, it can be moved using a single target movement method or multiple target movement methods. When there are multiple target movement methods, the target positions of these methods do not conflict.

[0239] In the embodiments of this application, the target movement method that matches the category information is found among multiple preset movement methods by using the category information of the target object, so as to ensure the accuracy of the positional relationship between the reference line segment and the target object in the semantic map after the target object is moved by the target movement method.

[0240] In any of the above embodiments, the reference line segment includes a first reference line segment and a filtering module, which is used to determine the first preset movement mode in the preset movement mode set as the target movement mode based on the category information.

[0241] The recognition module 1302 is used to recognize a first feature point in the target object, the first feature point being located on the peripheral sidewall of the target object;

[0242] The moving module 1306 is used to move the target object relative to the first reference line segment until the first distance between the first feature point and the first reference line segment is less than the first threshold.

[0243] In this embodiment, a first preset movement method is proposed, which moves the target object to a position that is in contact with the first reference line segment.

[0244] Specifically, the bounding box corresponding to the target object is cubic in shape, and includes a bottom face, a top face, and side walls. The bottom face is the surface of the target object that is in contact with the ground, the top face is opposite to the bottom face, and the side walls are the surfaces located between the top and bottom faces. The first feature point is located on the side wall of the bounding box corresponding to the target object. During the construction of the semantic map, a corresponding bounding box needs to be created for each target object, and the shape of the bounding box is either cuboid or cube.

[0245] In this embodiment, the first preset movement method involves moving the first feature point of the target object to a distance less than a first threshold from the first reference line segment, ensuring that the target object aligns with the first reference line segment in the semantic map. The first distance is the distance of the target object along the vertical direction of the first reference line segment.

[0246] In the embodiments of this application, the target object is moved toward the direction of the first reference line segment until the first distance between the two is less than the first threshold, ensuring that the target object is in contact with the wall.

[0247] In any of the above embodiments, the reference line segment includes a second reference line segment; the filtering module is used to determine the second preset movement mode in the preset movement mode set as the target movement mode based on the category information.

[0248] The recognition module 1302 identifies a second feature point in the target object, the second feature point being located inside the target object;

[0249] The moving module 1306 is used to move the target object relative to the second reference line segment until the second feature point coincides with the second reference line segment.

[0250] In this embodiment, a second preset movement method is proposed, which moves the target object to a position that coincides with the second reference line segment, and the target object is attracted to the middle of the second reference line segment, that is, by embedding the target object into the middle of the second reference line segment.

[0251] In this embodiment, the second preset movement method is to move the second feature point of the target object to a position that coincides with the second reference line segment, so as to ensure that the target object coincides with the second reference line segment in the semantic map.

[0252] It should be noted that the second preset movement method is applicable to target objects whose category information is windows, doors, etc.

[0253] In the embodiments of this application, by determining the second feature point inside the target object and moving the target object using the second preset movement method, the second feature point in the target object coincides with the second reference line segment, thereby moving the target object that may be embedded in the wall into the wall and further improving the accuracy of the semantic map.

[0254] In any of the above embodiments, the reference line segment includes a third reference line segment and a fourth reference line segment; the filtering module is used to determine the third preset movement mode in the preset movement mode set as the target movement mode based on the category information.

[0255] The recognition module 1302 is used to recognize the third reference line segment and the fourth reference line segment in the semantic map, wherein the third reference line segment is perpendicular to and intersects the fourth reference line segment;

[0256] The moving module 1306 is used to move the target object along the direction parallel to the third reference line segment until the second distance between the target object and the fourth reference line segment is less than the second threshold.

[0257] In this embodiment, a third preset movement method is proposed, which moves the bounding box of the target object to a position aligned with the third reference line segment and the fourth reference line segment. That is, the two side walls of the bounding box of the target object are aligned with the third reference line segment and the fourth reference line segment, respectively.

[0258] The second distance is the distance between the target object and the third reference line segment in the direction perpendicular to the fourth reference line segment. After the target object is moved by the third preset movement method, the second distance is less than the second threshold, so that the target object is aligned with the third reference line segment in the left and right directions.

[0259] In this embodiment, the third preset movement method is to move the target object to a position aligned with the adjacent third reference line segment and fourth reference line segment.

[0260] It should be noted that the third preset movement method is applicable to target objects whose category information is doors, wardrobes, etc.

[0261] In the embodiments of this application, by determining the fourth reference line segment adjacent to the third reference line segment and moving the target object through the third preset movement method, the moved target object is aligned with the reference line segment on the left and right. This realizes the alignment setting of the target object that may be aligned with two adjacent walls with the third and fourth reference line segments, further improving the accuracy of the semantic map.

[0262] In any of the above embodiments, the reference line segment includes a fifth reference line segment; the filtering module is used to determine the fourth preset movement mode in the preset movement mode set as the target movement mode based on the category information.

[0263] The moving module 1306 is used to move the target object along the direction parallel to the fifth reference line segment until the third distance between the center point of the target plane of the target object and the center point of the fifth reference line segment is less than the third threshold. The target plane is the plane of the target object that is close to the fifth reference line segment.

[0264] In this embodiment, a fourth preset movement method is proposed, which moves the target object to a position that is centered and aligned with the fifth reference line segment, that is, the position where the center point of the target plane in the target bounding box of the target object corresponds to the center point of the fifth reference line segment.

[0265] Here, the target plane is the plane within the target object that is closest to the fifth reference line segment. This plane is the sidewall of the target object closest to the fifth reference line segment. The third distance is the distance between the center point of the target plane and the center point of the fifth reference line segment. When the third distance is less than a third threshold, it can be determined that the target object is centered and aligned with the fifth reference line segment.

[0266] In this embodiment, the fourth preset movement method is to move the target object to the middle position of the fifth reference line segment so that the two are centered and aligned.

[0267] In the embodiments of this application, by determining the center point on the target plane of the target object and the center point on the fifth reference line segment, as well as the third distance between the two center points, the target object is moved in a fourth preset manner, so that the moved target object is centered and aligned with the fifth reference line segment. This achieves the center alignment of the target object that may be placed in the middle of the wall with the fifth reference line segment, further improving the accuracy of the semantic map.

[0268] In any of the above embodiments, the number of target movement methods is at least two;

[0269] The determination module 1304 is used to determine the execution order of at least two target movement methods based on category information, when at least two target movement methods are obtained through filtering.

[0270] The move module, 1306, is used to move target objects sequentially according to at least two target move methods, in the order of execution.

[0271] This embodiment provides a process for moving a target object according to multiple target movement methods.

[0272] Specifically, different target movement methods change the positional relationship between the target object and the reference line segment. Moving the target object relative to the reference line segment using multiple target movement methods can further improve the accuracy of the positional relationship between the target object and the reference line segment. Furthermore, based on the category information corresponding to the target object, the execution order of multiple target movement methods can be determined, and the multiple target movement methods can be executed sequentially according to this execution order.

[0273] In the embodiments of this application, when the extracted target object corresponds to multiple target movement methods, the target object is moved relative to the reference line segment in sequence according to the multiple target movement methods, thereby ensuring that the moved target object matches the actual placement of the object and improving the accuracy of the corrected semantic map.

[0274] In any of the above embodiments, the category information includes the object type; the determination module 1304 is used to determine the execution order based on the target mapping relationship and the object type, wherein the target mapping relationship is the mapping relationship between the object type and the execution order.

[0275] In this embodiment, a scheme for the execution order corresponding to multiple target movement methods is found based on the object type corresponding to the target object.

[0276] Specifically, when multiple target movement methods are found based on category information, the object type in the category information is obtained, and the execution order can be found based on the mapping relationship between object type and target.

[0277] In the embodiments of this application, the execution order corresponding to different target objects can be accurately determined by the object type in the category information. When it is necessary to move the target object using multiple target movement methods, the movement can be carried out in the straight-line order, which can improve the accuracy of the movement.

[0278] In any of the above embodiments, the semantic map correction device 1300 includes:

[0279] The acquisition module is used to acquire image data;

[0280] The building module is used to construct target objects in the semantic map based on image data.

[0281] This embodiment provides a scheme for adjusting the positional relationship between the target object and the reference line segment during the construction of a semantic map.

[0282] Specifically, during the construction of the semantic map, multiple bounding boxes corresponding to target objects are generated in the semantic map based on the collected image data. Each target object corresponds to a real-world object, and the shape of the bounding box corresponding to the target object is a cube. Furthermore, the category information corresponding to the constructed target object can be determined, thereby determining the target object's movement method, and moving the target object relative to the reference line segment according to the target movement method.

[0283] In the embodiments of this application, the accuracy of the completed semantic map is ensured by adjusting and correcting the generated target objects in real time during the construction of the semantic map, without the need for subsequent adjustments to the semantic map.

[0284] In any of the above embodiments, the semantic map correction device 1300 includes:

[0285] The acquisition module is used to acquire the semantic map;

[0286] The extraction module is used to extract target objects from the semantic map.

[0287] This embodiment provides a scheme for adjusting the positional relationship between the target object and the reference line segment after the target object has been acquired and constructed; that is, in the process of correcting the constructed target object. Specifically, after acquiring the constructed semantic map, the target object is extracted from the semantic map.

[0288] In the embodiments of this application, the construction of the semantic map is achieved by extracting the target objects in the constructed semantic map and moving the target objects relative to the reference line segments.

[0289] In any of the above embodiments, the acquisition module is used to acquire a set of reference line segments in the semantic map;

[0290] Semantic map correction device 1300, including:

[0291] The filtering module is used to filter reference segments in the reference segment set based on the target object. The reference segment is the reference segment in the reference segment set that is closest to the target object.

[0292] This embodiment provides a specific process for filtering reference line segments in a semantic map using target objects.

[0293] In the embodiments of this application, the accuracy of moving the target object is ensured by using the reference line segment closest to the target object in the set of reference line segments as the reference line segment and moving the target object based on the reference line segment.

[0294] In any of the above embodiments, the determining module 1304 is used to determine the target bounding box corresponding to the target object in the semantic map;

[0295] The moving module 1306 is used to move the target object based on feature points in the target bounding box until the feature points and the reference line segments satisfy a preset relationship.

[0296] The feature points include at least one of a first feature point and a second feature point, wherein the first feature point is located on the peripheral sidewall of the target bounding box and the second feature point is located inside the target bounding box.

[0297] The reference line segment includes at least one of the first reference line segment, the second reference line segment, the third reference line segment, the fourth reference line segment, and the fifth reference line segment. The third reference line segment and the fourth reference line segment are obtained based on semantic map recognition. The third reference line segment and the fourth reference line segment are perpendicular and intersect.

[0298] In this embodiment, by identifying the target bounding box corresponding to the target object in the semantic map, and based on the feature points and reference line segments in the target bounding box, the target object is moved in the semantic map so that the positional relationship between the target object and the reference line segments meets the preset relationship, thereby adjusting the positional relationship between the target object and the reference line segments in the semantic map and realizing the correction of the positional relationship between the target object and the reference line segments in the semantic map.

[0299] In the embodiments of this application, a target object is extracted from the semantic map, and a corresponding target bounding box is found based on the target object. The position of the target object is adjusted based on at least one of the first and second feature points in the target bounding box, and at least one of the corresponding first, second, third, and fourth reference line segments. This adjusts the positional relationship between the target object and the reference line segments in the semantic map, improving the accuracy of the semantic map and ensuring the precision of the service robot's navigation using the semantic map.

[0300] In one embodiment according to this application, such as Figure 14 As shown, a semantic map correction device 1400 is proposed, including: a processor 1402 and a memory 1404, wherein the memory 1404 stores a program or instructions; the processor 1402 executes the program or instructions stored in the memory 1404 to implement the steps of the semantic map correction method as described in any of the above embodiments, and thus has all the beneficial technical effects of the semantic map correction method in any of the above embodiments, which will not be elaborated further here.

[0301] In one embodiment of this application, a computer program product is proposed. When executed by a processor, the computer program product implements the steps of the semantic map correction method as described in any of the above embodiments, and thus has all the beneficial technical effects of the semantic map correction method in any of the above embodiments, which will not be elaborated further here.

[0302] In one embodiment of this application, a readable storage medium is provided, on which a program or instructions are stored. When executed by a processor, the program or instructions implement the steps of the semantic map correction method as described in any of the above embodiments. Therefore, it possesses all the beneficial technical effects of the semantic map correction method in any of the above embodiments, which will not be elaborated further here.

[0303] In one embodiment according to this application, such as Figure 15 As shown, a service robot 1500 is proposed, including: a semantic map correction device 1300 as in any of the above embodiments, and / or a computer program product 1502 as in any of the above embodiments, and / or a readable storage medium 1504 as in any of the above embodiments. Therefore, it has all the beneficial technical effects of the semantic map correction device 1300, and / or the computer program product 1502, and / or the readable storage medium 1504 as in any of the above embodiments, which will not be elaborated further here.

[0304] It should be clarified that in the claims, description, and accompanying drawings of this application, the term "multiple" refers to two or more objects. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description process, not to indicate or imply that the device or element referred to must have the described specific orientation, or be constructed and operated in a specific orientation. Therefore, these descriptions should not be construed as limitations on this application. The terms "connection," "installation," "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection between multiple objects, a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects or an indirect connection between multiple objects through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in this application can be understood based on the specific circumstances of the above data.

[0305] In the claims, description, and accompanying drawings of this application, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in the embodiments or examples of this application. In the claims, description, and accompanying drawings of this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0306] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for correcting a semantic map, characterized in that, include: Identify the target objects in the semantic map; Determine the target movement method corresponding to the target object; According to the target movement method, the target object is moved relative to the reference line segment, and the reference line segment is matched with the target object to correct the semantic map; The reference line segment is the reference line segment that is closest to the target object among the multiple reference line segments in the semantic map.

2. The semantic map correction method according to claim 1, characterized in that, Determining the target movement mode corresponding to the target object includes: Determine the category information corresponding to the target object; Based on the category information, at least one of the target movement methods from the preset movement method set is selected.

3. The semantic map correction method according to claim 2, characterized in that, The reference line segment includes a first reference line segment, and the step of filtering at least one target movement mode from a preset movement mode set based on the category information includes: Based on the category information, the first preset movement method in the preset movement method set is determined as the target movement method; The first preset movement method includes: Identify a first feature point in the target object, wherein the first feature point is located on the peripheral sidewall of the target object; The target object is moved relative to the first reference line segment until the first distance between the first feature point and the first reference line segment is less than a first threshold.

4. The semantic map correction method according to claim 2, characterized in that, The reference line segment includes a second reference line segment; The step of filtering at least one target mobility method from a preset set of mobility methods based on the category information includes: Based on the category information, the second preset movement method in the preset movement method set is determined as the target movement method; The second preset movement method includes: Identify a second feature point in the target object, wherein the second feature point is located inside the target object; Move the target object relative to the second reference line segment until the second feature point coincides with the second reference line segment.

5. The semantic map correction method according to claim 2, characterized in that, The reference line segment includes a third reference line segment and a fourth reference line segment; The step of filtering at least one target mobility method from a preset set of mobility methods based on the category information includes: Based on the category information, the third preset movement method in the preset movement method set is determined as the target movement method; The third preset movement method includes: Identify the third reference line segment and the fourth reference line segment in the semantic map, wherein the third reference line segment and the fourth reference line segment are perpendicular and intersect; Move the target object along the direction parallel to the third reference line segment until the second distance between the target object and the fourth reference line segment is less than the second threshold.

6. The semantic map correction method according to claim 2, characterized in that, The reference line segment includes the fifth reference line segment; The step of filtering at least one target mobility method from a preset set of mobility methods based on the category information includes: Based on the category information, the fourth preset movement method in the preset movement method set is determined as the target movement method; The fourth preset movement method includes: Move the target object along the direction parallel to the fifth reference line segment until the third distance between the center point of the target plane of the target object and the center point of the fifth reference line segment is less than the third threshold. The target plane is the plane of the target object that is close to the fifth reference line segment.

7. The method for correcting a semantic map according to any one of claims 2 to 6, characterized in that, Moving the target object relative to the reference line segment according to the target movement method includes: If at least two target movement methods are obtained through screening, the execution order of the at least two target movement methods is determined based on the category information; The target object is moved sequentially according to the execution order and at least two of the target movement methods.

8. The method for correcting a semantic map according to any one of claims 2 to 6, characterized in that, The category information includes the object type; If at least two target movement methods are obtained through filtering, the execution order of the at least two target movement methods is determined based on the category information, including: The execution order is determined based on the target mapping relationship and the object type, wherein the target mapping relationship is the mapping relationship between the object type and the execution order.

9. The method for correcting a semantic map according to any one of claims 1 to 6, characterized in that, The process of identifying target objects in the semantic map includes: Acquire image data; The target object in the semantic map constructed based on the image data; or Obtain the semantic map; Extract the target objects from the semantic map.

10. The method for correcting a semantic map according to any one of claims 1 to 6, characterized in that, After identifying the target object in the semantic map, the method further includes: Obtain the set of reference line segments in the semantic map; Based on the target object, the reference line segments in the reference line segment set are selected, and the reference line segments are the wall line segments in the reference line segment set that are closest to the target object.

11. The method for correcting a semantic map according to any one of claims 1 to 6, characterized in that, The method further includes: Determine the target bounding box corresponding to the target object in the semantic map; The target object is moved based on the feature points in the target bounding box until the preset relationship between the feature points and the reference line segment is satisfied. The feature point includes at least one of a first feature point and a second feature point, wherein the first feature point is located on the peripheral sidewall of the target bounding box, and the second feature point is located inside the target bounding box; The reference line segment includes at least one of a first reference line segment, a second reference line segment, a third reference line segment, a fourth reference line segment, and a fifth reference line segment. The third reference line segment and the fourth reference line segment are identified based on the semantic map. The third reference line segment and the fourth reference line segment are perpendicular and intersect.

12. A semantic map correction device, characterized in that, include: A memory that stores programs or instructions; A processor, configured to implement the steps of the semantic map correction method as described in any one of claims 1 to 11 when executing the program or instructions.

13. A readable storage medium, characterized in that, When the readable storage medium is executed by a processor, it implements the steps of the semantic map correction method as described in any one of claims 1 to 11.

14. A service robot, characterized in that, include: The semantic map correction apparatus as described in claim 12; and / or The readable storage medium as described in claim 13.

Images