Map generation method, generation device and readable storage medium

Abstract

The application provides a map generation method, a map generation device and a readable storage medium. The map generation method comprises the following steps: acquiring one or more first images, wherein the first images are acquired by scanning a surrounding environment; determining a tracking identification code of at least one object according to the first images; determining an object to be added with point cloud data according to associated information of the tracking identification code; and adding point cloud data of the object to be added with point cloud data to a grid map.

Description

Technical Field

[0001] This application relates to the field of robotics, and more specifically, to a map generation method, generation apparatus, and readable storage medium. Background Technology

[0002] In related technologies, mobile robots use depth cameras to scan indoor environments, acquire point cloud data of furniture or appliances, and add it to a grid map. Existing solutions consume significant resources in adding point cloud data to the grid map, resulting in slow processing speeds. When the robot is stationary, it continuously adds nearly identical point cloud data, leading to low map generation efficiency. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art or related technologies.

[0004] Therefore, the first aspect of this application proposes a method for generating maps.

[0005] The second aspect of this application proposes a map generation apparatus.

[0006] A third aspect of this application proposes a map generation apparatus.

[0007] The fourth aspect of this application proposes a readable storage medium.

[0008] The fifth aspect of this application proposes a computer program product.

[0009] The sixth aspect of this application proposes an electronic device.

[0010] In view of this, a first aspect of this application proposes a map generation method, comprising: acquiring one or more first images, wherein the first images are acquired by scanning the surrounding environment; determining a tracking identification code for at least one object based on the first images; determining an object to which point cloud data is to be added based on the association information of the tracking identification code; and adding the point cloud data of the object to which point cloud data is to be added to a raster map.

[0011] The map generation method provided in this application can be used for robots, specifically for home care robots. When the robot first enters the area it is caring for, it can first create a map of the area, that is, create a map of all the rooms in the space where the robot is located, so that the robot can move according to the map and track the caring target during the care process, thus ensuring the care effect.

[0012] Furthermore, the map generation method includes: First, scanning the surrounding environment using the robot's image acquisition device, specifically scanning the room environment where the robot is located, to obtain a first image of the room. The robot's image acquisition device can be a standard camera or a depth camera. When the image acquisition device is a standard camera, it can acquire an RGB image of the room; when the image acquisition device is a depth camera, it can acquire both the RGB image and the depth image of the room. The RGB image can be used to identify objects included in the image. The depth image can be used to identify the relative positions of objects in the image and the robot. The first image can be an RGB image.

[0013] Furthermore, based on the first image, at least one object detection box can be determined, and then the object's tracking identification code can be determined based on the object's detection box. That is, in the RGB image of the room, objects can be identified to determine their tracking identification codes, thus determining whether the object has appeared in a previous RGB image. If the object has appeared in a previous RGB image, its tracking identification code can be determined, which is to say, object identification is performed.

[0014] Furthermore, after determining the object's tracking identification code in the first image, the associated information of that tracking identification code can be obtained from the system. Specifically, this associated information can include the number of times the object's point cloud data was added to the raster map during the map generation process, the timing of such addition, and other information. Thus, during the map generation process based on the first image, the object to which point cloud data needs to be added can be determined based on the associated information of the object's tracking identification code, thereby improving the map generation.

[0015] Furthermore, after identifying the object to which point cloud data needs to be added—that is, when it is determined that the object needs its point cloud data added to the grid map—the point cloud data of that object can be added to the grid map. This allows for map generation based on the grid map and the point cloud data added to the grid map. In other words, the association information of the object's tracking identification code determines whether the object needs its point cloud data added to the grid map. This avoids repeatedly adding the same object's point cloud data to the grid map, effectively reducing the excessive computational load caused by multiple point cloud data additions, lessening the burden on the robot's system, and thus improving map generation efficiency.

[0016] The map generation method provided in this application can be used in home care robots. Through the robot's image acquisition device, a first image of the robot's environment can be obtained. Before adding point cloud data of objects, the tracking identification code of the object can be determined based on the first image. Then, based on the association information of the object's tracking identification code, the object to which point cloud data is to be added is identified, that is, whether the object's point cloud data needs to be added to the grid map. The association information can include the number of times the object's point cloud data is added to the grid map during the map generation process, the timing of the addition, etc. In other words, by determining whether the object needs to have its point cloud data added to the grid map based on the association information of the object's tracking identification code, it avoids repeatedly adding the same object's point cloud data to the grid map. This effectively reduces the excessive computational load caused by multiple point cloud data additions, reduces the burden on the robot's system, and thus improves the efficiency of map generation. It can also achieve map construction even with limited computing power.

[0017] According to a second aspect of this application, a map generation apparatus is proposed, comprising: an acquisition unit for acquiring one or more first images; a determination unit for determining a tracking identification code of at least one object based on the first images; and determining the object to which point cloud data is to be added based on the association information of the tracking identification code; an addition unit for adding the point cloud data to a raster map if the determination result is yes; and a generation unit for adding the point cloud data of the object to which point cloud data is to be added to the raster map.

[0018] According to a third aspect of this application, a map generation apparatus is proposed, comprising: a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, perform the following steps: acquiring one or more first images; determining a tracking identification code for at least one object based on the first image; determining an object to which point cloud data is to be added based on the association information of the tracking identification code; and adding the point cloud data of the object to which point cloud data is to be added to a raster map.

[0019] According to a fourth aspect of this application, a readable storage medium is proposed, on which a program or instructions are stored, which, when executed by a processor, perform the following steps: acquiring one or more first images; determining a tracking identification code for at least one object based on the first image; determining the object to which point cloud data is to be added based on the association information of the tracking identification code; and adding the point cloud data of the object to which point cloud data is to be added to a raster map.

[0020] According to a fifth aspect of this application, a computer program product is proposed, comprising a computer program or instructions, which, when executed by a processor, perform the following steps: acquiring one or more first images; determining a tracking identification code for at least one object based on the first image; determining an object to which point cloud data is to be added based on the association information of the tracking identification code; and adding the point cloud data of the object to which point cloud data is to be added to a raster map.

[0021] According to a sixth aspect of this application, an electronic device is proposed, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, the following steps are implemented: acquiring one or more first images; determining a tracking identification code of at least one object based on the first image; determining an object to which point cloud data is to be added based on the association information of the tracking identification code; and adding the point cloud data of the object to which point cloud data is to be added to a raster map.

[0022] Additional aspects and advantages of this application will become apparent in the following description or may be learned by practice of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0024] Figure 1 A schematic flowchart of a map generation method provided in one embodiment of this application is shown;

[0025] Figure 2 A flowchart illustrating a map generation method according to yet another embodiment of this application is shown;

[0026] Figure 3 A schematic flowchart of a map generation method provided in another embodiment of this application is shown;

[0027] Figure 4 A schematic flowchart of a map generation method provided in another embodiment of this application is shown;

[0028] Figure 5 A flowchart of a map generation method provided in one embodiment of this application is shown. Detailed Implementation

[0029] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.

[0031] The following reference Figures 1 to 5 This application describes a map generation method, a map generation apparatus, a readable storage medium, a computer program product, and an electronic device provided according to some embodiments of the present application.

[0032] like Figure 1 As shown, according to one embodiment of this application, a map generation method is proposed, comprising:

[0033] S102, acquire one or more first images;

[0034] S104, Based on the first image, determine the tracking identification code of at least one object;

[0035] S106, Based on the association information of the tracking identification code, determine the object to which point cloud data is to be added;

[0036] S108, Add the point cloud data of the object to which point cloud data is to be added to the raster map.

[0037] The map generation method provided in this application can be used for robots. Specifically, the robot can be a home care robot or a home service robot. When the robot first enters the area where it is being cared for, a map of the area can be created, that is, a map of all the rooms in the space where the robot is located. This allows the robot to move according to the map and track the caring target during the care process, ensuring the effectiveness of the care.

[0038] Furthermore, the map generation method includes: First, scanning the surrounding environment using the robot's image acquisition device, specifically scanning the room environment where the robot is located, to obtain a first image of the room. The robot's image acquisition device can be a standard camera or a depth camera. When the image acquisition device is a standard camera, it can acquire an RGB image of the room; when the image acquisition device is a depth camera, it can acquire both the RGB image and the depth image of the room. The RGB image can be used to identify objects included in the image. The depth image can be used to identify the relative positions of objects in the image and the robot. The first image can be an RGB image.

[0039] Furthermore, based on the first image, at least one object's detection box can be determined, and then the object's tracking identification code can be determined based on the object's detection box. That is, in the RGB image of the room, objects can be identified to determine their tracking identification codes, thus determining whether the object appeared in a previous RGB image. If the object appeared in a previous RGB image, its tracking identification code can be determined, i.e., object identification is performed. Specifically, when the robot first acquires an image of an object, it can generate the object's tracking identification code and store it in the robot's system. During map generation, when it is determined that an object appeared in a previous RGB image, the system can be searched for the object's tracking identification code, i.e., the tracking identification code of the object acquired in the first image can be determined.

[0040] Furthermore, after determining the object's tracking identification code in the first image, the associated information of that tracking identification code can be obtained from the system. Specifically, this associated information can include the number of times the object's point cloud data was added to the raster map during the map generation process, the timing of such additions, and other information. Thus, during the map generation process based on the first image, the tracking identification code of the object determines whether it is necessary to add the object's point cloud data to the raster map to improve the map generation.

[0041] Furthermore, after identifying the object to which point cloud data needs to be added, if the determination result is yes (i.e., when it is determined that the object needs to have its point cloud data added to the grid map), then the object's point cloud data can be added to the grid map so that a map can be generated based on the grid map and the point cloud data added to the grid map. In related technologies, during map generation, the robot adds the point cloud data of all furniture or appliances from the acquired room image to the grid map, resulting in the point cloud data of the same object with similar shapes being added multiple times to the grid map. This increases the computational load of the system and affects the efficiency of map generation. In other words, this application determines whether an object needs its point cloud data added to the grid map based on the association information of the object's tracking identification code. This avoids repeatedly adding the point cloud data of the same object to the grid map, effectively reducing the excessive computational load caused by multiple point cloud data additions, reducing the burden on the robot's system, and thus improving the efficiency of map generation.

[0042] Furthermore, after adding the point cloud data of an object to the raster map, it is also necessary to update the association information of the corresponding tracking identification code of the object so that the association information can be accurately judged when adding the point cloud data of the object next time.

[0043] The map generation method provided in this application can be used in home care robots. Through the robot's image acquisition device, a first image and a second image of the robot's environment can be acquired. Based on these images, point cloud data of at least one object within the environment can be obtained. During the generation process, by adding the point cloud data of the objects in the environment to the grid map, the three-dimensional dimensions and positions of the objects can be extracted. The map is then generated based on the three-dimensional dimensions and positions of the objects in the three maps. Before adding the object's point cloud data, the object's tracking identification code can be determined from the first image. Then, based on the association information of the object's tracking identification code, the object to which point cloud data is to be added is identified, i.e., whether the object's point cloud data needs to be added to the grid map. The association information can include the number of times the object's point cloud data is added to the grid map during map generation, the timing of the addition, and other information. In other words, the system determines whether point cloud data needs to be added to the grid map based on the association information of the object's tracking identification code. It then makes a judgment based on the object's point cloud data being added to the grid map, thus avoiding repeatedly adding the same object's point cloud data to the grid map. This effectively reduces the excessive computational load caused by multiple point cloud data additions, lessens the burden on the robot's system, and improves map generation efficiency. It also enables map construction even with limited computing power. Specifically, the association information includes at least one of the following: the frame interval between the first image and the second image, where the second image corresponds to the image where the object's point cloud data was last added to the grid map; the historical number of times the object's point cloud data has been added to the grid map; the quantity information of one or more tracking identification codes; and a preset quantity corresponding to the first image, where the tracking identification code corresponds to the first image.

[0044] In the above embodiments, the step of determining the object to which point cloud data is to be added based on the association information of the tracking identification code includes: determining the object as the object to which point cloud data is to be added when the frame interval is greater than a first preset number of times.

[0045] In this embodiment, the association information may include the frame interval between the first image and the second image. The first image is the image acquired before the point cloud data of the object being added this time, and correspondingly, the second image can be the image corresponding to the last time the point cloud data of an object was added to the raster map. For example, after acquiring the first image, the tracking identification code of the sofa is determined based on the first image to determine whether the sofa's point cloud data needs to be added to the raster map this time. Conversely, the image corresponding to the last time the sofa's point cloud data was added to the raster map is the second image.

[0046] Furthermore, in the process of determining the object to which point cloud data needs to be added based on the association information of the tracking identification code, it can be determined whether the frame interval between the first image and the second image is greater than a first preset number of times. Only if it is greater than the first preset number of times can the object be determined as the object whose point cloud data needs to be added to the raster map. That is, after the point cloud data of an object is added to the raster map, at least the first preset number of frames must be skipped before the point cloud image of the object can be added again.

[0047] It is understandable that a robot's image acquisition device typically acquires images of the surrounding environment continuously. Therefore, the differences between the first images of adjacent frames are often very small, meaning that the differences in the structural and positional features of the objects represented by the first images of adjacent frames are minimal. Consequently, when adding point cloud data of an object, the differences between the point cloud data of the object determined by the first images of adjacent frames are also very small, making the addition of point cloud data largely meaningless. Therefore, adding point cloud data of an object by skipping frames can effectively reduce the amount of computation in the process of acquiring and adding point cloud data, thereby improving the efficiency of map generation.

[0048] In any of the above embodiments, the associated information further includes the historical number of times the object's point cloud data has been added to the grid map. The step of determining the object to which point cloud data is to be added based on the associated information of the tracking identification code includes: determining the object as the object to which point cloud data is to be added if the historical number is less than a second preset number; wherein the second preset number is the maximum number of times the object has point cloud data added.

[0049] In this embodiment, the associated information may further include the historical number of times an object's point cloud data has been added to the grid map, that is, the total number of times an object has had its point cloud data added to the grid map. Furthermore, it can only be determined that the object's point cloud data needs to be added to the grid map if the historical number of times the object's point cloud data has been added to the grid map is less than a second preset number.

[0050] In other words, by limiting the historical number of times an object's point cloud data has been added to the raster map to less than the second predicted number, it can be determined whether the object's point cloud data needs to be added again. This limits the total number of times an object's point cloud data can be added to the raster map; only when the total number of times the object's point cloud data has been added to the raster map is less than the second preset number can the object's point cloud data be added again. This effectively reduces the number of times the same object's point cloud data is added to the raster map, thereby reducing the computational load during map construction and improving map construction efficiency while ensuring the accuracy of the object's three-dimensional dimensions and position.

[0051] In any of the above embodiments, before the step of adding the point cloud data of the object to be added to the raster map, the method further includes: if the number of objects to be added to the first image is greater than a preset number, determining the objects to be added to the raster map based on the historical number of times the point cloud data of the objects to be added to the raster map, wherein the number of objects to be added to the raster map is the same as the preset number; wherein the preset number is the maximum preset number of point cloud data of the object in one addition process.

[0052] In this embodiment, before adding the point cloud data of the object to be added to the raster map, the maximum number of point cloud data of the object that can be added in one addition process is first determined. This is the preset maximum number of point cloud data that can be added at one time, also known as the preset quantity. By setting the maximum quantity, the maximum number of point cloud data added in one addition process is limited, thus avoiding excessive computation caused by adding too much point cloud data at once. Combined with the setting of the image frame interval during the point cloud data addition process for an object, and the setting of the maximum number of times point cloud data for an object can be added, this ensures that the point cloud data of each object can be added to the raster map, while also balancing the number of times point cloud data for each object is added, thus guaranteeing the map's quality. Simultaneously, it reduces computational load and improves map construction efficiency.

[0053] Specifically, the preset quantity can be set in advance when the robot system is integrated, or it can be set according to the computing power of the robot system.

[0054] Furthermore, the number of objects requiring point cloud data is determined. If the number of objects in the first image requiring point cloud data is greater than a preset number, it indicates that the number of objects requiring point cloud data exceeds the maximum number of point cloud data to be added at one time. Therefore, based on the historical number of times each object's point cloud data has been added to the raster map, the number of objects requiring point cloud data is determined to be the same as the preset number. Then, the point cloud data of these objects is added to the raster map, thereby ensuring that the number of point cloud data added at one time does not exceed the preset number, thus ensuring the efficiency of point cloud data addition.

[0055] Furthermore, the step of determining the objects to be added to the raster map based on the historical number of times the objects to be added to the raster map have been added includes: determining the priority of multiple objects in ascending order of the historical number of times the objects to be added to the raster map have been added; and determining the objects to be added to the raster map based on the priority.

[0056] Specifically, when the number of objects to be added to point cloud data exceeds a preset number, the historical number of times each object's point cloud data has been added to the raster map is obtained. Then, these historical counts are prioritized in ascending order, meaning the object with the fewest historical counts has the highest priority, and the object with the most historical counts has the lowest priority. Furthermore, based on the priority of the objects requiring point cloud data addition, the same number of objects as the preset number are selected, and their point cloud data is added to the raster map. This ensures that the number of point cloud data added at one time does not exceed the preset number, guaranteeing efficient point cloud data addition.

[0057] Prioritization is determined by the historical number of times object point cloud data is added to the raster map. This ensures that point cloud data of objects with fewer additions are added in a limited number of times, thereby further balancing the number of additions of point cloud data for each object and ensuring the quality of the map.

[0058] In the above embodiments, the step of determining the object to which point cloud data is to be added based on the association information of the tracking identification code includes: when the number of tracking identification codes is greater than a preset number, randomly selecting the same number of tracking identification codes from the tracking identification codes and determining them as the objects to which point cloud data is to be added.

[0059] In this embodiment, if the number of tracking identification codes for an object determined from the first image exceeds a preset number, then tracking identification codes of the same preset number can be randomly selected from these codes, and these codes can be identified as objects for which power data is to be added. This limits the maximum amount of point cloud data added in a single process, thus avoiding excessive computation due to adding too much point cloud data at once, reducing computational load, and improving map construction efficiency.

[0060] According to one embodiment of this application, such as Figure 2 As shown, a map generation method is proposed, including:

[0061] S202, acquire one or more first images;

[0062] S204, Based on the first image, determine the detection box of at least one object shown;

[0063] S206, compare the shape of the detection box with the shape of the detection boxes of objects in other previously acquired images;

[0064] S208, Based on the comparison results, determine the tracking identification code of at least one object;

[0065] S210, Based on the association information of the tracking identification code, determine the object to which point cloud data is to be added;

[0066] S212, Add the point cloud data of the object to which point cloud data is to be added to the raster map.

[0067] In this embodiment, such as Figure 5 As shown, based on the first image, at least one object's detection box can be determined, and then the object's tracking identification code can be determined based on the object's detection box. That is, in the RGB image of the room, objects can be identified to determine their tracking identification codes, thus determining whether the object has appeared in a previous RGB image. If the object has appeared in a previous RGB image, its tracking identification code can be determined.

[0068] Specifically, the object's detection bounding box can be matched with the detection bounding boxes of multiple objects in the previous RGB image to determine the overlap. If the overlap is greater than a preset value, it can be determined that the current object appeared in the previous RGB image. Then, based on the previous RGB image, the tracking identification code of the object in the first image can be determined. For example, when the object is first acquired, it can be marked as a sofa, television, or cabinet, etc., which is the object's tracking identification code. When the object is recognized in the first image, the tracking identification code of the object can be determined.

[0069] According to one embodiment of this application, such as Figure 3 As shown, a map generation method is proposed, including:

[0070] S302, acquire one or more first images;

[0071] S304, Based on the first image, determine the detection box of at least one object shown;

[0072] S306, compare the shape of the detection box with the shape of the detection boxes of objects in other previously acquired images;

[0073] S308, Based on the comparison results, determine the tracking identification code of at least one object;

[0074] S310, based on the association information of the tracking identification code, determines the object to which point cloud data is to be added;

[0075] S312, Obtain the third image;

[0076] S314, Generate point cloud data of the object to be added with point cloud data based on the first image and the third image;

[0077] S316, Add the point cloud data of the object to which point cloud data is to be added to the raster map.

[0078] In this embodiment, the robot's image acquisition device can acquire a third image simultaneously with the first image. Specifically, the third image can be a depth image corresponding to the content displayed in the first image. The positional relationship between the object in the third image and the robot can be determined through the third image. Thus, based on the first image (RGB image) and the third image (depth image), point cloud data of the object can be generated, and then the point cloud data of the object can be added to a raster map to generate a map.

[0079] Furthermore, such as Figure 5 As shown, the steps for generating point cloud data of an object based on the first image and the third image include: determining a mask of the object to which point cloud data is to be added based on the first image; determining the first position information of the object to which point cloud data is to be added in the surrounding environment based on the third image; and generating point cloud data of the object to which point cloud data is to be added based on the mask and the first position information.

[0080] Specifically, firstly, based on the first image acquired by the image acquisition device, the mask of the object to which point cloud data is to be added can be determined, that is, the area occupied by the object to which point cloud data is to be added in the first image, so as to determine the structural features of the object.

[0081] Then, based on the third image, the first position information of the object to which point cloud data is to be added can be determined in the surrounding environment, that is, the specific location of the object's point cloud data in the raster map. Then, the object's mask is combined with the first position information, that is, the object's structural features are combined with its specific location in the surrounding environment, thereby determining the object's point cloud data. After determining the object's point cloud data, it can be added to the raster map.

[0082] Further, the step of determining the first position information of the object to which point cloud data is to be added in the surrounding environment based on the third image includes: determining the positional relationship between the object to which point cloud data is to be added and the robot based on the third image; acquiring the second position information of the robot in the surrounding environment through the robot's position acquisition device; and determining the first position information based on the positional relationship and the second position information.

[0083] Specifically, firstly, the positional relationship between the object and the robot can be determined based on the third image. It is understood that the third image is a depth image; through the depth image, the distance between the object displayed in the image and the image acquisition device, as well as the shooting angle, can be identified. Therefore, based on the distance and shooting angle, the positional relationship between the object and the robot can be determined.

[0084] Furthermore, the robot's position acquisition device can also determine a second position information of the robot in the surrounding environment. Specifically, the robot's position acquisition device can detect the distance between the robot and walls in the surrounding environment, thereby determining the robot's specific position in the surrounding environment, i.e., the second position information.

[0085] Furthermore, after determining the robot's second position information in the surrounding environment, the first position information of the object in the surrounding environment can be determined by combining the positional relationship between the object and the robot.

[0086] According to one embodiment of this application, such as Figure 4 As shown, a map generation method is proposed, including:

[0087] S402, acquire one or more first images;

[0088] S404, Based on the first image, determine the detection box of at least one object shown;

[0089] S406, compare the shape of the detection box with the shape of the detection boxes of objects in other previously acquired images;

[0090] S408, Based on the comparison results, determine the tracking identification code of at least one object;

[0091] S410, based on the association information of the tracking identification code, determines the object to which point cloud data is to be added;

[0092] S412, Obtain the third image;

[0093] S414, Generate point cloud data of the object to which point cloud data is to be added based on the first image and the third image;

[0094] S416, Add the point cloud data of the object to which point cloud data is to be added to the raster map;

[0095] S418, Based on the point cloud data, extract the 3D dimensions and position of the object whose point cloud data is to be added in the raster map;

[0096] S420 generates a map in a raster map based on the three-dimensional dimensions and location.

[0097] In this embodiment, such as Figure 5As shown, in the map generation process, a raster map is first constructed based on the environment corresponding to the map. The raster map only represents the outer framework of the environment. Further, using the raster map as a base, the point cloud data of all furniture, appliances, and other objects in the room are added to the raster map. This allows the extraction of the three-dimensional dimensions of the objects and their specific locations within the raster map. Thus, based on the size and location of the objects, a map of the robot's surrounding environment can be generated from the raster map.

[0098] Specifically, the map can be a semantic map, meaning that during the map generation process, semantic settings can be applied to all identified objects to generate a semantic map, allowing users to operate on the map by inputting semantic information, thus improving user convenience.

[0099] According to a second aspect of this application, a map generation apparatus is proposed, comprising: an acquisition unit for acquiring one or more first images; a determination unit for determining a tracking identification code of at least one object based on the first images; and determining the object to which point cloud data is to be added based on the association information of the tracking identification code; an addition unit for adding the point cloud data to a raster map if the determination result is yes; and a generation unit for adding the point cloud data of the object to which point cloud data is to be added to the raster map.

[0100] The map generation device provided in this application can be used in home care robots. Through the robot's image acquisition device, a first image of the robot's environment can be acquired. Before adding point cloud data of objects, the tracking identification code of the object can be determined based on the first image. Then, based on the association information of the object's tracking identification code, the object to which point cloud data is to be added is identified, that is, whether the object's point cloud data needs to be added to the grid map. The association information can include the number of times the object's point cloud data is added to the grid map during the map generation process, the timing of the addition, etc. In other words, by determining whether the object needs to have its point cloud data added to the grid map based on the association information of the object's tracking identification code, it avoids repeatedly adding the same object's point cloud data to the grid map. This effectively reduces the excessive computational load caused by multiple point cloud data additions, reduces the burden on the robot's system, and thus improves the efficiency of map generation. It also enables map construction even with limited computing power.

[0101] Furthermore, the determining unit is specifically used to determine the object as the object to be added with point cloud data when the frame interval is greater than a first preset number of times.

[0102] Furthermore, the determining unit is specifically used to determine the object as the object to be added point cloud data when the number of historical occurrences is less than the second preset number of occurrences;

[0103] The second preset number of times is the maximum number of times point cloud data is added to the object.

[0104] Furthermore, the determining unit is specifically used to determine the objects to be added to the point cloud data when the number of objects to be added to the point cloud data in the first image is greater than the preset number, based on the historical number of times the objects to be added to the raster map have been added, and the number of objects to be added to the point cloud data is the same as the preset number.

[0105] The preset quantity is the maximum number of point cloud data of an object that can be added in a single addition process.

[0106] Furthermore, the determining unit is specifically used to determine the priority of multiple objects to be added to the point cloud data based on the historical number of times the objects to be added to the raster map have been added, in ascending order;

[0107] Based on priority, determine the objects for which point cloud data will be added.

[0108] Furthermore, the determining unit is specifically used to randomly select the same number of tracking identification codes from the tracking identification codes when the number of tracking identification codes is greater than the preset number, and to determine the object to be added with point cloud data.

[0109] Furthermore, the determining unit is specifically used to determine a detection box for at least one object shown, based on the first image;

[0110] The detection bounding box is compared with the shape of the detection bounding boxes of objects in other previously acquired images;

[0111] Based on the comparison results, determine the tracking identification code of at least one object.

[0112] Furthermore, the acquisition unit is also used to acquire a third image;

[0113] The generating apparatus also includes a generating unit for generating point cloud data of the object based on the first image and the third image.

[0114] Furthermore, the generating unit is specifically used to determine a mask for at least one object based on the first image;

[0115] Based on the third image, determine the first location information of at least one object in the surrounding environment;

[0116] Based on the mask and the first position information, point cloud data of at least one object is generated.

[0117] Furthermore, the generation unit is also specifically used to determine the positional relationship between at least one object and the robot based on the third image;

[0118] The robot's position acquisition device is used to acquire secondary position information of the robot in the surrounding environment;

[0119] The first location information is determined based on the location relationship and the second location information.

[0120] Furthermore, the generation unit is also used to extract the three-dimensional size and position of the object in the raster map based on the point cloud data;

[0121] A map is generated in a grid map based on its 3D dimensions and position. The generation device provided in this application can be used in home care robots. Through the robot's image acquisition device, a first image of the robot's environment can be acquired. Based on this first image, the tracking identification code of an object is determined. Then, based on the object's tracking identification code, it is determined whether the object's point cloud data needs to be added to the grid map. Information such as the number of times the object's point cloud data is added to the grid map and the timing of this addition can be provided during the generation process. In other words, the decision to add the object's point cloud data to the grid map is based on the object's point cloud data, thus avoiding repeatedly adding the same object's point cloud data to the grid map. This effectively reduces the excessive computational load caused by multiple point cloud data additions, reduces the burden on the robot's system, and improves generation efficiency. It can also be implemented even with limited computing power.

[0122] According to a third aspect of this application, a map generation apparatus is proposed, comprising: a processor and a memory, wherein the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the map generation method as described in any of the above embodiments. Therefore, this map generation apparatus possesses all the beneficial effects of the aforementioned map generation methods, which will not be elaborated further here.

[0123] According to a fourth aspect of this application, a readable storage medium is proposed, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the map generation method as described in any of the above embodiments. Therefore, this readable storage medium possesses all the beneficial effects of the above-described map generation method, which will not be elaborated further here.

[0124] According to a fifth aspect of this application, a computer program product is proposed, comprising a computer program or instructions that, when executed by a processor, implement the steps of the map generation method as described in any of the above embodiments. Therefore, this computer program product possesses all the beneficial effects of the aforementioned map generation method, which will not be elaborated further here.

[0125] According to a sixth aspect of this application, an electronic device is proposed, including a processor and a memory. The memory stores a program or instructions executable on the processor. When the program or instructions are executed by the processor, they implement the steps of any of the map generation methods described above. Therefore, this electronic device possesses all the beneficial effects of the map generation methods described above, which will not be elaborated further here.

[0126] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified and limited. The terms "connection," "installation," and "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0127] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating a map, characterized in that, include: Acquire one or more first images, wherein the first images are acquired by scanning the surrounding environment; Based on the first image, determine the tracking identification code of at least one object; Based on the association information of the tracking identification code, determine the object to which point cloud data is to be added; Add the point cloud data of the object to which point cloud data is to be added to the raster map; The associated information includes at least one of the following: The frame interval between the first image and the second image, wherein the second image is the image corresponding to the last time the point cloud data of the object was added to the raster map; The point cloud data of the object has been added to the historical number of times in the raster map; The tracking identification code includes information on the number of one or more of the tracking identification codes and a preset number corresponding to the first image, wherein the tracking identification code corresponds to the first image.

2. The generation method according to claim 1, characterized in that, The step of determining the object to which point cloud data is to be added based on the association information of the tracking identification code includes: If the frame interval is greater than a first preset number of times, the object is determined to be the object to be added point cloud data.

3. The generation method according to claim 1, characterized in that, The step of determining the object to which point cloud data is to be added based on the association information of the tracking identification code includes: If the number of historical occurrences is less than the second preset number, the object is determined to be the object whose point cloud data is to be added. The second preset number of times is the maximum number of times point cloud data is added to the object.

4. The generation method according to claim 2 or 3, characterized in that, Before the step of adding the point cloud data of the object to which point cloud data is to be added to the raster map, the method further includes: If the number of objects to be added to point cloud data in the first image is greater than the preset number, the objects to be added to point cloud data are determined according to the historical number of times the objects to be added to point cloud data have been added to the raster map, and the number of objects to be added to point cloud data is the same as the preset number. The preset quantity is the maximum number of point cloud data of the object that can be added during a single addition process.

5. The generation method according to claim 4, characterized in that, The step of determining one or more objects to be added to the point cloud data based on the historical number of times the objects to be added to the point cloud data have been added to the raster map includes: The priority of multiple objects to be added to the point cloud data is determined according to the historical number of times the objects have been added to the raster map, from smallest to largest. Based on the priority, the objects for which point cloud data is to be added are determined.

6. The generation method according to claim 1, characterized in that, The step of determining the object to which point cloud data is to be added based on the association information of the tracking identification code includes: If the number of tracking identification codes is greater than the preset number, a tracking identification code of the same preset number is randomly selected from the tracking identification codes and identified as the object to be added to the point cloud data.

7. The generation method according to any one of claims 1 to 3, characterized in that, The step of determining the tracking identification code of at least one object based on the first image includes: Based on the first image, determine a detection box for at least one object shown; The shape of the detection box is compared with the shape of the detection boxes of objects in other previously acquired images; Based on the comparison results, the tracking identification code of the at least one object is determined.

8. The generation method according to any one of claims 1 to 3, characterized in that, Before the step of adding the point cloud data of the object to which point cloud data is to be added to the raster map, the method further includes: Obtain the third image; The point cloud data of the object to which point cloud data is to be added is generated based on the first image and the third image.

9. The generation method according to claim 8, characterized in that, The step of generating point cloud data of the object to be added with point cloud data based on the first image and the third image includes: Based on the first image, determine the mask of the object to which point cloud data is to be added; Based on the third image, determine the first location information of the object to which point cloud data is to be added in the surrounding environment; Based on the mask and the first position information, point cloud data of the object to which point cloud data is to be added is generated.

10. The generation method according to claim 9, characterized in that, The generation method is used for robots. The step of determining the first position information of the object to which point cloud data is to be added in the surrounding environment based on the third image includes: Based on the third image, determine the positional relationship between the object to which point cloud data is to be added and the robot; The robot's position acquisition device acquires the robot's second position information in the surrounding environment; The first location information is determined based on the location relationship and the second location information.

11. The generation method according to any one of claims 1 to 3, characterized in that, The generation method further includes: The map is generated based on the raster map and the point cloud data added to the raster map.

12. The generation method according to claim 11, characterized in that, The step of generating the map based on the raster map and the point cloud data added to the raster map includes: Based on the point cloud data, extract the three-dimensional dimensions and position of the object whose point cloud data is to be added in the grid map; The map is generated in the grid map based on the three-dimensional dimensions and location.

13. A map generation apparatus, characterized in that, include: The acquisition unit is used to acquire one or more first images; The determining unit is configured to determine the tracking identification code of at least one object based on the first image; as well as Based on the association information of the tracking identification code, determine the object to which point cloud data is to be added; An adding unit is used to add the point cloud data to the raster map if the judgment result is yes; The generation unit is used to add the point cloud data of the object to which point cloud data is to be added to the raster map; The associated information includes at least one of the following: the frame interval between the first image and the second image, wherein the second image is the image corresponding to the last time the point cloud data of the object was added to the grid map; the historical number of times the point cloud data of the object has been added to the grid map; the quantity information of one or more tracking identification codes and a preset quantity corresponding to the first image, wherein the tracking identification code corresponds to the first image.

14. A map generation apparatus, characterized in that, include: A processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the map generation method as described in any one of claims 1 to 12.

15. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the map generation method as described in any one of claims 1 to 12.

16. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the map generation method as described in any one of claims 1 to 12.

17. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the map generation method as described in any one of claims 1 to 12.

Images