A method and device for constructing a map

Abstract

The embodiment of the present invention provides a map construction method and device, which relates to the field of image processing technology. The above scheme includes: performing object segmentation on the environmental image collected by the image acquisition device carried by the robot to obtain the image area where the object is located; determining the environmental area corresponding to the image area according to the environmental position of the robot; searching for the target grid where the environmental area is located in the initial environmental map constructed based on the grid; obtaining the division coefficient of the environmental area according to the influence of the object on the robot's task execution reflected by the image area; dividing the target grid into multiple sub-grids based on the division coefficient, and using the multiple sub-grids to update the target grid. The map construction scheme provided by the embodiment of the present invention can realize map construction.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a map construction method and device. Background Art

[0002] Robots need to complete their work based on environmental maps, so building maps for robots has become a hot research topic today. Robots can be equipped with various sensors. In the process of building maps for robots, as the robots move, the sensors can collect data in real time, so as to build maps based on the collected data. The above sensors can include position sensors, distance sensors, image acquisition devices, etc.

[0003] Therefore, a map construction solution is needed to build a map based on the data collected by the sensors carried by the robot. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a map construction method and device to achieve map construction. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a map construction method, the method comprising:

[0006] Performing object segmentation on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located;

[0007] Determining an environmental area corresponding to the image area according to the environmental position of the robot;

[0008] In an initial environment map constructed based on a grid, a target grid where the environment area is located is searched, wherein the attributes of the grid are used to describe the environment information of the area corresponding to the grid in the environment;

[0009] Obtaining a division coefficient of the environmental area according to the degree of influence of the object on the robot's execution of a task as reflected by the image area;

[0010] Based on the division coefficient, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

[0011] In one embodiment of the present invention, the step of performing object segmentation on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located includes:

[0012] Extract features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map;

[0013] Performing semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects;

[0014] Performing instance segmentation on the initial feature map to obtain a second feature map describing different objects;

[0015] According to the obtained first feature map and the second feature map, an image region where each object is located is determined in the environment image.

[0016] In one embodiment of the present invention, determining the image area where each object is located in the environment image according to the obtained first feature map and the second feature map includes:

[0017] Determining, according to the first feature map, areas where objects of different types are located in the environment image;

[0018] Determining object pixels describing different objects in the environment image according to the second feature map;

[0019] Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel in the area where each type of object is located, the object pixel closest to the regional pixel is determined based on the coordinates of the object pixel after offset;

[0020] The region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located is determined as the image region where the type of object is located.

[0021] In one embodiment of the present invention, dividing the target grid into a plurality of sub-grids based on the division coefficient includes:

[0022] The target grid is divided into a plurality of sub-grids based on the division coefficient and the area of ​​the environmental region.

[0023] In one embodiment of the present invention, dividing the target grid into a plurality of sub-grids based on the division coefficient includes:

[0024] Calculating the size of the sub-grid based on the corresponding relationship between the division coefficient and the size of the sub-grid;

[0025] The target grid is divided into multiple sub-grids according to the calculated size.

[0026] In one embodiment of the present invention, when all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid is expressed by the following expression:

[0027]

[0028] Wherein, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

[0029] In a second aspect, an embodiment of the present invention further provides a map construction device, the device comprising:

[0030] An object segmentation module is used to segment the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located;

[0031] An area determination module, used to determine the environmental area corresponding to the image area according to the environmental position of the robot;

[0032] A grid search module, used to search for a target grid where the environmental area is located in an initial environmental map constructed based on grids, wherein the attributes of the grid are used to describe environmental information of the area corresponding to the grid in the environment;

[0033] A coefficient obtaining module, used for obtaining the division coefficient of the environmental area according to the influence degree of the object on the robot's execution of the task reflected by the image area;

[0034] The grid updating module is used to divide the target grid into a plurality of sub-grids based on the division coefficient, and use the plurality of sub-grids to update the target grid.

[0035] In one embodiment of the present invention, the object segmentation module includes:

[0036] The feature extraction submodule is used to extract features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map;

[0037] A semantic segmentation submodule, used to perform semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects;

[0038] An instance segmentation submodule, used to perform instance segmentation on the initial feature map to obtain a second feature map describing different objects;

[0039] The region determination submodule is used to determine the image region where each object is located in the environment image according to the obtained first feature map and the second feature map.

[0040] In one embodiment of the present invention, the region determination submodule is specifically used to:

[0041] Determining, according to the first feature map, areas where objects of different types are located in the environment image;

[0042] Determining object pixels describing different objects in the environment image according to the second feature map;

[0043] Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel in the area where each type of object is located, the object pixel closest to the regional pixel is determined based on the coordinates of the object pixel after offset;

[0044] The region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located is determined as the image region where the type of object is located.

[0045] In one embodiment of the present invention, the grid updating module is specifically used for:

[0046] Based on the division coefficient and the area of ​​the environment area, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

[0047] In one embodiment of the present invention, the grid updating module is specifically used for:

[0048] Calculating the size of the sub-grid based on the corresponding relationship between the division coefficient and the size of the sub-grid;

[0049] The target grid is divided into a plurality of sub-grids according to the calculated size, and the target grid is updated using the plurality of sub-grids.

[0050] In one embodiment of the present invention, when all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid is expressed by the following expression:

[0051]

[0052] Wherein, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

[0053] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0054] Memory, used to store computer programs;

[0055] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0056] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0057] Beneficial effects of the embodiments of the present invention:

[0058] It can be seen from the above that when the scheme provided by the embodiment of the present invention is applied to construct a map, the environmental image is segmented to obtain the image area where the object is located, and the environmental area corresponding to the image area is determined according to the environmental position of the robot. The environmental area can be regarded as the environmental area of ​​the above-mentioned object. After determining the above-mentioned environmental area, the target grid where the environmental area is located can be found in the initial environmental map. Since the above-mentioned image area can reflect the degree of influence of the object on the robot's execution of the task, the division coefficient of the above-mentioned environmental area can be obtained according to the degree of influence. In this way, after finding the target grid, the target grid can be divided into multiple sub-grids based on the above-mentioned division coefficient, and the target grid is updated using the divided multiple sub-grids, thereby realizing map construction.

[0059] In addition, each grid in the map corresponds to an area in the environment, and the attributes of the grid are used to describe the environmental information of the area corresponding to the grid. After the target grid is divided into multiple sub-grids, each sub-grid corresponds to a partial area of ​​the area corresponding to the target grid, and the attributes of each sub-grid are used to describe the environmental information of the partial area corresponding to it, so that the environmental information of the area corresponding to the target grid is described by the attributes of multiple sub-grids rather than by the attributes of one target grid. In this way, in the constructed map, the detailed information of the area corresponding to the target grid is richer, and for non-target grids, it is not necessary to divide them. In this way, when applying the map construction scheme provided by the embodiment of the present invention, the accuracy of the constructed map can be improved, and the data storage volume of the map can be guaranteed to be small. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0061] Figure 1a A schematic diagram of a flow chart of a first map construction method provided by an embodiment of the present invention;

[0062] Figure 1b A schematic diagram of the distribution of a main grid provided in an embodiment of the present invention;

[0063] Figure 1c A schematic diagram of the distribution of a map grid provided by an embodiment of the present invention;

[0064] Figure 2A schematic diagram of a flow chart of a second map construction method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of a flow chart of a third map construction method provided by an embodiment of the present invention;

[0066] Figure 4 A schematic diagram of a fourth map construction method provided by an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of a fifth map construction method provided by an embodiment of the present invention;

[0068] Figure 6 A schematic diagram of the structure of a first map construction device provided by an embodiment of the present invention;

[0069] Figure 7 A schematic diagram of the structure of a second map construction device provided by an embodiment of the present invention;

[0070] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.

[0072] First, the application scenario of the map construction solution provided by the embodiment of the present invention is described.

[0073] Since the real environment of the robot is usually constantly changing during its work, the constructed map needs to be updated to ensure its accuracy.

[0074] When updating the map, the image collected by the image acquisition device carried by the robot can be analyzed and compared with the existing map. If the map area reflected by the image is different from the map area of ​​the same position in the existing map, it means that the map area has changed. At this time, the map area in the existing map needs to be updated. Updating part of the map area in the existing map can be regarded as rebuilding the part of the map area, and updating the entire map can be regarded as rebuilding the entire map.

[0075] In addition, when updating the map, the map can be updated once at intervals, or the map can be updated continuously.

[0076] The map construction solution provided by the embodiment of the present invention is described in detail below through specific examples.

[0077] See also Figure 1a , Figure 1a This is a flow chart of a first map construction method provided by an embodiment of the present invention. The method includes the following steps S101-S105.

[0078] Step S101: performing object segmentation on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located.

[0079] Among them, the above-mentioned image acquisition device can be an RGB (Red Green Blue) camera, a depth camera, an infrared camera, a point cloud camera, etc.

[0080] The above-mentioned environment image may be collected in real time by the image acquisition device, or may be any one of multiple images collected by the image acquisition device during the working process of the robot.

[0081] The above objects can be roads, vehicles, trees, etc.

[0082] Specifically, the environment image may be segmented into objects in any of the following two implementations to obtain the image region where the object is located.

[0083] In a first implementation, object recognition may be performed on the environment image, so that the region where the object is located in the environment image may be determined, and the environment image may be segmented according to the determined region, thereby obtaining the image region where the object is located.

[0084] In the second implementation, you can Figure 2 In the illustrated embodiment, steps S101A-S101D perform object segmentation on the environment image to obtain the image region where the object is located, which will not be described in detail here.

[0085] In addition, the environment image may be segmented by using existing image segmentation technology to obtain the image region where the object is located, for example, existing panoramic segmentation technology.

[0086] Step S102: Determine the environmental area corresponding to the image area according to the environmental position of the robot.

[0087] Among them, the above-mentioned environmental position can be regarded as the position of the robot in the environment.

[0088] In one embodiment of the present invention, the robot may be equipped with a positioning system, so that the position of the robot detected by the positioning system may be obtained as the environmental position of the robot.

[0089] Specifically, the above environmental area can be determined by any one of the following two implementation methods.

[0090] In the first implementation method, the environmental position of the robot in various places in the environment can be obtained in advance, and the image position of the robot in various places in the environment in the image coordinate system can be obtained. According to the environmental position and image position of the robot, a conversion relationship between the image coordinate system and the environmental coordinate system is established. In this way, after obtaining the above-mentioned image area, the environmental area corresponding to the image area is determined according to the position information of the image area and the above-mentioned conversion relationship.

[0091] In the second implementation method, after obtaining the robot's environmental position and the robot's image position in the image coordinate system, the relative position between the robot and the object can be determined based on the robot's image position in the image coordinate system and the position of the above-mentioned image area, and the area of ​​the object in the environment can be determined based on the robot's environmental position and the relative position as the environmental area corresponding to the image area.

[0092] In addition, there may be multiple objects, and therefore, there may be multiple image regions where the objects are located. For each image region, the environmental region corresponding to the image region may be determined according to the environmental position of the robot.

[0093] Step S103: searching for the target grid where the environmental area is located in the initial environmental map constructed based on the grid.

[0094] The attributes of the grid are used to describe the environmental information of the area corresponding to the grid in the environment.

[0095] The environmental information of the corresponding area may include information such as the type and location of objects in the area and the location of the area.

[0096] The above-mentioned initial environment map may be an existing environment map, or may be constructed in real time based on various data currently collected by various sensors carried by the robot.

[0097] The construction process of the above initial environment map can be found in the subsequent embodiments and will not be described in detail here.

[0098] Specifically, in the initial environment map, each grid corresponds to an area in the environment. After determining the above environment area, the grid corresponding to the environment area can be found as the target grid according to the correspondence between the grids in the initial environment map and each area in the environment.

[0099] In addition, one environmental region may correspond to multiple grids. After finding multiple grids corresponding to the environmental region, all of the multiple grids can be used as target grids. Furthermore, there can be multiple environmental regions, so in the initial environmental map, multiple target grids where each environmental region is located can be found.

[0100] Step S104: Obtaining a division coefficient of the environment area according to the degree of influence of the object on the robot's task execution as reflected in the image area.

[0101] The above division coefficient represents the degree of division of the environmental area.

[0102] The larger the division coefficient is, the higher the degree of division of the environmental area is, and the more sub-areas are obtained after division.

[0103] The degree of influence of the above-mentioned objects on the robot's task execution can be understood as the robot's attention to the object. During the robot's task execution, the higher the robot's attention to the object, the more detailed information about the area where the object is located needs to be obtained, and thus the greater the influence of the object on the robot's task execution.

[0104] For example, when a robot is moving on the road while performing a task, it needs to focus on the vehicles on the road. At this time, it is necessary to obtain more detailed information about the area where the vehicles are located in order to perform braking, avoidance and other actions based on the information obtained. Therefore, the vehicles on the road have a greater impact on the robot's execution of tasks, and the division coefficient of the environmental area where the vehicles are located is larger.

[0105] For another example, when the robot is performing a task, it pays less attention to the sky and buildings. Therefore, the sky and buildings have less influence on the robot's task performance, and the division coefficient of the environmental area where the sky and buildings are located is smaller.

[0106] Specifically, when obtaining the division coefficient of the environmental area, an influence parameter representing the degree of influence of the object on the robot's task execution reflected by the image area can be obtained, and the division coefficient of the environmental area can be determined according to the obtained influence parameter.

[0107] In one embodiment of the present invention, when obtaining the above-mentioned influence parameters, an attention mechanism can be used to analyze the above-mentioned environmental image to obtain an attention parameter that characterizes the degree of attention of the image area in the environmental image, and the attention parameter is used as the influence parameter of the environmental area corresponding to the image area.

[0108] In addition, influence parameters that characterize the degree of influence of the object on the robot's task execution can also be preset manually.

[0109] For example, when planning the robot's route, obstacles, vehicles, road guide lines and other objects in the road have a greater impact on the planned route. At this time, larger impact parameters can be preset for obstacles, vehicles, road guide lines and other objects. Pedestrians, trees and other objects on both sides of the road have a smaller impact on the planned route. At this time, smaller impact parameters can be preset for pedestrians, trees and other objects.

[0110] Step S105: based on the division coefficient, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

[0111] Specifically, the target grid may be divided into a plurality of sub-grids by any one of the following two implementation methods.

[0112] In the first implementation, the number of sub-grids obtained after dividing the target grid can be calculated according to the division coefficient, and the target grid can be divided according to the calculated number of sub-grids. The larger the division coefficient, the more sub-grids there are.

[0113] For example, a calculation coefficient k greater than 0 may be preset, and after obtaining the above-mentioned division coefficient n, the calculation coefficient k is multiplied by the division coefficient n to obtain the multiplication result as the number of sub-grids m=k*n.

[0114] In the second implementation, the size of the sub-grid obtained after dividing the target grid can also be calculated according to the division coefficient, so as to divide the target grid according to the calculated sub-grid size. The larger the division coefficient, the smaller the sub-grid size.

[0115] For example, when the target grid is a rectangular grid, the height h1 of the rectangular grid can be divided by the division coefficient n to obtain the height h2 = h1 / n of the rectangular sub-grid, and the width of the sub-grid can be the same as the width of the target grid.

[0116] The specific implementation method of realizing the target grid division by calculating the sub-grid size can be found in the following Figure 5 Steps S105B-S105C in the illustrated embodiment are not described in detail here.

[0117] In addition, when dividing the target grid, for each environmental area, the target grid where the environmental area is located can be divided based on the division coefficient of the environmental area. Since different objects have different degrees of influence on the robot's execution of tasks, the division coefficients of different environmental areas are different, so the division degrees of the target areas where different environmental areas are located are different.

[0118] For example, one target grid may be divided into 4 sub-grids and another target grid may be divided into 16 sub-grids.

[0119] After the target grid is divided into multiple sub-grids, each sub-grid corresponds to a partial area in the area corresponding to the target grid. Therefore, the attributes of the multiple sub-grids can be generated according to the environmental information of the partial areas corresponding to the multiple sub-grids, so that the attributes of each sub-grid are used to describe the environmental information of the partial area in the area corresponding to the target grid, and the attributes of the multiple sub-grids jointly describe the environmental information of the area corresponding to the target grid, thereby realizing the update of the target grid.

[0120] It can be seen from the above that when the scheme provided by the embodiment of the present invention is applied to construct a map, the environmental image is segmented to obtain the image area where the object is located, and the environmental area corresponding to the image area is determined according to the environmental position of the robot. The environmental area can be regarded as the environmental area of ​​the above-mentioned object. After determining the above-mentioned environmental area, the target grid where the environmental area is located can be found in the initial environmental map. Since the above-mentioned image area can reflect the degree of influence of the object on the robot's execution of the task, the division coefficient of the above-mentioned environmental area can be obtained according to the degree of influence. In this way, after finding the target grid, the target grid can be divided into multiple sub-grids based on the above-mentioned division coefficient, and the target grid is updated using the divided multiple sub-grids, thereby realizing map construction.

[0121] In addition, each grid in the map corresponds to an area in the environment, and the attributes of the grid are used to describe the environmental information of the area corresponding to the grid. After the target grid is divided into multiple sub-grids, each sub-grid corresponds to a partial area of ​​the area corresponding to the target grid, and the attributes of each sub-grid are used to describe the environmental information of the partial area corresponding to it, so that the environmental information of the area corresponding to the target grid is described by the attributes of multiple sub-grids rather than by the attributes of one target grid. In this way, in the constructed map, the detailed information of the area corresponding to the target grid is richer, and for non-target grids, it is not necessary to divide them. In this way, when applying the map construction scheme provided by the embodiment of the present invention, the accuracy of the constructed map can be improved, and the data storage volume of the map can be guaranteed to be small.

[0122] The above-mentioned robot can be used as the executor of the map construction solution provided by the embodiment of the present invention. In this way, during the operation of the robot, the robot can realize functions such as rapid path finding and path planning based on the constructed map. Moreover, since the robot can continuously update the map based on the images collected by the onboard image acquisition device, the robot can be applied to a variety of complex and changeable environments such as shopping malls and hospitals.

[0123] The following is an explanation of the process of constructing the initial environment map based on the grid.

[0124] The grid can be of various shapes, such as a triangular grid, a rectangular grid, a hexagonal grid, etc. There can be different initial environment map construction processes for grids of different shapes.

[0125] In one embodiment of the present invention, when the shape of the above-mentioned grid is a rectangle, the length and width of the rectangular grid can be determined according to the length and width of the initial environment map to be constructed, and then the initial environment map can be constructed using the grid of the determined size.

[0126] In one embodiment of the present invention, when the shape of the above-mentioned grid is a regular hexagon, the map range of the initial environment map to be constructed can be first obtained, and within the map range, the position of the first regular hexagonal grid and the side length of the regular hexagonal grid are preset, and then the positions of other grids in the initial environment map are determined according to the preset grid positions and grid side lengths. The first regular hexagonal grid is called the central grid.

[0127] In one embodiment of the present invention, after setting the position and side length of the first regular hexagonal grid, the grid expansion can be performed within the initial environment map range according to the following expression to obtain multiple grids:

[0128]

[0129]

[0130] in, are the x-axis coordinate and y-axis coordinate of the center of the i-th grid obtained by expansion in the preset map coordinate system, C x , C y are the x-axis coordinate and y-axis coordinate of the center of the central grid in the map coordinate system, L is the side length of the preset regular hexagonal grid, and i is an integer between 0 and 6.

[0131] The grid obtained by expansion is called the main grid. After obtaining multiple main grids, each main grid can be used as the expansion center to perform grid expansion again to fill the map area around the main grid, thereby completing the construction of the initial environment map.

[0132] Figure 1b This is a schematic diagram of the distribution of some main grids in the initial environment map. Figure 1b In the figure, the main node hexagon in the middle of the image is the central grid. Each main node hexagon except the main node hexagon represents a main grid, and adjacent main node hexagons are separated by a certain distance.

[0133] Figure 1c This is a schematic diagram of the distribution of some grids in the initial environment map. Figure 1cIn the figure, the main node hexagon located in the center is the main grid or the central grid determined according to the above expression, and the hexagons around the main node hexagon are hexagonal grids expanded with the main node hexagon as the expansion center.

[0134] When performing object segmentation on the environment image, in addition to using the aforementioned Figure 1a In addition to the method mentioned in step S101 in the embodiment shown, the following method can also be used Figure 2 In the illustrated embodiment, steps S101A-S101D perform object segmentation on the environment image, thereby obtaining the image region where the object is located.

[0135] In one embodiment of the present invention, see Figure 2 , a flow chart of a second map construction method is provided. In this embodiment, the above step S101 can be implemented by the following steps S101A-S101D.

[0136] Step S101A: extracting features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map.

[0137] Extracting features of environmental images can be achieved through existing feature extraction models, algorithms, etc.

[0138] For example, the environment image can be processed using a Backbone network for extracting image features to obtain an initial feature map of the environment image. The Backbone network can be a neural network model trained using the Image-Net dataset.

[0139] When training the Backbone network, the network training loss can be calculated according to the following expression, so as to adjust the network parameters according to the calculated training loss.

[0140]

[0141] Where L is the training loss, N is the total number of feature maps output by the network, i is the i-th feature map, M is the total number of feature categories, c is the c-th feature category, and y ic is the actual value of the ith feature for the cth feature category. If the ith feature belongs to the cth feature category, then y ic is 1, if the i-th feature does not belong to the c-th feature category, then y ic is 0, p ic It is the predicted value of the i-th feature output by the network for the c-th feature category.

[0142] Step S101B: Perform semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects.

[0143] The first feature map describing objects of the same type may include the region of the object of the same type in the initial feature map, and the first feature map describing objects of different types may include different regions of objects of different types in the initial feature map.

[0144] For example, the object type may be a vehicle, and the initial feature map may include areas where multiple vehicles are located, such as area a where vehicle a is located and area b where vehicle b is located. Area a and area b can both describe the type of vehicle. Therefore, the first feature map describing the vehicle includes area a and area b.

[0145] For another example, based on the above example, there may be another object type, which is a pedestrian. The above initial feature map may include region c where pedestrian c is located. Region c can describe the type of pedestrian. In this way, the first feature map describing the vehicle includes region c.

[0146] In one embodiment of the present invention, semantic context information of each pixel in a feature map can be extracted according to a preset plurality of extraction coefficients on the semantic dimension of the image, thereby obtaining a feature map at a deeper level than the initial feature map. Based on the obtained features, the areas where objects of different types are located in the initial feature map are predicted, and then the initial feature map is segmented to obtain a first feature map describing different types.

[0147] For example, in the semantic dimension of the image, multi-scale semantic context information of the image can be extracted based on the decoupled ASPP (atrous spatial pyramid pooling).

[0148] The dilated convolution formula in ASPP is:

[0149]

[0150] Among them, i is the pixel value of the i-th pixel in the initial feature map, y[i] is the semantic context information of the i-th pixel extracted, K is the preset maximum value of the convolution size, k is the current convolution kernel size, x is the pixel value matrix of the above initial feature map, r is the preset void rate, and w[k] is the weight parameter of the current convolution kernel.

[0151] In addition, in the above step S101A, when the Backbone backbone network is used to extract features of the environmental image, after obtaining the semantic context information of the image, the obtained context information, that is, the deeper features, can be mapped to the preset pixel space to obtain a feature map in the pixel space. In this way, the area where the pixels with the same or small difference in pixel values ​​are located in the feature map can be determined, and then the area corresponding to the determined area in the initial feature map is searched according to the determined area, and then the initial feature map is divided according to the found area to obtain multiple first feature maps describing different types of objects, each first feature map describing one type of object.

[0152] In another embodiment of the present invention, the initial feature map can be processed by using existing semantic segmentation technology to obtain a first feature map describing different types of objects, which will not be described in detail here.

[0153] Step S101C: perform instance segmentation on the initial feature map to obtain a second feature map describing different objects.

[0154] In one implementation, the above-mentioned second feature map can be understood as a feature map including the area where the object is located, one second feature map is used to describe an object, one second feature map includes the area where an object is located in the initial feature map, different object pixels are used to describe different objects, and different second feature maps include the areas where different objects are located in the initial feature map.

[0155] In one embodiment of the present invention, the distance context information of each pixel in the initial feature map can be extracted according to a plurality of preset extraction coefficients in the dimension of pixel distance of the image, thereby obtaining a feature map at a deeper level than the extracted initial feature map. Based on the obtained features, the areas where different objects in the initial feature map are located are predicted, and then the initial feature map is reversely segmented to obtain a second feature map describing different objects.

[0156] For example, multi-scale distance context information of an image can be extracted based on the decoupled ASPP in the dimension of pixel distance of the image.

[0157] In addition, when the Backbone network is used to extract features of the environment image in the above step S101A, after obtaining deeper features of the image in the distance dimension, pixel points whose pixel values ​​are greater than a preset threshold can be determined from the obtained features, and based on the determined pixel points, the area corresponding to the determined pixel points is detected in the initial feature map as the area where the object is located, thereby performing reverse segmentation on the initial feature map to obtain a second feature map describing different objects.

[0158] In another embodiment of the present invention, the image may be processed by using existing instance segmentation technology to obtain object pixels describing different objects, which will not be described in detail here.

[0159] Step S101D: Determine the image region where each object is located in the environment image according to the obtained first feature map and the second feature map.

[0160] Specifically, since the first feature map and the second feature map are feature maps segmented from the initial feature map, and there is a correspondence between the pixels in the initial feature map and the pixels in the environment image, the areas where different types of objects are located can be determined in the environment image according to the positions of the pixels in the first feature map, and the areas where different objects are located can be determined in the environment image according to the positions of the pixels in the second feature map. Based on the two determined areas, the image area where each object is located can be finally determined.

[0161] For the sake of simplicity of description, the area where objects of different types are located is referred to as the first area, and the area where different objects are located is referred to as the second area.

[0162] When determining the image area where each object is located based on the above two areas, the regional center of each second area can be calculated, and for each pixel point in the first area, the regional center closest to the pixel point can be determined, and the area where multiple pixels corresponding to the same regional center are located can be determined as the image area where the object of the object type described by the first area is located.

[0163] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to construct a map, semantic segmentation and instance segmentation are performed on the extracted initial feature map, so that the first feature map describing different types of objects and the second feature map describing different objects can be accurately determined. According to the first feature map, the type of object corresponding to each pixel in the environment image can be known, and according to the second feature map, the distribution of objects corresponding to each pixel in the environment image can be known. In this way, according to the first feature map and the second feature map, the image area where each object is located can be accurately determined, and according to the image area, the environment area and the target grid can be accurately determined, so that after updating the target grid, the accuracy of the constructed map can be improved.

[0164] Since the accuracy of the region where the object is located predicted based on the deeper features of the image in the distance dimension may be poor when performing instance segmentation on the initial feature map, the accuracy of the region center of the calculated second region may be poor.

[0165] To solve the above problems, in one embodiment of the present invention, see Figure 3, a flow chart of a third map construction method is provided. In this embodiment, the above step S101D can be implemented by the following steps S101D1-S101D4.

[0166] Step S101D1: Determine areas where objects of different types are located in the environment image according to the first feature map.

[0167] The implementation method of determining the areas where the different types of objects in the environment image are located can refer to the above step S101D, which will not be repeated here.

[0168] Step S101D2: Determine object pixels describing different objects in the environment image according to the second feature map.

[0169] Specifically, the regions where different objects in the environment image are located can be determined based on the second feature map, and the region centers of the determined regions can be calculated as object pixel points describing the objects formed by the second feature map.

[0170] Step S101D3: Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel points in the area where each type of object is located, the object pixel point closest to the regional pixel point is determined based on the coordinates of the object pixel points after offset.

[0171] The position offset parameters may be manually set or obtained based on model parameters in a pre-trained instance segmentation model.

[0172] The above position offset parameter may be a vector with size and direction.

[0173] Specifically, for each object pixel point, based on the above-mentioned position offset parameters, the offset coordinates can be calculated according to the coordinates of the object pixel point according to the direction of the position offset parameters and the offset amount, and then for each regional pixel point, based on the coordinates of the object pixel point after the offset, the object pixel point closest to the regional pixel point is determined.

[0174] Step S101D4: Determine the region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located as the image region where the object of this type is located.

[0175] Specifically, in the above step S101D3, the object pixel point that is closest to the offset regional pixel point is determined, and multiple regional pixel points corresponding to the same object pixel point can be found. The multiple regional pixel points can be considered to be pixel points belonging to the same object. Therefore, the area where the multiple regional pixel points are located is the image area where the single object is located.

[0176] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to construct a map, the coordinates of each object pixel after offset are calculated based on the preset position offset parameters. It can be considered that before determining the image area, the position of the object pixel is corrected, and the accuracy of the object description by the offset object pixel is higher than that of the original object pixel. In this way, based on the regional pixel points and the offset object pixel points, the accuracy of the image area where each object is located can be further improved, thereby further improving the accuracy of the constructed map.

[0177] When dividing the target grid into multiple sub-grids, in addition to dividing based on the division coefficient, the division can also be combined with the division coefficient and other information of the environmental area.

[0178] In one embodiment of the present invention, see Figure 4 , a flowchart of a fourth map construction method is provided. In this embodiment, the above step S105 can be implemented by the following step S105A.

[0179] Step S105A: based on the division coefficient and the area of ​​the environment area, the target grid is divided into a plurality of sub-grids, and the plurality of sub-grids are used to update the target grid.

[0180] Specifically, when the target grid is divided into multiple sub-grids, the division may be performed based on the division coefficient and the area of ​​the environmental area. The larger the area of ​​the environmental area, the higher the degree of division of the target grid and the more sub-grids obtained after the division.

[0181] Similar to the above step S105, the number or size of the sub-grids may be calculated according to the division coefficient and the area of ​​the environment region, so that the target grid may be divided according to the number or size of the sub-grids.

[0182] In one embodiment of the present invention, multiple coefficient intervals of the division coefficient and multiple area intervals of the regional area can be preset, and the combination of each coefficient interval and each area interval corresponds to the number or size of a sub-grid. In this way, after obtaining the above-mentioned division coefficient and the regional area of ​​the environmental area, the interval where the division coefficient is located and the area interval where the regional area of ​​the environmental area is located can be determined, thereby determining the number or size of the sub-grid based on the two determined intervals.

[0183] In addition, according to the division coefficient and the area of ​​the environmental area, the number or size of the sub-grids may be calculated by other calculation methods, which are not listed here one by one.

[0184] In addition, when dividing the target grid, the division can also be combined with other information of the environmental area, such as the shape and position of the environmental area.

[0185] After the multiple sub-grids are divided, the method of using the multiple sub-grids to update the target grid is the same as the method of updating the target grid in the above step S105, which will not be described in detail here.

[0186] From the above, it can be seen that in the map construction scheme provided by the embodiment of the present invention, since the larger the area of ​​the environmental area, the more vague the environmental information described by the attributes of the corresponding grid is, and it is difficult to describe the details of the area, therefore, when dividing the target grid, considering the two factors of the division coefficient and the area of ​​the environmental area, the details of the larger environmental area in the constructed map can be richer, thereby improving the accuracy of the map.

[0187] The specific implementation method of achieving the target grid division by calculating the sub-grid size mentioned in the above step S105 is described below.

[0188] In one embodiment of the present invention, see Figure 5 , a flowchart of a fifth map construction method is provided. In this embodiment, the above step S105 can be implemented by the following steps S105B-S105C.

[0189] Step S105B: Calculate the size of the sub-grid based on the correspondence between the division coefficient and the size of the sub-grid.

[0190] The corresponding relationship between the above-mentioned division coefficient and the size of the sub-grid may be preset based on the shape of the sub-grid.

[0191] In one embodiment of the present invention, when all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid can be expressed by the following expression:

[0192]

[0193] Among them, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

[0194] In the above expression, the area S of the initial environment map can be considered as a fixed constant, so after obtaining the above division coefficient n, the side length L of the sub-grid can be calculated according to the expression. The larger the division coefficient n, the smaller the side length of the sub-grid.

[0195] In this solution, through the above expression, the size of the sub-grid can be accurately calculated according to the division coefficient, so as to accurately divide the target grid and improve the accuracy of map construction.

[0196] In addition, when all grids are regular hexagonal grids, the distance between each grid and the other six adjacent grids is the same. In this way, when the robot performs path planning based on the grid, the complexity of the robot's path planning can be reduced, thereby helping the robot to perform tasks.

[0197] The above grids may also be grids of other shapes, and there may be multiple corresponding relationships between the division coefficients and the sizes of the sub-grids.

[0198] Step S105C: Divide the target grid into a plurality of sub-grids according to the calculated size, and use the plurality of sub-grids to update the target grid.

[0199] After the size of the sub-grid is calculated, the target grid may be divided into a plurality of sub-grids of the size according to the obtained size.

[0200] After the multiple sub-grids are divided, the method of using the multiple sub-grids to update the target grid is the same as the method of updating the target grid in the above step S105, which will not be described in detail here.

[0201] From the above, it can be seen that when a map is constructed using the solution provided by an embodiment of the present invention, after obtaining the division coefficient, the size of the sub-grid can be accurately calculated based on the correspondence between the division coefficient and the size of the sub-grid, so that the target grid can be accurately divided into multiple sub-grids according to the calculated size, so that after the target grid is updated using the multiple sub-grids, the accuracy of the map can be improved.

[0202] Corresponding to the above-mentioned map construction method, an embodiment of the present invention further provides a map construction device.

[0203] In one embodiment of the present invention, see Figure 6 , a schematic diagram of the structure of a first map construction device is provided, the device comprising:

[0204] The object segmentation module 601 is used to segment the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located;

[0205] The region determination module 602 is used to determine the environment region corresponding to the image region according to the environment position of the robot;

[0206] A grid search module 603 is used to search for a target grid where the environmental area is located in an initial environmental map constructed based on grids, wherein the attributes of the grid are used to describe environmental information of the area corresponding to the grid in the environment;

[0207] A coefficient obtaining module 604 is used to obtain a division coefficient of the environment area according to the influence of the object on the robot's execution of the task reflected by the image area;

[0208] The grid updating module 605 is used to divide the target grid into a plurality of sub-grids based on the division coefficient, and use the plurality of sub-grids to update the target grid.

[0209] It can be seen from the above that when the scheme provided by the embodiment of the present invention is applied to construct a map, the environmental image is segmented to obtain the image area where the object is located, and the environmental area corresponding to the image area is determined according to the environmental position of the robot. The environmental area can be regarded as the environmental area of ​​the above-mentioned object. After determining the above-mentioned environmental area, the target grid where the environmental area is located can be found in the initial environmental map. Since the above-mentioned image area can reflect the degree of influence of the object on the robot's execution of the task, the division coefficient of the above-mentioned environmental area can be obtained according to the degree of influence. In this way, after finding the target grid, the target grid can be divided into multiple sub-grids based on the above-mentioned division coefficient, and the target grid is updated using the divided multiple sub-grids, thereby realizing map construction.

[0210] In addition, each grid in the map corresponds to an area in the environment, and the attributes of the grid are used to describe the environmental information of the area corresponding to the grid. After the target grid is divided into multiple sub-grids, each sub-grid corresponds to a partial area of ​​the area corresponding to the target grid, and the attributes of each sub-grid are used to describe the environmental information of the partial area corresponding to it, so that the environmental information of the area corresponding to the target grid is described by the attributes of multiple sub-grids rather than by the attributes of one target grid. In this way, in the constructed map, the detailed information of the area corresponding to the target grid is richer, and for non-target grids, it is not necessary to divide them. In this way, when applying the map construction scheme provided by the embodiment of the present invention, the accuracy of the constructed map can be improved, and the data storage volume of the map can be guaranteed to be small.

[0211] In one embodiment of the present invention, see Figure 7 , a schematic diagram of the structure of a second map construction device is provided, wherein the object segmentation module 601 comprises:

[0212] The feature extraction submodule 601A is used to extract features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map;

[0213] The semantic segmentation submodule 601B is used to perform semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects;

[0214] The instance segmentation submodule 601C performs instance segmentation on the initial feature map to obtain a second feature map describing different objects;

[0215] The region determination submodule 601D is used to determine the image region where each object is located in the environment image according to the obtained first feature map and the second feature map.

[0216] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to construct a map, semantic segmentation and instance segmentation are performed on the extracted initial feature map, so that the first feature map describing different types of objects and the second feature map describing different objects can be accurately determined. According to the first feature map, the type of object corresponding to each pixel in the environment image can be known, and according to the second feature map, the distribution of objects corresponding to each pixel in the environment image can be known. In this way, according to the first feature map and the second feature map, the image area where each object is located can be accurately determined, and according to the image area, the environment area and the target grid can be accurately determined, so that after updating the target grid, the accuracy of the constructed map can be improved.

[0217] In one embodiment of the present invention, the region determination submodule 601D is specifically configured to:

[0218] Determining, according to the first feature map, areas where objects of different types are located in the environment image;

[0219] Determining object pixels describing different objects in the environment image according to the second feature map;

[0220] Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel in the area where each type of object is located, the object pixel closest to the regional pixel is determined based on the coordinates of the object pixel after offset;

[0221] The region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located is determined as the image region where the type of object is located.

[0222] It can be seen from the above that when the solution provided by the embodiment of the present invention is used to construct a map, the coordinates of each object pixel after offset are calculated based on the preset position offset parameters. It can be considered that the position of the object pixel is corrected before determining the image area. The accuracy of the object description by the offset object pixel is higher than that of the original object pixel. In this way, based on the regional pixel points and the object pixel points, the accuracy of the image area where each object is located can be further improved, thereby further improving the accuracy of the constructed map.

[0223] In one embodiment of the present invention, the grid updating module 605 is specifically used for:

[0224] Based on the division coefficient and the area of ​​the environment area, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

[0225] From the above, it can be seen that in the map construction scheme provided by the embodiment of the present invention, when dividing the target grid, two factors, namely the division coefficient and the area of ​​the environmental area, are taken into consideration, so that the target grid division is more reasonable and accurate. In this way, after using multiple sub-grids to update the target grid, the accuracy of the map can be improved.

[0226] In one embodiment of the present invention, the grid updating module 605 is specifically used for:

[0227] Calculating the size of the sub-grid based on the corresponding relationship between the division coefficient and the size of the sub-grid;

[0228] The target grid is divided into a plurality of sub-grids according to the calculated size, and the target grid is updated using the plurality of sub-grids.

[0229] From the above, it can be seen that when a map is constructed using the solution provided by an embodiment of the present invention, after obtaining the division coefficient, the size of the sub-grid can be accurately calculated based on the correspondence between the division coefficient and the size of the sub-grid, so that the target grid can be accurately divided into multiple sub-grids according to the calculated size, so that after the target grid is updated using the multiple sub-grids, the accuracy of the map can be improved.

[0230] In one embodiment of the present invention, when all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid is expressed by the following expression:

[0231]

[0232] Wherein, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

[0233] In this solution, through the above expression, the size of the sub-grid can be accurately calculated according to the division coefficient, so as to accurately divide the target grid and improve the accuracy of map construction.

[0234] In addition, when all grids are regular hexagonal grids, the distance between each grid and the other six adjacent grids is the same. In this way, when the robot performs path planning based on the grid, the complexity of the robot's path planning can be reduced, thereby helping the robot to perform tasks.

[0235] The embodiment of the present invention further provides an electronic device, such as Figure 8As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0236] Memory 803, used for storing computer programs;

[0237] The processor 801 is used to execute the program stored in the memory 803, and implements the following steps:

[0238] Performing object segmentation on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located;

[0239] Determining an environmental area corresponding to the image area according to the environmental position of the robot;

[0240] In an initial environment map constructed based on a grid, a target grid where the environment area is located is searched, wherein the attributes of the grid are used to describe the environment information of the area corresponding to the grid in the environment;

[0241] Obtaining a division coefficient of the environmental area according to the degree of influence of the object on the robot's execution of a task as reflected by the image area;

[0242] Based on the division coefficient, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

[0243] The processor 801 executes the program stored in the memory 803 to implement other solutions for map construction, which are the same as those mentioned in the above method embodiments and will not be described in detail here.

[0244] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0245] The communication interface is used for communication between the above electronic device and other devices.

[0246] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0247] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0248] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned map construction methods are implemented.

[0249] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any map construction method in the above embodiments.

[0250] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0251] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0252] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0253] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A map construction method, characterized in that: The method comprises: Performing object segmentation on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located; Determining an environmental area corresponding to the image area according to the environmental position of the robot; In an initial environment map constructed based on a grid, a target grid where the environment area is located is searched, wherein the attributes of the grid are used to describe the environment information of the area corresponding to the grid in the environment, and the environment information includes the type and location of the object in the area and the location of the area; Obtaining a division coefficient of the environmental area according to the degree of influence of the object on the robot's execution of a task as reflected by the image area; Based on the division coefficient, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids, wherein the attributes of each sub-grid are used to describe the environmental information of the corresponding partial area.

2. The method according to claim 1, characterized in that The object segmentation is performed on the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located, including: Extract features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map; Performing semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects; Performing instance segmentation on the initial feature map to obtain a second feature map describing different objects; According to the obtained first feature map and the second feature map, an image region where each object is located is determined in the environment image.

3. The method according to claim 2, characterized in that The step of determining the image area where each object is located in the environment image according to the obtained first feature map and the second feature map includes: Determining, according to the first feature map, areas where objects of different types are located in the environment image; Determining object pixels describing different objects in the environment image according to the second feature map; Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel in the area where each type of object is located, the object pixel closest to the regional pixel is determined based on the coordinates of the object pixel after offset; The region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located is determined as the image region where the type of object is located.

4. The method according to any one of claims 1 to 3, characterized in that The step of dividing the target grid into a plurality of sub-grids based on the division coefficient comprises: The target grid is divided into a plurality of sub-grids based on the division coefficient and the area of ​​the environmental region.

5. The method according to any one of claims 1 to 3, characterized in that: The step of dividing the target grid into a plurality of sub-grids based on the division coefficient comprises: Calculating the size of the sub-grid based on the corresponding relationship between the division coefficient and the size of the sub-grid; The target grid is divided into multiple sub-grids according to the calculated size.

6. The method according to claim 5, characterized in that In the case where all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid is expressed by the following expression: Wherein, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

7. A map construction device, characterized in that: The device comprises: An object segmentation module is used to segment the environment image acquired by the image acquisition device carried by the robot to obtain the image area where the object is located; An area determination module, used to determine the environmental area corresponding to the image area according to the environmental position of the robot; A grid search module is used to search for a target grid where the environmental area is located in an initial environmental map constructed based on grids, wherein the attributes of the grid are used to describe the environmental information of the area corresponding to the grid in the environment, and the environmental information includes the type and location of the object in the area and the location of the area; A coefficient obtaining module, used for obtaining the division coefficient of the environmental area according to the influence degree of the object on the robot's execution of the task reflected by the image area; A grid update module is used to divide the target grid into multiple sub-grids based on the division coefficient, and use the multiple sub-grids to update the target grid, wherein the attributes of each sub-grid are used to describe the environmental information of the corresponding partial area.

8. The device according to claim 7, characterized in that The object segmentation module comprises: The feature extraction submodule is used to extract features from the environment image acquired by the image acquisition device carried by the robot to obtain an initial feature map; A semantic segmentation submodule, used to perform semantic segmentation on the initial feature map to obtain a first feature map describing different types of objects; An instance segmentation submodule, used to perform instance segmentation on the initial feature map to obtain a second feature map describing different objects; The region determination submodule is used to determine the image region where each object is located in the environment image according to the obtained first feature map and the second feature map.

9. The device according to claim 8, characterized in that The region determination submodule is specifically used for: Determining, according to the first feature map, areas where objects of different types are located in the environment image; Determining object pixels describing different objects in the environment image according to the second feature map; Based on the preset position offset parameters, the coordinates of each object pixel after offset are calculated, and for the regional pixel in the area where each type of object is located, the object pixel closest to the regional pixel is determined based on the coordinates of the object pixel after offset; The region where multiple regional pixel points corresponding to the same object pixel point and describing the same type of object are located is determined as the image region where the type of object is located.

10. The device according to any one of claims 7 to 9, characterized in that: The grid updating module is specifically used for: Based on the division coefficient and the area of ​​the environment area, the target grid is divided into a plurality of sub-grids, and the target grid is updated using the plurality of sub-grids.

11. The device according to any one of claims 7 to 9, characterized in that: The grid updating module is specifically used for: Calculating the size of the sub-grid based on the corresponding relationship between the division coefficient and the size of the sub-grid; The target grid is divided into a plurality of sub-grids according to the calculated size, and the target grid is updated using the plurality of sub-grids.

12. The device according to claim 11, characterized in that In the case where all grids are regular hexagonal grids, the corresponding relationship between the division coefficient and the size of the sub-grid is expressed by the following expression: Wherein, S is the area of ​​the initial environment map, L is the side length of the sub-grid, and n is the division coefficient.

13. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the steps of any method described in claims 1-6 when executing a program stored in a memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the methods of claims 1-6 are implemented.

Images