Method and device for constructing sparse semantic map, equipment and storage medium

Abstract

The present application relates to the technical field of unmanned map construction, and discloses a sparse semantic map construction method, device, equipment and storage medium, which is used for constructing a sparse semantic map, reducing the consumption of computing resources, and improving the generation efficiency of the semantic map. The sparse semantic map construction method comprises the following steps: acquiring multiple frames of image data and vehicle pose information corresponding to each frame of image data, wherein each frame of image data comprises multiple pixel points; performing semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data; constructing an initial semantic map according to the semantic contour of the initial frame of image data and the vehicle pose information; and updating the initial semantic map according to the subsequent frame of image data and the vehicle pose information corresponding to each frame of image data to obtain the sparse semantic map.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving map construction technology, and in particular to a method, apparatus, device and storage medium for constructing sparse semantic maps. Background Technology

[0002] Map building is one of the core tasks to be solved in the development of autonomous driving. Autonomous vehicles acquire image data through sensors such as cameras, perform semantic segmentation on the image data using deep learning methods, distinguish objects in the image through semantic labels, and generate a map with semantic labels to achieve the purpose of image scene segmentation and recognition.

[0003] After semantic segmentation, each pixel in the existing image data carries a semantic label. The objects in the image are composed of a dense semantic region. As the map area expands, this dense semantic block will consume a lot of computing resources, making the construction of semantic maps inefficient and difficult to meet the timeliness requirements of autonomous vehicles. Moreover, there are a lot of segmentation errors in semantic segmentation, which will lead to a lot of incorrect labels in the semantic map. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for constructing sparse semantic maps, which reduces computational resource consumption and improves the generation efficiency of semantic maps.

[0005] The first aspect of this invention provides a method for constructing a sparse semantic map, comprising: acquiring multiple frames of image data and vehicle pose information corresponding to each frame of image data, wherein each frame of image data includes multiple pixels, and the multiple frames of image data include initial frame image data and subsequent frame image data; performing semantic segmentation and contour extraction on each frame of image data to obtain a semantic contour of each frame of image data, wherein each frame of image data has at least one semantic contour, and each semantic contour corresponds to a semantic label; constructing an initial semantic map based on the semantic contour of the initial frame image data and the vehicle pose information; and updating the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame of image data to obtain a sparse semantic map.

[0006] In one feasible implementation, semantic segmentation and contour extraction are performed on each frame of image data to obtain the semantic contour of each frame of image data, including: performing semantic segmentation on each frame of image data to obtain initial image data, the initial image data including multiple pixels and semantic labels carried by each pixel; and performing contour extraction on multiple pixels according to the semantic labels and a preset contour extraction algorithm to obtain the semantic contour of each frame of image data, each semantic contour containing multiple semantic contour points with the same semantic label.

[0007] In one feasible implementation, an initial semantic map is constructed based on the semantic contours and vehicle pose information of the initial frame image data. Constructing the initial semantic map includes: parsing the semantic contours of the initial frame image data to obtain semantic contour point information, which includes pixel coordinates, pixel depth, and semantic labels; calculating the world coordinates of each semantic contour point based on the pixel coordinates, pixel depth, and vehicle pose information corresponding to each frame of image data; calculating the grid index of each semantic contour point based on its world coordinates and a preset grid side length to obtain a three-dimensional hash table; and generating the initial semantic map based on the three-dimensional hash table and the semantic labels of each semantic contour.

[0008] In one feasible implementation, the world coordinates of each semantic contour point are calculated based on pixel coordinates, pixel depth, and vehicle pose information corresponding to each frame of image data. This includes: calculating the three-dimensional coordinates of each semantic contour point based on pixel coordinates, pixel depth, a preset camera intrinsic parameter matrix, and a preset vehicle coordinate system extrinsic parameter matrix; and projecting the three-dimensional coordinates of each semantic contour point onto the world coordinate system based on the three-dimensional coordinates of each semantic contour point and the vehicle pose information corresponding to each frame of image data to obtain the world coordinates of each semantic contour point.

[0009] In one feasible implementation, an initial semantic map is generated based on a three-dimensional hash table and the semantic label of each semantic contour, including: parsing the three-dimensional hash table to obtain grid indices for multiple semantic contour points; accessing the world coordinate system based on the grid indices of the multiple semantic contour points, and creating a corresponding voxel grid when any voxel grid is accessed; saving the semantic labels carried by the semantic contour points as semantic labels for the corresponding voxel grids; initializing and saving the probabilities of the voxel grids to obtain initial probabilities; and outputting the voxel grids, the semantic labels of the voxel grids, and the initial probabilities as the initial semantic map.

[0010] In one feasible implementation, the initial semantic map is updated based on subsequent frame image data and vehicle pose information corresponding to each frame image data to obtain a sparse semantic map. This includes: acquiring subsequent frame image data, where the subsequent frame image data includes at least a first subsequent frame image data; projecting the semantic contours in the first subsequent frame image data onto the initial semantic map based on the vehicle pose information corresponding to each frame image data; when any unaccessed voxel grid is accessed, creating the corresponding voxel grid and saving the semantic label and initial probability of the voxel grid; when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, deleting the corresponding voxel grid; and updating and iterating based on the subsequent frame image data to obtain the sparse semantic map.

[0011] In one feasible implementation, when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, after deleting the corresponding voxel grid, before updating and iterating according to the subsequent frame image data to obtain the sparse semantic map, the method further includes: acquiring the initial frame image data, which includes first semantic contour points carrying first semantic labels, and the first semantic contour points corresponding to first voxel grids in the initial semantic map; when the first subsequent frame image data includes first subsequent semantic contour points carrying first semantic labels, and the grid index of the first subsequent semantic contour points accesses the first voxel grid, increasing the probability of the first semantic label in the first voxel grid to obtain the updated probability of the first voxel grid; and updating each semantic contour point in the first subsequent frame image data to the initial semantic map to obtain the first semantic map.

[0012] In one feasible implementation, after updating each semantic contour point in the first subsequent frame image data to the initial semantic map to obtain the first semantic map, the method further includes: processing the first semantic map according to a preset self-attenuation rule to output a second semantic map.

[0013] In one feasible implementation, the first semantic map is processed according to a preset self-decay rule to output a second semantic map, including: obtaining the first semantic map, which includes multiple voxel grids and the probability of each voxel grid; traversing the multiple voxel grids of the first semantic map according to the preset self-decay rule; when the probability of any voxel grid is less than a preset minimum threshold, deleting the corresponding voxel grid and outputting the second semantic map.

[0014] A second aspect of the present invention provides an apparatus for constructing a sparse semantic map, comprising: an acquisition module for acquiring multiple frames of image data and vehicle pose information corresponding to each frame of image data, wherein each frame of image data includes multiple pixels and the multiple frames of image data include initial frame image data and subsequent frame image data; a processing module for performing semantic segmentation and contour extraction on each frame of image data to obtain a semantic contour of each frame of image data, wherein each frame of image data has at least one semantic contour and each semantic contour corresponds to a semantic label; an initial map module for constructing an initial semantic map based on the semantic contour of the initial frame image data and the vehicle pose information; and an update module for updating the initial semantic map based on subsequent frame image data and the vehicle pose information corresponding to each frame of image data to obtain a sparse semantic map.

[0015] In one feasible implementation, the processing module is specifically used to: perform semantic segmentation on each frame of image data to obtain initial image data, the initial image data including multiple pixels and semantic labels carried by each pixel; and extract contours from multiple pixels according to the semantic labels and a preset contour extraction algorithm to obtain the semantic contours of each frame of image data, each semantic contour containing multiple semantic contour points with the same semantic labels.

[0016] In one feasible implementation, the initial map module includes: a processing unit for parsing the semantic contours of the initial frame image data to obtain semantic contour point information, the semantic contour point information including pixel coordinates, pixel depth, and semantic labels; a coordinate transformation unit for calculating the world coordinates of each semantic contour point based on the pixel coordinates, pixel depth, and vehicle pose information corresponding to each frame image data; a calculation unit for calculating the grid index of each semantic contour point based on the world coordinates of each semantic contour point and a preset grid side length to obtain a three-dimensional hash table; and a map generation unit for generating an initial semantic map based on the three-dimensional hash table and the semantic labels of each semantic contour.

[0017] In one feasible implementation, the coordinate transformation unit is specifically used to: calculate the three-dimensional coordinates of each semantic contour point based on the pixel coordinates, pixel depth, preset camera intrinsic parameter matrix, and preset vehicle coordinate system extrinsic parameter matrix; and project the three-dimensional coordinates onto the world coordinate system based on the three-dimensional coordinates of each semantic contour point and the vehicle pose information corresponding to each frame of image data to obtain the world coordinates of each semantic contour point.

[0018] In one feasible implementation, the map generation unit is specifically used for: parsing a three-dimensional hash table to obtain grid indices for multiple semantic contour points; accessing the world coordinate system based on the grid indices of the multiple semantic contour points, and creating a corresponding voxel grid when any voxel grid is accessed; saving the semantic labels carried by the semantic contour points as semantic labels for the corresponding voxel grids; initializing and saving the probabilities of the voxel grids to obtain initial probabilities; and outputting the voxel grids, the semantic labels of the voxel grids, and the initial probabilities as an initial semantic map.

[0019] In one feasible implementation, the update module includes: a first acquisition unit, configured to acquire subsequent frame image data, wherein the subsequent frame image data includes at least first subsequent frame image data; an access unit, configured to project the semantic contours in the first subsequent frame image data onto an initial semantic map based on the vehicle pose information corresponding to each frame image data; a mesh creation unit, configured to create a corresponding voxel mesh and save the semantic label and initial probability of the voxel mesh when any unaccessed voxel mesh is accessed; an error correction unit, configured to delete the corresponding voxel mesh when any voxel mesh in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data; and an iteration unit, configured to perform update iteration based on the subsequent frame image data to obtain a sparse semantic map.

[0020] In one feasible implementation, the sparse semantic map construction apparatus further includes: a second acquisition unit, configured to acquire initial frame image data, the initial frame image data including first semantic contour points carrying first semantic labels, the first semantic contour points corresponding to a first voxel grid in the initial semantic map; a reward unit, configured to increase the probability of the first semantic label in the first voxel grid when the first subsequent frame image data includes first subsequent semantic contour points carrying first semantic labels, and the grid index of the first subsequent semantic contour points accesses the first voxel grid, thereby obtaining the updated probability of the first voxel grid; and an update unit, configured to update each semantic contour point in the first subsequent frame image data to the initial semantic map, thereby obtaining the first semantic map.

[0021] In one feasible implementation, the sparse semantic map construction apparatus further includes: a self-decay unit for processing the first semantic map according to a preset self-decay rule and outputting a second semantic map.

[0022] In one feasible implementation, the self-decay unit is specifically used to acquire a first semantic map, which includes multiple voxel grids and the probability of each voxel grid; according to a preset self-decay rule, the multiple voxel grids of the first semantic map are traversed; when the probability of any voxel grid is less than a preset minimum threshold, the corresponding voxel grid is deleted, and a second semantic map is output.

[0023] A third aspect of the present invention provides a sparse semantic map construction apparatus, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the sparse semantic map construction apparatus to execute the sparse semantic map construction method described above.

[0024] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions which, when executed on a computer, cause the computer to perform the above-described method for constructing a sparse semantic map.

[0025] In the technical solution provided by this invention, semantic contours are used instead of dense semantic blocks for subsequent calculations to construct a semantic map. Only semantic contour points need to be calculated to complete the construction and updating of the semantic map, which can greatly reduce the consumption of computing resources and accelerate the construction efficiency of the semantic map. Based on the reward rule, the observed points can be kept in the semantic map for a long time. Based on the penalty rule, the points with semantic segmentation errors in the semantic map can be processed quickly. Based on the self-decay rule, the pixels that move out of the field of view of the image acquisition device as the vehicle moves can be slowly deleted from the semantic map, maintaining the semantic map at a small level, thus improving the speed and accuracy of semantic map construction. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of an embodiment of the method for constructing a sparse semantic map in this invention.

[0027] Figure 2 This is a schematic diagram of another embodiment of the sparse semantic map construction method in this invention;

[0028] Figure 3 This is a schematic diagram of one embodiment of the sparse semantic map construction device in this invention.

[0029] Figure 4 This is a schematic diagram of another embodiment of the sparse semantic map construction device in this invention;

[0030] Figure 5 This is a schematic diagram of an embodiment of a sparse semantic map construction device according to an embodiment of the present invention. Detailed Implementation

[0031] This invention provides a method, apparatus, device, and storage medium for constructing sparse semantic maps, which reduces computational resource consumption and improves the generation efficiency of semantic maps.

[0032] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the sparse semantic map construction method in this invention includes:

[0034] 101. Obtain multi-frame image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels, and the multi-frame image data includes initial frame image data and subsequent frame image data.

[0035] It is understood that the executing entity of this invention can be a sparse semantic map construction device, a vehicle terminal, or other map construction terminals with receiving and processing functions, or a server. When a server or other map construction terminal is used as the executing entity, the sparse semantic map construction method provided by this invention is executed by remotely acquiring image data and pose information. The specific executing device is not limited. This embodiment of the invention uses a vehicle terminal as an example for illustration.

[0036] The vehicle terminal acquires multiple frames of image data and the vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels. The image data can be image data collected by image sensors such as cameras, point cloud data collected by lidar sensors, or image data collected by other image sensors. This image data can show the external environment during the autonomous vehicle's driving process. The specific method of acquiring the image data is not limited here.

[0037] In this embodiment, the multi-frame image data includes initial frame image data and subsequent frame image data. The initial frame image data is used to construct the initial semantic map, which generally refers to the first frame image data acquired when the vehicle terminal starts up, or other image data that meets the requirements. The subsequent frame image data is used to update the initial semantic map. Its acquisition time is after the initial frame image data. According to the preset acquisition time interval, the external environment is collected by the corresponding sensors during the operation of the autonomous vehicle until the vehicle terminal stops running. It includes the first subsequent frame image data, the second subsequent frame image data, ..., the Nth subsequent frame image data (N is a positive integer).

[0038] In this embodiment, vehicle pose information refers to the vehicle's position and posture information. Position information can be obtained directly from the Global Positioning System (GPS) on the vehicle terminal, or from the odometer on the vehicle terminal, or from a Simultaneous Localization and Mapping (SLAM) system, or through other methods; the specific method is not limited. Posture information can be obtained directly from the inertial sensors on the vehicle terminal, or calculated by fusing visual information from multiple sensors, or estimated based on the sparse semantic map constructed in this application, or through other methods, which can be selected according to actual needs.

[0039] In one feasible implementation, the vehicle terminal preprocesses multi-frame image data. The preprocessing includes at least pixel brightness transformation, geometric transformation, local image smoothing, and image enhancement to enhance the detectability of relevant information in the image data and simplify the data to the maximum extent, thereby improving the accuracy of subsequent feature extraction and semantic segmentation.

[0040] 102. Perform semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data. There is at least one semantic contour for each frame of image data, and each semantic contour corresponds to a semantic label.

[0041] The vehicle terminal can use a deep learning algorithm to perform semantic segmentation on each frame of image data to obtain processed initial image data. Each pixel in the initial image data carries a corresponding semantic label. Based on the semantic label and the contour extraction algorithm, semantic contours are extracted from each frame of image data. In this embodiment, a semantic contour consists of a semantic label and a series of pixels. The semantic contour can be used to indicate objects in the external environment during vehicle driving. The pixels on the semantic contour are called semantic contour points, and the semantic contour points on a semantic contour carry the same semantic label.

[0042] In one feasible implementation, after obtaining initial image data through semantic segmentation, the vehicle terminal calculates the semantic gradient g(x,y) of each pixel in the camera coordinate system based on the semantic label I(x,y), where I(x,y) represents the semantic label at pixel position (x,y). Based on the semantic gradient and corresponding semantic label of each pixel, a semantic contour is obtained. The formula for calculating the semantic gradient at a given position is as follows:

[0043]

[0044] When g(x,y)! = 0, the position is the semantic contour, and the pixel point corresponding to the position is the semantic contour point.

[0045] 103. Construct an initial semantic map based on the semantic contours and vehicle pose information of the initial frame image data.

[0046] The vehicle terminal constructs an initial semantic map based on the initial frame image data. First, using the vehicle pose information, the semantic contours in the initial frame image data are projected from the camera coordinate system to the world coordinate system to obtain the world coordinates of each semantic contour point. A hash table is constructed based on the world coordinates of each semantic contour point and the preset grid side length. The hash table contains grid indices that can uniquely identify the corresponding semantic contour point. The world coordinate system is accessed based on the grid indices in the hash table to create a voxel grid. Semantic contours composed of voxel grids can be constructed, and then an initial semantic map is formed by stitching together multiple semantic contours. This initial semantic map is composed of sparse semantic contour points. Compared with traditional semantic maps composed of dense semantic blocks, it can greatly reduce the consumption of computing resources and improve the generation efficiency of semantic maps.

[0047] It should be further explained that the preset grid side length mentioned above can be set according to the actual situation. It can be that a voxel grid contains one pixel (semantic contour point) to determine the corresponding grid side length, or it can contain multiple semantic contour points carrying the same semantic label to determine the corresponding grid side length. Since the distribution of pixels in the actual acquired image data is not uniform, in one feasible implementation, the semantic contour points can first be calculated according to the first side length to obtain the first grid index. When the semantic contour accesses the voxel grid in the world coordinate system through the first grid index, a corresponding voxel grid with the first side length is created. The first side length voxel grid can contain multiple semantic contour points carrying the same semantic label. This first side length should not be set too large, as an excessively large first side length will result in an overly coarse semantic map, while an excessively small first side length will lead to a waste of computational resources.

[0048] When any voxel grid with a first side length is accessed by a semantic contour point carrying a different semantic label, the first side length voxel grid is deleted to obtain the corresponding voxel grid. The semantic contour points carrying different semantic labels include at least a first semantic contour point carrying a first semantic label and a second semantic contour point carrying a second semantic label. The second grid index is calculated based on the world coordinates of the first and second semantic contour points according to the preset second side length. The second side length ensures that the semantic contour points carrying different semantic labels are located in different voxel grids. The world coordinates are accessed according to the second grid index corresponding to the first and second semantic contour points, respectively, and the corresponding second side length voxel grids are created. The probabilities corresponding to the semantic labels are initialized and saved. The output of the first and second side length voxel grids is used as the initial semantic map. The initial semantic map obtained by the above method can save computing resources and ensure the efficiency of quickly building the semantic map while taking into account the richness of the semantic map details.

[0049] 104. Update the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data to obtain a sparse semantic map.

[0050] The vehicle terminal projects the semantic contours in subsequent frame image data onto the world coordinate system and updates the initial semantic map according to preset update rules to obtain a sparse semantic map. In this embodiment, the update rules include reward rules, penalty rules, and self-decay rules. The semantic map constructed by the previous frame image data (such as the first subsequent frame image data) is updated according to the reward rules and penalty rules to obtain the first semantic map. After the voxel grid of the previous frame image data is updated, the first semantic map is updated according to the self-decay rules to obtain a sparse semantic map.

[0051] In one feasible implementation, the reward rule includes creating a voxel grid when any voxel grid is accessed, and storing the corresponding semantic label and initial probability P0 in the voxel grid, where P0 is the probability that the voxel grid is classified as the corresponding semantic label when it is accessed for the first time; the reward rule also includes rewarding the probability of the voxel grid when a semantic contour point carrying the same semantic label as the initial frame image data accesses the same voxel grid in the first subsequent frame image data, where the rewarded probability P is... n If the probability of the corresponding semantic label of the voxel grid is greater than P0 (where n represents the nth time the voxel grid is accessed), then to prevent the probability of the corresponding semantic label of the voxel grid from being too high, P... n It should be less than the maximum threshold δ maxThe penalty rule is that if a semantic contour point in the first subsequent frame image data carries a semantic label different from that in the initial frame image data, and accesses the same voxel grid, the semantic segmentation of that semantic contour point is judged to be incorrect, and the voxel grid, along with the semantic label and probability stored therein, is deleted. By traversing all semantic contour points in the first subsequent frame image data through the above reward and penalty rules, and updating the voxel grid corresponding to the initial frame image data, a first semantic map is obtained. The first semantic map contains multiple voxel grids, each storing a corresponding probability P', where P' refers to the probability after each update. Each voxel grid in the first semantic map is processed using a self-decay rule; when P' is less than a preset minimum threshold δ... min When the corresponding voxel grid is deleted, the second semantic map is output to maintain the sparsity of the semantic map. The second subsequent frame image data updates the second semantic map in sequence according to the above reward rule, penalty rule and self-decay rule. The subsequent frame image data updates the previous frame image data in sequence, so as to obtain the real-time updated sparse semantic map.

[0052] In one feasible implementation, the reward rule further includes: when either the first-side-length voxel grid or the second-side-length voxel grid includes multiple semantic contour points carrying the same semantic label, the probability of the first-side-length voxel grid... This is the sum of the initial probabilities of multiple semantic contour points within the grid. When a semantic contour point carrying a different semantic label is accessed in the first-side-length voxel grid, the penalty rule includes deleting the first-side-length voxel grid, constructing a second-side-length voxel grid according to the above method, and initializing the probability of each semantic contour point in the second-side-length voxel grid. The initial probability of multiple semantic contour points within the grid is the sum of the initial probabilities. Using the aforementioned reward and penalty rules, all semantic contour points in the first and subsequent frame image data are traversed. After updating the voxel grid corresponding to the initial frame image data, a third semantic map is obtained. The third semantic map contains multiple voxel grids, including voxel grids with first and second side lengths. Each voxel grid stores a corresponding probability P', which is the probability that the voxel grid in the third semantic map is classified as the corresponding semantic label. Each voxel grid in the third semantic map is processed using a self-decay rule. When P' is less than a preset minimum threshold δ... min At that time, the corresponding voxel grid is deleted, and the fourth semantic map is output to maintain the sparsity of the semantic map; where the initial probability of the second-side-length voxel grid is... Minimum threshold corresponding to the self-decay rule It can be set to the initial value corresponding to the voxel mesh with a side length greater than the first side length. The minimum corresponding to the self-decay rule This ensures that the details of the semantic map are preserved more completely during subsequent updates. The second and subsequent frame image data are then used to update the fourth semantic map sequentially according to the aforementioned reward rule, penalty rule, and self-decay rule. Each subsequent frame image data updates the previous frame image data in turn, thus obtaining a real-time updated sparse semantic map.

[0053] In this embodiment of the invention, semantic contours are used instead of dense semantic blocks for subsequent calculations to construct a semantic map. Only semantic contour points need to be calculated to complete the construction and updating of the semantic map, which can greatly reduce the consumption of computing resources and accelerate the construction efficiency of the semantic map. Based on the reward rule, the observed points can be kept in the semantic map for a long time. Based on the penalty rule, the points with semantic segmentation errors in the semantic map can be processed quickly. Based on the self-decay rule, the pixels that move out of the field of view of the image acquisition device as the vehicle moves can be slowly deleted from the semantic map, maintaining the semantic map at a small level, thus improving the speed and accuracy of semantic map construction.

[0054] Please see Figure 2 Another embodiment of the sparse semantic map construction method in this invention includes:

[0055] 201. Obtain multi-frame image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels, and the multi-frame image data includes initial frame image data and subsequent frame image data.

[0056] Step 201 is similar to step 101 above, and will not be repeated here.

[0057] 202. Perform semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data. There is at least one semantic contour for each frame of image data, and each semantic contour corresponds to a semantic label.

[0058] The vehicle terminal performs semantic segmentation on each frame of image data to obtain initial image data, which includes multiple pixels and semantic labels carried by each pixel. Based on the semantic labels and a preset contour extraction algorithm, contour extraction is performed on the multiple pixels to obtain the semantic contour of each frame of image data. Each semantic contour contains multiple semantic contour points with the same semantic label.

[0059] 203. Construct an initial semantic map based on the semantic contours and vehicle pose information of the initial frame image data.

[0060] In one feasible implementation, the vehicle terminal parses the semantic contours of the initial frame image data to obtain semantic contour point information, which includes pixel coordinates, pixel depth, and semantic labels. Based on the pixel coordinates, pixel depth, and vehicle pose information corresponding to each frame image data, the world coordinates of each semantic contour point are calculated. Based on the world coordinates of each semantic contour point and the preset grid side length, the grid index of each semantic contour point is calculated to obtain a three-dimensional hash table. Based on the three-dimensional hash table and the semantic labels of each semantic contour, an initial semantic map is generated.

[0061] In one feasible implementation, the vehicle terminal calculates the three-dimensional coordinates of each semantic contour point based on pixel coordinates, pixel depth, a preset camera intrinsic parameter matrix, and a preset vehicle coordinate system extrinsic parameter matrix; based on the three-dimensional coordinates of each semantic contour point and the vehicle pose information corresponding to each frame of image data, the three-dimensional coordinates are projected onto the world coordinate system to obtain the world coordinates of each semantic contour point.

[0062] In one feasible implementation, the vehicle terminal parses a 3D hash table to obtain grid indices for multiple semantic contour points; accesses the world coordinate system based on the grid indices of the multiple semantic contour points; when any voxel grid is accessed, a corresponding voxel grid is created; the semantic labels carried by the semantic contour points are saved as semantic labels for the corresponding voxel grids; the probabilities of the voxel grids are initialized and saved to obtain initial probabilities; and the voxel grids, the semantic labels of the voxel grids, and the initial probabilities are output as an initial semantic map.

[0063] For example, suppose the semantic contour carrying the first semantic label consists of n semantic contour points, where the pixel coordinates of the i-th semantic contour point are p. i =(x i y i The depth corresponding to this pixel is d. i The pre-calibrated intrinsic parameter matrix of the camera is K, and the extrinsic parameter matrix from the camera to the vehicle coordinate system is T. bc Projecting the pixel coordinates of the semantic contour in the camera coordinate system onto the vehicle coordinate system yields the 3D coordinates p in the vehicle coordinate system. bi The formula is as follows:

[0064] p bi =T bc ×K -1 ×p i ×d i

[0065] The vehicle pose information corresponding to the initial frame image data is T. wb The three-dimensional coordinates p in the vehicle coordinate system bi Projecting onto the world coordinate system, we obtain the world coordinates p of the semantic contour point. wiThe formula is as follows:

[0066] p wi =T wb ×p bi

[0067] Assuming the preset grid side length is 'a', and based on the world coordinates p of this semantic contour point... wi Calculate the grid index id of semantic contour points. i The formula is as follows:

[0068]

[0069] 204. Update the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data to obtain a sparse semantic map.

[0070] In one feasible implementation, the vehicle terminal acquires subsequent frame image data, which includes at least first subsequent frame image data; the semantic contours in the first subsequent frame image data are projected onto an initial semantic map based on the vehicle pose information corresponding to each frame image data; when any unaccessed voxel grid is accessed, a corresponding voxel grid is created, and the semantic label and initial probability of the voxel grid are saved; when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, the corresponding voxel grid is deleted; and the map is updated and iterated based on the subsequent frame image data to obtain a sparse semantic map.

[0071] In one feasible implementation, the vehicle terminal acquires initial frame image data, which includes first semantic contour points carrying first semantic labels. The first semantic contour points correspond to a first voxel grid in the initial semantic map. When the first subsequent frame image data includes first subsequent semantic contour points carrying first semantic labels, and the grid index of the first subsequent semantic contour points accesses the first voxel grid, the probability of the first semantic label in the first voxel grid is increased to obtain the updated probability of the first voxel grid. Each semantic contour point in the first subsequent frame image data is updated to the initial semantic map to obtain the first semantic map.

[0072] For example, when any voxel grid is accessed, the probability P(id) of initializing that grid is... i ) = P0, where P0 represents the probability that the voxel grid is classified as the first semantic label at this observation.

[0073] When the first subsequent frame image data includes a first subsequent semantic contour point carrying a first semantic label, and the grid index of the first subsequent semantic contour point accesses the first voxel grid, a reward P(id) is given to the first voxel grid. i=P0×α, increase the probability of the first semantic label in the first voxel grid to obtain the updated probability of the first voxel grid. The probability after reward should be less than the maximum threshold δ. max When the same semantic label in subsequent frame image data accesses the same voxel grid, the reward function formula for the probability of that voxel grid is as follows:

[0074] P(id i )=min((P(id i )×α),δ max )

[0075] Where α is a reward coefficient greater than 1, δ max This is the maximum threshold. To prevent other labels from having excessively low probabilities, the probability of the first semantic label in the voxel grid should be less than δ. max Furthermore, in order to ensure that the axiom of total probability is satisfied, the probability of other semantic labels in the voxel grid should be reduced by the probability of the first semantic label being increased.

[0076] In one feasible implementation, the vehicle terminal processes the first semantic map according to a preset self-attenuation rule and outputs a second semantic map.

[0077] In one feasible implementation, the vehicle terminal acquires a first semantic map, which includes multiple voxel grids and the probability of each voxel grid; it then iterates through the multiple voxel grids of the first semantic map according to a preset self-decay rule, and when the probability of any voxel grid is less than a preset minimum threshold δ... min When the time is reached, the corresponding voxel grid is deleted, and a second semantic map is output, which is a sparse semantic map.

[0078] In one feasible implementation, the self-decay rule can also be: the vehicle terminal acquires a first semantic map, which includes multiple voxel grids and the probability of each voxel grid; the probability of each voxel grid in the first semantic map is subtracted from the self-decay probability P. a The probability of each voxel grid after self-decay is obtained. When the probability of any voxel grid after self-decay is less than the preset minimum threshold δ, min At that time, delete the corresponding voxel grid and output the second semantic map.

[0079] In this embodiment, the maximum and minimum thresholds can be set according to the actual situation. The semantic map corresponding to the previous frame image data can be updated by the subsequent frame image data. The time interval between the subsequent frame image data and the previous frame image data can be set according to the actual computing resources.

[0080] In this embodiment of the invention, semantic contours are used instead of dense semantic blocks for subsequent calculations to construct a semantic map. Only semantic contour points need to be calculated to complete the construction and updating of the semantic map, which can greatly reduce the consumption of computing resources and accelerate the construction efficiency of the semantic map. Based on the reward rule, the observed points can be kept in the semantic map for a long time. Based on the penalty rule, the points with semantic segmentation errors in the semantic map can be processed quickly. Based on the self-decay rule, the pixels that move out of the field of view of the image acquisition device as the vehicle moves can be slowly deleted from the semantic map, maintaining the semantic map at a small level, thus improving the speed and accuracy of semantic map construction.

[0081] The method for constructing sparse semantic maps in the embodiments of the present invention has been described above. The apparatus for constructing sparse semantic maps in the embodiments of the present invention is described below. Please refer to [link to relevant documentation]. Figure 3 One embodiment of the sparse semantic map construction apparatus of the present invention includes:

[0082] The acquisition module 301 is used to acquire multiple frames of image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels, and the multiple frames of image data include initial frame image data and subsequent frame image data.

[0083] Processing module 302 is used to perform semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data. Each frame of image data has at least one semantic contour, and each semantic contour corresponds to a semantic label.

[0084] The initial map module 303 is used to construct an initial semantic map based on the semantic contours and vehicle pose information of the initial frame image data;

[0085] The update module 304 is used to update the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data, so as to obtain a sparse semantic map.

[0086] In this embodiment of the invention, a sparse semantic map is constructed to reduce computational resource consumption and improve the generation efficiency of the semantic map.

[0087] Please see Figure 4 Another embodiment of the sparse semantic map construction apparatus in this invention includes:

[0088] The acquisition module 301 is used to acquire multiple frames of image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels, and the multiple frames of image data include initial frame image data and subsequent frame image data.

[0089] Processing module 302 is used to perform semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data. Each frame of image data has at least one semantic contour, and each semantic contour corresponds to a semantic label.

[0090] The initial map module 303 is used to construct an initial semantic map based on the semantic contours and vehicle pose information of the initial frame image data;

[0091] The update module 304 is used to update the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data, so as to obtain a sparse semantic map.

[0092] Optionally, the processing module 302 is specifically used to: perform semantic segmentation on each frame of image data to obtain initial image data, the initial image data including multiple pixels and semantic labels carried by each pixel; extract contours from multiple pixels according to the semantic labels and a preset contour extraction algorithm to obtain the semantic contours of each frame of image data, each semantic contour containing multiple semantic contour points with the same semantic labels.

[0093] Optionally, the initial map module 303 includes:

[0094] Processing unit 3031 is used to parse the semantic contour of the initial frame image data to obtain semantic contour point information, which includes pixel coordinates, pixel depth and semantic label.

[0095] The coordinate transformation unit 3032 is used to calculate the world coordinates of each semantic contour point based on the pixel coordinates, pixel depth and vehicle pose information corresponding to each frame of image data;

[0096] The computing unit 3033 is used to calculate the grid index of each semantic contour point based on the world coordinates of each semantic contour point and the preset grid side length, and obtain a three-dimensional hash table.

[0097] Map generation unit 3034 is used to generate an initial semantic map based on a three-dimensional hash table and the semantic label of each semantic contour.

[0098] Optionally, the coordinate transformation unit 3032 is specifically used to: calculate the three-dimensional coordinates of each semantic contour point based on the pixel coordinates, pixel depth, preset camera intrinsic parameter matrix and preset vehicle coordinate system extrinsic parameter matrix; and project the three-dimensional coordinates onto the world coordinate system based on the three-dimensional coordinates of each semantic contour point and the vehicle pose information corresponding to each frame of image data to obtain the world coordinates of each semantic contour point.

[0099] Optionally, the map generation unit 3034 is specifically used for: parsing a 3D hash table to obtain grid indices for multiple semantic contour points; accessing the world coordinate system based on the grid indices of multiple semantic contour points, and creating a corresponding voxel grid when any voxel grid is accessed; saving the semantic labels carried by the semantic contour points as semantic labels for the corresponding voxel grids; initializing and saving the probabilities of the voxel grids to obtain initial probabilities; and outputting the voxel grids, the semantic labels of the voxel grids, and the initial probabilities as an initial semantic map.

[0100] Optional, update module 304, including:

[0101] The first acquisition unit 3041 is used to acquire subsequent frame image data, wherein the subsequent frame image data includes at least the first subsequent frame image data.

[0102] Access unit 3042 is used to project the semantic contour in the first subsequent frame image data onto the initial semantic map according to the vehicle pose information corresponding to each frame image data.

[0103] Mesh creation unit 3043 creates the corresponding voxel mesh when any unvisited voxel mesh is accessed, and saves the semantic label and initial probability of the voxel mesh.

[0104] The error correction unit 3044 is used to delete the corresponding voxel grid when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data.

[0105] Iteration unit 3045 is used to update and iterate based on subsequent frame image data to obtain a sparse semantic map.

[0106] Optionally, the sparse semantic map construction apparatus, following the error correction unit 3044 and preceding the iteration unit 3045, further includes:

[0107] The second acquisition unit 3046 is used to acquire initial frame image data, which includes first semantic contour points carrying first semantic labels, and the first semantic contour points correspond to a first voxel grid in the initial semantic map.

[0108] The reward unit 3047 is used to increase the probability of the first semantic label in the first voxel grid when the first subsequent frame image data includes a first subsequent semantic contour point carrying a first semantic label, and the grid index of the first subsequent semantic contour point accesses the first voxel grid, so as to obtain the updated probability of the first voxel grid.

[0109] The update unit 3048 is used to update each semantic contour point in the first subsequent frame image data to the initial semantic map to obtain the first semantic map.

[0110] Optionally, the sparse semantic map construction apparatus further includes, after the update unit 3048, a self-decay unit 3049 for processing the first semantic map according to a preset self-decay rule and outputting a second semantic map.

[0111] The self-decay unit 3049 is specifically used to obtain a first semantic map, which includes multiple voxel grids and the probability of each voxel grid. According to the preset self-decay rule, the multiple voxel grids of the first semantic map are traversed. When the probability of any voxel grid is less than the preset minimum threshold, the corresponding voxel grid is deleted and the second semantic map is output.

[0112] In this embodiment of the invention, a sparse semantic map is constructed, which reduces the consumption of computing resources and improves the generation efficiency of the semantic map. Based on the reward rule, the observed points can be kept in the semantic map for a long time. Based on the penalty rule, the points with semantic segmentation errors in the semantic map can be processed quickly. Based on the self-decay rule, the pixels that move out of the field of view of the image acquisition device as the vehicle moves can be slowly deleted from the semantic map, maintaining the semantic map at a small level and improving the speed and accuracy of semantic map construction.

[0113] above Figure 3 and Figure 4 The sparse semantic map construction device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The sparse semantic map construction device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0114] Figure 5 This is a schematic diagram of a sparse semantic map construction device 500 provided in an embodiment of the present invention. The sparse semantic map construction device 500 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the sparse semantic map construction device 500. Furthermore, the processor 510 may be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the sparse semantic map construction device 500.

[0115] The sparse semantic map construction device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 5 The illustrated sparse semantic map construction device structure does not constitute a limitation on the sparse semantic map construction device, which may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0116] The present invention also provides a sparse semantic map construction device. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the sparse semantic map construction method in the above embodiments.

[0117] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a method for constructing a sparse semantic map.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0119] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a sparse semantic map, characterized in that, The method for constructing the sparse semantic map includes: Acquire multiple frames of image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels. The multiple frames of image data include initial frame image data and subsequent frame image data. Semantic segmentation and contour extraction are performed on each frame of image data to obtain the semantic contour of each frame of image data. Each frame of image data has at least one semantic contour, and each semantic contour corresponds to a semantic label. An initial semantic map is constructed based on the semantic contours of the initial frame image data and the vehicle pose information; The initial semantic map is updated based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data to obtain a sparse semantic map; The step of constructing an initial semantic map based on the semantic contours of the initial frame image data and the vehicle pose information includes: parsing the semantic contours of the initial frame image data to obtain semantic contour point information, wherein the semantic contour point information includes pixel coordinates, pixel depth, and semantic labels; calculating the world coordinates of each semantic contour point based on the pixel coordinates, the pixel depth, and the vehicle pose information corresponding to each frame image data; calculating the grid index of each semantic contour point based on the world coordinates of each semantic contour point and a preset grid side length to obtain a three-dimensional hash table; and generating the initial semantic map based on the three-dimensional hash table and the semantic labels of each semantic contour. The step of updating the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data to obtain a sparse semantic map includes: acquiring the subsequent frame image data, the subsequent frame image data including at least a first subsequent frame image data; projecting the semantic contours in the first subsequent frame image data onto the initial semantic map based on the vehicle pose information corresponding to each frame image data; when any unaccessed voxel grid is accessed, creating a corresponding voxel grid and saving the semantic label and initial probability of the voxel grid; when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, deleting the corresponding voxel grid; and updating and iterating based on the subsequent frame image data to obtain a sparse semantic map.

2. The method for constructing a sparse semantic map according to claim 1, characterized in that, The step of performing semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data includes: Semantic segmentation is performed on each frame of image data to obtain initial image data, which includes multiple pixels and semantic labels carried by each pixel; Based on the semantic label of each pixel, the semantic gradient of each pixel is calculated. Based on the semantic gradient of each pixel and the corresponding semantic label, the semantic contour of each frame of image data is obtained. Each semantic contour contains multiple semantic contour points with the same semantic label.

3. The method for constructing a sparse semantic map according to claim 1, characterized in that, The step of calculating the world coordinates of each semantic contour point based on the pixel coordinates, the pixel depth, and the vehicle pose information corresponding to each frame of image data includes: The three-dimensional coordinates of each semantic contour point are calculated based on the pixel coordinates, the pixel depth, the preset camera intrinsic parameter matrix, and the preset vehicle coordinate system extrinsic parameter matrix. Based on the three-dimensional coordinates of each semantic contour point and the vehicle pose information corresponding to each frame of image data, the three-dimensional coordinates are projected onto the world coordinate system to obtain the world coordinates of each semantic contour point.

4. The method for constructing a sparse semantic map according to claim 1, characterized in that, The step of generating an initial semantic map based on the three-dimensional hash table and the semantic labels of each semantic contour includes: Parse the three-dimensional hash table to obtain the grid index of multiple semantic contour points; Access the world coordinate system based on the grid index of the multiple semantic contour points, and create the corresponding voxel grid when any voxel grid is accessed; Save the semantic labels carried by the semantic contour points as the semantic labels of the corresponding voxel mesh. The probabilities of the voxel grid are initialized and saved to obtain the initial probabilities; The voxel grid, the semantic labels of the voxel grid, and the initial probabilities are output as an initial semantic map.

5. The method for constructing a sparse semantic map according to claim 1, characterized in that, After deleting the corresponding voxel grid when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, and before updating and iterating according to the subsequent frame image data to obtain the sparse semantic map, the method further includes: Acquire initial frame image data, which includes first semantic contour points carrying first semantic labels, and the first semantic contour points correspond to a first voxel grid in the initial semantic map. When the first subsequent frame image data includes a first subsequent semantic contour point carrying a first semantic label, and the grid index of the first subsequent semantic contour point accesses the first voxel grid, the probability of the first semantic label in the first voxel grid is increased to obtain the updated probability of the first voxel grid. Each semantic contour point in the first subsequent frame image data is updated to the initial semantic map to obtain the first semantic map.

6. The method for constructing a sparse semantic map according to claim 5, characterized in that, After updating each semantic contour point in the first subsequent frame image data to the initial semantic map to obtain the first semantic map, the method further includes: The first semantic map is processed according to a preset self-decay rule to output a second semantic map.

7. The method for constructing a sparse semantic map according to claim 6, characterized in that, The step of processing the first semantic map according to a preset self-decay rule and outputting a second semantic map includes: Obtain the first semantic map, which includes multiple voxel grids and the probability of each voxel grid; According to the preset self-decay rule, the first semantic map is traversed through multiple voxel grids. When the probability of any voxel grid is less than the preset minimum threshold, the corresponding voxel grid is deleted and the second semantic map is output.

8. A device for constructing a sparse semantic map, characterized in that, The apparatus for constructing the sparse semantic map includes: The acquisition module is used to acquire multiple frames of image data and vehicle pose information corresponding to each frame of image data. Each frame of image data includes multiple pixels, and the multiple frames of image data include initial frame image data and subsequent frame image data. The processing module is used to perform semantic segmentation and contour extraction on each frame of image data to obtain the semantic contour of each frame of image data. Each frame of image data has at least one semantic contour, and each semantic contour corresponds to a semantic label. An initial map module is used to construct an initial semantic map based on the semantic contours of the initial frame image data and the vehicle pose information; The update module is used to update the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data, so as to obtain a sparse semantic map. The step of constructing an initial semantic map based on the semantic contours of the initial frame image data and the vehicle pose information includes: parsing the semantic contours of the initial frame image data to obtain semantic contour point information, wherein the semantic contour point information includes pixel coordinates, pixel depth, and semantic labels; calculating the world coordinates of each semantic contour point based on the pixel coordinates, the pixel depth, and the vehicle pose information corresponding to each frame image data; calculating the grid index of each semantic contour point based on the world coordinates of each semantic contour point and a preset grid side length to obtain a three-dimensional hash table; and generating the initial semantic map based on the three-dimensional hash table and the semantic labels of each semantic contour. The step of updating the initial semantic map based on the subsequent frame image data and the vehicle pose information corresponding to each frame image data to obtain a sparse semantic map includes: acquiring the subsequent frame image data, the subsequent frame image data including at least a first subsequent frame image data; projecting the semantic contours in the first subsequent frame image data onto the initial semantic map based on the vehicle pose information corresponding to each frame image data; when any unaccessed voxel grid is accessed, creating a corresponding voxel grid and saving the semantic label and initial probability of the voxel grid; when any voxel grid in the initial semantic map is accessed by a semantic contour point carrying a different semantic label in the first subsequent frame image data, deleting the corresponding voxel grid; and updating and iterating based on the subsequent frame image data to obtain a sparse semantic map.

9. A device for constructing sparse semantic maps, characterized in that, The device for constructing the sparse semantic map includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the sparse semantic map construction device to perform the sparse semantic map construction method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, The instructions, when read and executed, perform the method for constructing a sparse semantic map as described in any one of claims 1-7.

Images