Semantic map construction method, device, equipment and storage medium

Abstract

The present application relates to a semantic map construction method, device, equipment and storage medium. The method includes: after the target vehicle starts to drive, sequentially obtaining each frame of depth scene image captured by a preset depth camera and generating a semantic depth scene image corresponding to each frame of depth scene image; calculating the camera pose corresponding to each frame of semantic depth scene image according to the parameters of the target vehicle; for each frame of semantic depth scene image, inputting the semantic depth scene image and its corresponding camera pose into a semantic map construction model; the semantic map construction model is a SLAM model based on 3D Gaussian splashing; constructing a semantic scene map based on the volume rendering image corresponding to each frame of semantic depth scene image output by the semantic map construction model. The present application constructs a semantic scene map, and uses a SLAM model based on 3D Gaussian splashing to project the semantic depth scene image into a volume rendering image. The semantic scene map not only integrates semantic information but also has high accuracy.

Description

Technical Field

[0001] The present application relates to the field of map construction technology, and in particular to a semantic map construction method, apparatus, device and storage medium. Background Art

[0002] Dense SLAM (Simultaneous Localization and Mapping) is a technology that has been widely studied in the fields of robotics and computer vision. Dense SLAM uses sensor data (such as cameras or lidar, etc.) to estimate the robot's position in real time and build a dense map of the surrounding environment. Unlike sparse SLAM, dense SLAM aims to generate more detailed and accurate maps that contain more geometric details of objects in the environment. Currently, the more popular dense SLAMs all use SLAM methods based on NERF (Neural Radiance Fields). However, the NERF algorithm uses a multi-layer perceptron network architecture. This network architecture not only consumes a lot of computing time, but also loses some detailed information of the scene. The constructed map has low accuracy, which ultimately makes it impossible to achieve the goal of building an accurate map. Summary of the Invention

[0003] The present application provides a semantic map construction method, apparatus, device and storage medium to solve the problem of low map accuracy constructed by existing mapping methods.

[0004] In order to solve the above technical problems, the technical solution of this application is solved through the following embodiments:

[0005] An embodiment of the present application provides a semantic map construction method, including: after a target vehicle starts driving, sequentially obtaining each frame of depth scene image captured by a preset depth camera and generating a semantic depth scene image corresponding to each frame of the depth scene image; calculating the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle; for each frame of the semantic depth scene image, inputting the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model; the semantic map construction model is a simultaneous localization and mapping (SLAM) model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into two-dimensional space according to the camera pose corresponding to the semantic depth scene image, to obtain a volume rendering image corresponding to the semantic depth scene image; and constructing a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

[0006] Among them, generating the semantic depth scene image corresponding to each frame of the depth scene image includes: according to the acquisition order of the depth scene images of each frame, using a preset semantic segmentation network to sequentially generate the semantic depth scene image corresponding to each frame of the depth scene image.

[0007] Among them, calculating the camera pose corresponding to each frame of the semantic depth scene image based on the parameters of the target vehicle includes: obtaining a preset initial camera pose and using the initial camera pose as the camera pose corresponding to the first frame of the semantic depth scene image; starting from the second frame of the semantic depth scene image, according to the frame interval between two adjacent frames of the semantic depth scene image and the driving information of the target vehicle, using a preset wheel speed odometer method to calculate the camera pose corresponding to each frame of the semantic depth scene image.

[0008] Wherein, before inputting the semantic depth scene image and its corresponding camera pose into the pre-trained semantic map construction model, the method also includes: after the sample vehicle starts driving, sequentially collecting sample depth scene images and generating sample semantic depth scene images corresponding to each frame of the sample depth scene image; calculating the camera pose corresponding to each frame of the sample semantic depth scene image according to the parameters of the sample vehicle; using the sequentially generated multiple frames of the sample semantic depth scene images and the camera pose corresponding to each frame of the sample semantic depth scene image, training the semantic map construction model in two stages and using different loss functions to optimize the semantic map construction model in different stages.

[0009] Among them, the semantic map construction model is trained in two stages and different loss functions are used to optimize the semantic map construction model in different stages, including: obtaining part of the sample semantic depth scene images in multiple frames of the sample semantic depth scene images; generating a frame of coarse-grained scene image based on each of the obtained sample semantic depth scene images; wherein the coarse-grained scene image is formed by retaining a preset number of sampling pixels in its corresponding sample semantic depth scene image; in the first stage, the semantic map construction model is trained using multiple frames of the coarse-grained scene images and their corresponding camera poses until the semantic map construction model is initially converged according to a first loss function; in the second stage, the semantic map construction model is trained using multiple frames of the sample semantic depth scene images and their corresponding camera poses until the semantic map construction model is converged again according to a second loss function.

[0010] The sample depth scene image includes: an RGB true value map, a depth true value map, and a semantic true value map; in the model training stage, the semantic map construction model outputs a volume rendering image corresponding to the sample depth scene image; wherein the volume rendering image includes: an RGB volume rendering map, a depth volume rendering map, and a semantic volume rendering map;

[0011] The first loss function is:

[0012]

[0013] And / or, the second loss function is:

[0014]

[0015] Among them, L track represents the first loss value calculated by the first loss function; L ba represents the second loss value calculated by the second loss function; M′ represents the number of pixels in the coarse-grained scene image; M represents the number of pixels in the sample semantic depth scene image; K represents the number of sample semantic depth scene images used in the second stage; d m represents the depth value of the mth pixel in the depth volume rendering; d gt represents the depth value of the pixel corresponding to the mth pixel in the depth truth map; c n represents the RGB value of the mth pixel in the RGB volume rendering image; c gt represents the RGB value of the pixel corresponding to the mth pixel in the RGB true value map; s m represents the semantic RGB value of the mth pixel in the semantic body rendering image; s gt represents the semantic RGB value of the pixel point corresponding to the mth pixel point in the semantic truth map; μ and α are both weight coefficients; || 1 represents the L1 norm.

[0016] The method further comprises: obtaining, for each frame of the coarse-grained scene image, a volume rendering image output by the semantic map construction model for the coarse-grained scene image; wherein the coarse-grained scene image corresponds to a frame of a sample depth scene image; reading the opacity and depth values ​​corresponding to each pixel in the volume rendering image; adjusting, for each pixel, parameters in the semantic map construction model if the opacity corresponding to the pixel is less than a preset opacity threshold and the difference between the depth value corresponding to the pixel and the depth value of the three-dimensional point corresponding to the pixel in the sample depth scene image is greater than a preset depth deviation threshold; and / or projecting, for each pixel, the opacity and depth values ​​corresponding to the pixel into three-dimensional space using the camera pose corresponding to the coarse-grained scene image to obtain a three-dimensional opacity point and a three-dimensional depth point corresponding to the pixel; and adjusting the parameters in the semantic map construction model if the Euclidean distance between the three-dimensional opacity point and the three-dimensional depth point is greater than a preset distance threshold.

[0017] An embodiment of the present application also provides a semantic map construction device, including: a generation module, which is used to sequentially obtain each frame of depth scene image captured by a preset depth camera after the target vehicle starts to travel and generate a semantic depth scene image corresponding to each frame of the depth scene image; a calculation module, which is used to calculate the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle; a processing module, which is used to input the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model for each frame of the semantic depth scene image; the semantic map construction model is a simultaneous localization and mapping SLAM model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into two-dimensional space according to the camera pose corresponding to the semantic depth scene image, and obtain a volume rendering image corresponding to the semantic depth scene image; a construction module, which is used to construct a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

[0018] An embodiment of the present application also provides a semantic map construction device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to: execute a semantic map construction program stored in the memory to implement any of the above-mentioned semantic map construction methods.

[0019] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed to implement any of the semantic map construction methods described above.

[0020] The above-mentioned technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application can sequentially obtain each frame of depth scene image captured by a preset depth camera after the target vehicle starts driving and generate a semantic depth scene image corresponding to each frame of the depth scene image; calculate the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle; for each frame of the semantic depth scene image, input the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model; the semantic map construction model is a simultaneous localization and mapping SLAM model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into two-dimensional space according to the camera pose corresponding to the semantic depth scene image, and obtain a volume rendering image corresponding to the semantic depth scene image; and construct a semantic scene map based on the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model. The embodiment of the present application constructs a semantic scene map based on the semantic depth scene image. In the mapping process, the semantic depth scene image is projected into a volume rendering image using a SLAM model based on 3D Gaussian splattering. The semantic scene map finally constructed not only integrates semantic information but also has high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0024] Figure 1 is a flowchart of a method for constructing a semantic map according to an embodiment of the present application;

[0025] Figure 2is a schematic diagram of a semantic map construction method according to an embodiment of the present application;

[0026] Figure 3 A flowchart of the training steps for constructing a semantic map model according to an embodiment of the present application;

[0027] Figure 4 is a structural diagram of a semantic map construction device according to an embodiment of the present application;

[0028] Figure 5 A structural diagram of a semantic map construction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0031] The present application embodiment provides a method for constructing a semantic map. Figure 1 , which is a flowchart of a semantic map construction method according to an embodiment of the present application.

[0032] Step S110 : After the target vehicle starts to travel, each frame of the depth scene image captured by the preset depth camera is sequentially acquired and a semantic depth scene image corresponding to each frame of the depth scene image is generated.

[0033] The target vehicle is a vehicle that requires map building. For example, the target vehicle is a vehicle with autonomous driving capabilities.

[0034] The depth camera is mounted on the target vehicle and, under the control of the target vehicle's controller, collects depth images of the target scene. The target scene is the environment in which the target vehicle is located. For example, the target scene can be an urban road environment, a town road environment, or a mountain road environment.

[0035] A depth scene image is an RGB-D image (depth RGB image) with depth information within the depth camera's field of view. A depth scene image includes a depth image and an RGB image. An RGB image is a three-channel color image of red, green, and blue.

[0036] Semantic depth scene images refer to depth scene images that are fused with semantic information.

[0037] Semantic information refers to the semantic categories of each object in the depth scene image, including but not limited to: object category, position, posture, and functional attributes.

[0038] Furthermore, during the sequential acquisition of depth scene images, a semantic depth scene image corresponding to each frame of the depth scene image is sequentially generated using a preset semantic segmentation network in the order in which the depth scene images are acquired. The semantic segmentation network is used to identify each pixel in the depth scene image and assign semantic information to each pixel, thereby completing pixel-wise dense classification of the depth scene image.

[0039] Step S120: Calculate the camera pose corresponding to each frame of the semantic depth scene image based on the parameters of the target vehicle.

[0040] The target vehicle parameters include, but are not limited to, the target vehicle's driving information and the frame interval used when acquiring the depth scene image. The target vehicle's driving information includes at least the rotational speed of each wheel of the target vehicle.

[0041] Specifically, a preset initial camera pose is obtained, and the initial camera pose is used as the camera pose corresponding to the first frame of the semantic depth scene image; starting from the second frame of the semantic depth scene image, the camera pose corresponding to each frame of the semantic depth scene image is calculated using a preset wheel speed odometer method according to the frame interval between two adjacent frames of the semantic depth scene image and the driving information of the target vehicle.

[0042] In other words, the initial camera pose of the depth camera is taken as the unit matrix in the world coordinate system, the unit matrix corresponding to the first frame of the semantic depth scene image, starting from the second frame of the semantic depth scene image, the camera pose corresponding to each frame of the semantic depth scene image is: the relative pose of the depth camera relative to the initial camera pose.

[0043] Furthermore, the relative position of the target vehicle relative to the initial vehicle position can be determined based on the frame interval between the semantic depth scene image of the current frame and the semantic depth scene image of the previous frame, as well as the rotational speed of each wheel of the target vehicle during the time period corresponding to this frame interval. The camera pose of the depth camera, i.e., the camera pose corresponding to the semantic depth scene image of the current frame, can be determined based on the relative position of the target vehicle relative to the initial vehicle pose and the installation position relationship between the target vehicle and the depth camera. For example, when the depth camera's viewing angle is directly in front of the vehicle, the vehicle pose of the target vehicle can be directly determined as the camera pose of the depth camera. The initial vehicle pose of the target vehicle is the same as the initial camera pose of the depth camera, and the relative pose of the target vehicle is the same as the relative pose of the depth camera.

[0044] Generally speaking, the frame interval used by the target camera is a fixed value, so the frame intervals of any two adjacent frames of semantic depth scene images are equal.

[0045] Step S130: For each frame of the semantic depth scene image, the semantic depth scene image and its corresponding camera pose are input into a pre-trained semantic map construction model.

[0046] The semantic map construction model is a SLAM model based on 3D Gaussian splattering.

[0047] The semantic map construction model is used to project the semantic depth scene image into a two-dimensional space according to the camera pose corresponding to the semantic depth scene image, so as to obtain a volume rendering image corresponding to the semantic depth scene image.

[0048] Step S140 , constructing a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

[0049] The volume-rendered image corresponding to the semantic depth scene image is an RGB-D image captured by the depth camera at a certain camera pose within the corresponding viewing angle of the target scene. Volume-rendered images include RGB volume renderings, depth volume renderings, and semantic volume renderings.

[0050] A semantic scene map is a map of the target scene that incorporates semantic information. It is an RGB-D image. Furthermore, the semantic map construction model updates the semantic scene map with each frame of volume-rendered image output. This continuous updating improves the semantic scene map, ultimately resulting in a complete semantic map of the target scene.

[0051] In an embodiment of the present application, after the target vehicle starts driving, each frame of depth scene image captured by a preset depth camera is sequentially acquired and a semantic depth scene image corresponding to each frame of the depth scene image is generated; based on the parameters of the target vehicle, the camera pose corresponding to each frame of the semantic depth scene image is calculated; for each frame of the semantic depth scene image, the semantic depth scene image and its corresponding camera pose are input into a pre-trained semantic map construction model; the semantic map construction model is a simultaneous localization and mapping SLAM model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into two-dimensional space according to the camera pose corresponding to the semantic depth scene image, and obtain a volume rendering image corresponding to the semantic depth scene image; a semantic scene map is constructed based on the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model. The embodiment of the present application constructs a semantic scene map based on the semantic depth scene image, and in the mapping process, the semantic depth scene image is projected into a volume rendering image using a SLAM model based on 3D Gaussian splashing. Since 3D Gaussian splashing relies on the ellipsoid space and explicitly represents the three-dimensional spatial relationship of multiple images, there is a significant improvement in comprehensive performance such as computational efficiency and positioning accuracy, so that the semantic scene map finally constructed by this application not only integrates semantic information but also has high accuracy.

[0052] Furthermore, since the SLAM model based on 3D Gaussian splashing does not involve an intermediate network, the loss of scene information is avoided, and the 3D Gaussian splashing algorithm has the characteristics of fast rendering speed and high positioning accuracy. Therefore, the volume rendering image output by the SLAM model based on 3D Gaussian splashing has high image accuracy and good rendering speed and effect.

[0053] Furthermore, the volume-rendered image of the embodiment of the present application is an RGB-D image that integrates semantic information. Compared with a simple RGB map, the semantic scene map of the present application is easier to apply to downstream tasks.

[0054] The following describes the semantic map construction process of the SLAM model (semantic map construction model) based on 3D Gaussian splattering in an embodiment of the present application.

[0055] Figure 2 Schematic diagram of a semantic map construction method according to an embodiment of the present application.

[0056] After the target vehicle starts to move, the depth camera installed in the target vehicle is called to collect a depth scene image. The depth scene image is an RGB-D image, which includes an RGB true value image and a depth true value image.

[0057] The RGB-D image is passed through a semantic segmentation network to obtain the semantic ground truth image corresponding to the RGB-D image. This semantic ground truth image is then added to the RGB-D image to produce a semantic depth scene image, i.e., a depth scene image that incorporates semantic information. This semantic depth scene image consists of the RGB ground truth image, the depth ground truth image, and the semantic ground truth image.

[0058] After the target vehicle begins driving, the camera pose corresponding to the depth camera is initialized and set to the identity matrix. Subsequently, the depth camera pose is tracked using the vehicle speedometer. Furthermore, the camera pose corresponding to the semantic depth scene image corresponding to the first depth scene image frame captured by the depth camera is the initial camera pose. Based on this initial camera pose, the camera position is updated for each subsequent semantic depth scene image frame using the wheel speedometer.

[0059] According to the generation order of the semantic depth scene images, the semantic depth scene images and their corresponding camera poses are sequentially input into the pre-trained semantic map construction model.

[0060] The semantic map construction model initializes the Gaussian scene based on the semantic depth scene image and its corresponding camera pose. For example, a Gaussian scene includes multiple 3D Gaussian distributions, each of which includes the position X of the center point, the ellipsoid shape Σ, the color (spherical harmonic function Y i ), opacity Λ, semantic information S and other parameters, then this 3D scene can be expressed as G = {G i :(X i ,Σ i ,Y i ,Λ i ,S i )|i=1,…,N}. Here, i represents the i-th Gaussian distribution.

[0061] The semantic map construction model projects each 3D Gaussian distribution onto a 2D plane according to the camera pose P to obtain a volume rendered image. Furthermore, the projection function is: Σ ′ =JP -1 ΣP -T J T . Where J represents the Jacobian matrix of the affine estimate of the projection function.

[0062] Volume rendering images include: color volume rendering images (RGB volume rendering images), depth volume rendering images and semantic volume rendering images.

[0063] The RGB value of each pixel in the RGB volume rendering can be expressed as:

[0064]

[0065] Among them, c i is the RGB value corresponding to the i-th 3D Gaussian distribution, which can be obtained through the spherical harmonic function; α i represents the i-th 3D Gaussian distribution, α j Denotes the jth 3D Gaussian distribution. The 3D Gaussian distribution can be obtained by multiplying the projection function Σ′ and the opacity Λ of the 3D Gaussian distribution.

[0066] The depth value of each pixel in the depth volume rendering can be expressed as:

[0067]

[0068] Among them, d i Represents the depth information corresponding to the i-th 3D Gaussian distribution.

[0069] The semantic RGB value of each pixel in the semantic volume rendering can be expressed as:

[0070]

[0071] Among them, s i Represents the RGB value represented by the semantic information corresponding to the i-th 3D Gaussian distribution.

[0072] The volume-rendered image is an RGB-D image fused with semantic information. When the semantic map construction model sequentially outputs the volume-rendered image corresponding to each frame of the semantic depth scene image, the semantic scene map corresponding to the current scene can be dynamically constructed based on the currently output volume-rendered image.

[0073] In order to enable the semantic map construction model to better complete the projection task, the semantic map construction model can be pre-trained. The following further describes the model training process of the embodiment of the present application. Figure 3 , which is a flow chart of the training steps for constructing a semantic map model according to an embodiment of the present application.

[0074] Step S310 : After the sample vehicle starts traveling, sample depth scene images are sequentially collected and a sample semantic depth scene image corresponding to each frame of the sample depth scene image is generated.

[0075] The sample depth scene image includes: RGB truth map, depth truth map and semantic truth map.

[0076] Step S320: Calculate the camera pose corresponding to each frame of the sample semantic depth scene image based on the parameters of the sample vehicle.

[0077] You can refer to the above method to obtain a pre-set initial camera pose, and use the initial camera pose as the camera pose corresponding to the first frame sample semantic depth scene image; starting from the second frame sample semantic depth scene image, according to the frame interval between two adjacent frames of sample semantic depth scene images and the driving information of the sample vehicle, use the preset wheel speed odometer method to calculate the camera pose corresponding to each frame sample semantic depth scene image.

[0078] Step S330, using the sequentially generated multiple frames of the sample semantic depth scene images and the camera pose corresponding to each frame of the sample semantic depth scene image, train the semantic map construction model in two stages and use different loss functions in different stages to optimize the semantic map construction model.

[0079] During model training, the semantic map construction model outputs a volume rendering image (i.e., the model's predicted image) corresponding to the sample depth scene image. The volume rendering image includes an RGB volume rendering image, a depth volume rendering image, and a semantic volume rendering image. Parameters in the semantic map construction model are adjusted based on the loss between the predicted value and the true value.

[0080] In this embodiment, the loss function may be a minimization objective function.

[0081] In this embodiment, among the sample depth scene images (true value images), RGB true value maps, depth true value maps and semantic true value maps, as well as volume rendering images (predicted images), RGB volume rendering maps, depth volume rendering maps and semantic volume rendering maps that have corresponding relationships, the image scale of each image is the same and the number of pixels contained in each image is the same.

[0082] In this embodiment, the following steps may be used to train the semantic map construction model.

[0083] Step S1, obtaining part of the sample semantic depth scene images from multiple frames of the sample semantic depth scene images; generating a frame of coarse-grained scene image based on each of the obtained sample semantic depth scene images; wherein the coarse-grained scene image is formed by retaining a preset number of sampling pixels in its corresponding sample semantic depth scene image.

[0084] The embodiment of the present application trains the semantic map construction model in two stages: the first stage is coarse-grained training, and the second stage is fine-grained training.

[0085] Before coarse-grained training, the sample semantic depth scene images can be coarse-grained processed, that is, a portion of the sample semantic depth scene images is obtained; in each frame of the obtained sample semantic depth scene image, a preset number of sampled pixels are retained and the rest of the pixels are filtered out.

[0086] Furthermore, the sampling data can be empirical values ​​or values ​​obtained through experiments. For example, retaining 1 / 4 of the pixels in the sample semantic depth scene image can be performed using either uniform sampling or random sampling.

[0087] Step S2, in the first stage, using multiple frames of the coarse-grained scene images and their corresponding camera poses to train the semantic map construction model until it is determined that the semantic map construction model has initially converged according to a first loss function.

[0088] The camera pose corresponding to the coarse-grained scene image is the camera pose corresponding to the sample semantic depth scene image corresponding to the coarse-grained scene image.

[0089] The first loss function is:

[0090]

[0091] Among them, L track represents the first loss value calculated by the first loss function; M′ represents the number of pixels in the coarse-grained scene image; c m represents the RGB value of the mth pixel in the RGB volume rendering image; c gt represents the RGB value of the pixel corresponding to the mth pixel in the RGB true value map; s m represents the semantic RGB value of the mth pixel in the semantic body rendering image; s gt represents the semantic RGB value of the pixel corresponding to the mth pixel in the semantic truth map; ||1 represents the L1 norm (also known as the Manhattan norm). Furthermore, in the first stage, the value of M′ is the number of pixels retained in the sample semantic depth scene image.

[0092] Furthermore, in the first stage, a frame of the coarse-grained scene image and its corresponding camera pose are sequentially obtained; wherein, the coarse-grained scene image corresponds to a frame of sample depth scene image; the coarse-grained scene image and its corresponding camera pose are input into the semantic map construction model, and a volume rendering image output by the semantic map construction model is obtained; after the semantic map construction model outputs the volume rendering image each time, the first lossy image is used to obtain the volume rendering image according to the RGB true value map, depth true value map and semantic true value map included in the coarse-grained scene image, and the RGB volume rendering map, depth volume rendering map and semantic volume rendering map included in the corresponding volume rendering image. The loss function calculates a first loss value; when the first loss value is greater than a preset coarse-grained loss threshold, the parameters in the semantic map construction model are adjusted, and then the next frame of the coarse-grained scene image and its corresponding camera pose are continuously input into the semantic map construction model to continue training the semantic map construction model; when the first loss value is less than or equal to the coarse-grained loss threshold, it is determined whether the current first loss value is already the first loss value with the minimum value in the first stage; if so, the training of the first stage is completed; if not, the next frame of the coarse-grained scene image and its corresponding camera pose are continuously input into the semantic map construction model to continue training the semantic map construction model.

[0093] Step S3, in the second stage, the semantic map construction model is trained using multiple frames of the sample semantic depth scene images and their corresponding camera poses until the semantic map construction model is determined to converge again according to the second loss function.

[0094] The second loss function is:

[0095]

[0096] Among them, L ba represents the second loss value calculated by the second loss function; M represents the number of pixels in the sample semantic depth scene image; K represents the number of samples used in the second stage, that is, the number of sample semantic depth scene images used in the second stage; d m represents the depth value of the mth pixel in the depth volume rendering; d gt represents the depth value of the pixel corresponding to the mth pixel in the depth truth map; c m represents the RGB value of the mth pixel in the RGB volume rendering image; c gt represents the RGB value of the pixel corresponding to the mth pixel in the RGB true value map; s m represents the semantic RGB value of the mth pixel in the semantic body rendering image; s gtrepresents the semantic RGB value of the pixel corresponding to the mth pixel in the semantic truth map; μ and α are both weight coefficients; ||1 represents the L1 norm. Furthermore, the weight coefficients μ and α can reach optimal values ​​through continuous training.

[0097] Furthermore, in the second stage, a frame of sample depth scene image and its corresponding camera pose are sequentially obtained; the sample depth scene image and its corresponding camera pose are input into the semantic map construction model, and a volume rendering image output by the semantic map construction model is obtained; after each volume rendering image output by the semantic map construction model, a second loss value is calculated using a second loss function based on the RGB true value map, depth true value map and semantic true value map included in the sample depth scene image, and the RGB volume rendering map, depth volume rendering map and semantic volume rendering map included in the corresponding volume rendering image; when the second loss value is greater than a preset fine-grained loss threshold, the parameters in the semantic map construction model are adjusted, and then the next frame of the sample depth scene image and its corresponding camera pose are continuously input into the semantic map construction model to continue training the semantic map construction model; when the second loss value is less than or equal to the fine-grained loss threshold, it is determined whether the current second loss value is already the second loss value with the minimum value in the second stage; if so, the second stage training is completed; if not, the next frame of the sample depth scene image and its corresponding camera pose are continuously input into the semantic map construction model to continue training the semantic map construction model.

[0098] After completing the first and second phases of training, the training of the semantic map construction model is completed, and the semantic map construction model can be applied to the semantic map construction task.

[0099] In an embodiment of the present application, in order to further improve the accuracy of mapping, in the first stage, in addition to adjusting the semantic map construction model according to the first loss function, the semantic map construction model can also be adjusted in the following manner.

[0100] Specifically, for each frame of the coarse-grained scene image, a volume rendering image output by the semantic map construction model for the coarse-grained scene image is obtained; wherein the coarse-grained scene image corresponds to a frame of sample depth scene image.

[0101] Method 1, reading the opacity and depth values ​​corresponding to each pixel in the volume rendered image; for each pixel, if the opacity corresponding to the pixel is less than a preset opacity threshold, and the difference between the depth value corresponding to the pixel and the depth value of the three-dimensional point corresponding to the pixel in the sample depth scene image is greater than a preset depth deviation threshold, then adjusting the parameters in the semantic map construction model.

[0102] Among them, the opacity corresponding to the i-th pixel is:

[0103] If the opacity corresponding to the pixel point is too small and the depth value deviates greatly from the corresponding depth value in the sample depth scene image, it means that the pixel point is unreliable. By adjusting the parameters in the semantic map construction model, the semantic map construction model can initialize multiple Gaussian distributions around the Gaussian distribution corresponding to the pixel point when initializing the sample depth scene image again.

[0104] Method 2: Read the opacity and depth values ​​corresponding to each pixel in the volume rendered image; for each pixel, use the camera pose corresponding to the coarse-grained scene image to project the opacity and depth values ​​corresponding to the pixel into three-dimensional space, respectively, to obtain a three-dimensional opacity point and a three-dimensional depth point corresponding to the pixel; if the Euclidean distance between the three-dimensional opacity point and the three-dimensional depth point is greater than a preset distance threshold, adjust the parameters in the semantic map construction model.

[0105] During the mapping process, some 3D Gaussian distributions that are not on the surface of the object may have high opacity. Therefore, during the iteration process, the opacity of the pixel point can be reprojected back to the three-dimensional space. At the same time, the depth value of the pixel point is also projected back to the three-dimensional space. If the Euclidean distance between the two three-dimensional points is greater than the preset distance threshold, it means that the pixel point is unreliable. At this time, the parameters in the semantic map construction model can be adjusted so that the opacity of the 3D Gaussian distribution is reduced when the semantic map construction model initializes the sample depth scene image again.

[0106] In an embodiment of the present application, when the first loss value is less than the coarse-grained loss threshold, method 1 and method 2 can be used to determine whether the parameters in the semantic map construction model need to be adjusted; or, when the first loss value is greater than or equal to the coarse-grained loss threshold, the parameters in the semantic map construction model can be adjusted with reference to the structures of method 1 and method 2.

[0107] In the embodiment of the present application, a 3D Gaussian scene can be projected onto a 2D image, and then the rendered 2D image is compared with the true value image captured by the depth camera to calculate the LOSS (loss) between the two sets of images. Then, when the LOSS is minimized, the 3D Gaussian scene parameters are updated through the volume rendering formula, including: the position X of the center point, the shape Σ of the ellipsoid, the color (spherical harmonic function Y i ), opacity Λ, semantic information S and other parameters, by repeatedly iteratively optimizing the semantic map construction model, a semantic map construction model with higher accuracy is finally obtained.

[0108] Compared to the NERF-SLAM algorithm, the SLAM model based on 3D Gaussian splattering in the embodiments of the present application can increase the algorithm's mapping and positioning speed, greatly improving the algorithm's efficiency. Furthermore, the SLAM model based on 3D Gaussian splattering significantly enhances the algorithm's ability to understand the environment. Traditional SLAM methods primarily focus on the geometric structure of the environment, while the SLAM model based on 3D Gaussian splattering further extracts semantic information from the environment, such as the category, location, posture, and functional attributes of objects. This enables the vehicle to not only perceive the spatial layout of the environment but also understand the meaning and purpose of each object in the environment. This in-depth understanding of the environment enables the vehicle to perform tasks more intelligently, such as identifying obstacles, identifying traversable areas, and understanding human commands. Using semantic information as constraints during the mapping process can improve mapping quality and algorithm accuracy. Traditional SLAM methods primarily rely on geometric features for positioning and mapping, but in some complex environments, these features may not be rich or stable enough. Semantic information, on the other hand, can provide rich information about object categories, attributes, and relationships. This information can serve as additional features and constraints to help the algorithm more accurately locate and understand the environment.

[0109] The present application also provides a semantic map construction device. Figure 4 , which is a structural diagram of a semantic map construction device according to an embodiment of the present application.

[0110] The semantic map construction device includes:

[0111] The generating module 410 is configured to sequentially acquire each frame of the depth scene image captured by the preset depth camera after the target vehicle starts to travel, and generate a semantic depth scene image corresponding to each frame of the depth scene image.

[0112] The calculation module 420 is used to calculate the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle.

[0113] The processing module 430 is used to input the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model for each frame of the semantic depth scene image; the semantic map construction model is a simultaneous localization and mapping SLAM model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into a two-dimensional space according to the camera pose corresponding to the semantic depth scene image, and obtain a volume rendering image corresponding to the semantic depth scene image.

[0114] The construction module 440 is configured to construct a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

[0115] The functions of the device described in the embodiment of the present application have been described in the above method embodiment. Therefore, for any details not fully described in the description of this embodiment, please refer to the relevant description in the above embodiment and will not be repeated here.

[0116] The present application also provides a semantic map construction device, such as Figure 5 , which is a structural diagram of a semantic map construction device according to an embodiment of the present application.

[0117] The semantic map construction device includes: a processor 510, a communication interface 520, a memory 530 and a communication bus 540. The processor 510, the communication interface 520 and the memory 530 communicate with each other via the communication bus 540.

[0118] The memory 530 is used to store computer programs.

[0119] In one embodiment of the present application, the processor 510, when executing the program stored on the memory 530, implements the semantic map construction method provided by any of the aforementioned method embodiments, including: after the target vehicle starts driving, sequentially obtaining each frame of depth scene image captured by a preset depth camera and generating a semantic depth scene image corresponding to each frame of the depth scene image; calculating the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle; for each frame of the semantic depth scene image, inputting the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model; the semantic map construction model is a simultaneous localization and mapping SLAM model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into two-dimensional space according to the camera pose corresponding to the semantic depth scene image, to obtain a volume rendering image corresponding to the semantic depth scene image; and constructing a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

[0120] Among them, generating the semantic depth scene image corresponding to each frame of the depth scene image includes: according to the acquisition order of the depth scene images of each frame, using a preset semantic segmentation network to sequentially generate the semantic depth scene image corresponding to each frame of the depth scene image.

[0121] Among them, calculating the camera pose corresponding to each frame of the semantic depth scene image based on the parameters of the target vehicle includes: obtaining a preset initial camera pose and using the initial camera pose as the camera pose corresponding to the first frame of the semantic depth scene image; starting from the second frame of the semantic depth scene image, according to the frame interval between two adjacent frames of the semantic depth scene image and the driving information of the target vehicle, using a preset wheel speed odometer method to calculate the camera pose corresponding to each frame of the semantic depth scene image.

[0122] Wherein, before inputting the semantic depth scene image and its corresponding camera pose into the pre-trained semantic map construction model, the method also includes: after the sample vehicle starts driving, sequentially collecting sample depth scene images and generating sample semantic depth scene images corresponding to each frame of the sample depth scene image; calculating the camera pose corresponding to each frame of the sample semantic depth scene image according to the parameters of the sample vehicle; using the sequentially generated multiple frames of the sample semantic depth scene images and the camera pose corresponding to each frame of the sample semantic depth scene image, training the semantic map construction model in two stages and using different loss functions to optimize the semantic map construction model in different stages.

[0123] Among them, the semantic map construction model is trained in two stages and different loss functions are used to optimize the semantic map construction model in different stages, including: obtaining part of the sample semantic depth scene images in multiple frames of the sample semantic depth scene images; generating a frame of coarse-grained scene image based on each of the obtained sample semantic depth scene images; wherein the coarse-grained scene image is formed by retaining a preset number of sampling pixels in its corresponding sample semantic depth scene image; in the first stage, the semantic map construction model is trained using multiple frames of the coarse-grained scene images and their corresponding camera poses until the semantic map construction model is initially converged according to a first loss function; in the second stage, the semantic map construction model is trained using multiple frames of the sample semantic depth scene images and their corresponding camera poses until the semantic map construction model is converged again according to a second loss function.

[0124] The sample depth scene image includes: an RGB true value map, a depth true value map, and a semantic true value map; in the model training stage, the semantic map construction model outputs a volume rendering image corresponding to the sample depth scene image; wherein the volume rendering image includes: an RGB volume rendering map, a depth volume rendering map, and a semantic volume rendering map;

[0125] The first loss function is:

[0126]

[0127] And / or, the second loss function is:

[0128]

[0129] Among them, L track represents the first loss value calculated by the first loss function; L ba represents the second loss value calculated by the second loss function; M′ represents the number of pixels in the coarse-grained scene image; M represents the number of pixels in the sample semantic depth scene image; K represents the number of sample semantic depth scene images used in the second stage; d m represents the depth value of the mth pixel in the depth volume rendering; d gt represents the depth value of the pixel corresponding to the mth pixel in the depth truth map; c n represents the RGB value of the mth pixel in the RGB volume rendering image; c gt represents the RGB value of the pixel corresponding to the mth pixel in the RGB true value map; s m represents the semantic RGB value of the mth pixel in the semantic body rendering image; s gt represents the semantic RGB value of the pixel point corresponding to the mth pixel point in the semantic truth map; μ and α are both weight coefficients; || 1 represents the L1 norm.

[0130] The method further comprises: obtaining, for each frame of the coarse-grained scene image, a volume rendering image output by the semantic map construction model for the coarse-grained scene image; wherein the coarse-grained scene image corresponds to a frame of a sample depth scene image; reading the opacity and depth values ​​corresponding to each pixel in the volume rendering image; adjusting, for each pixel, parameters in the semantic map construction model if the opacity corresponding to the pixel is less than a preset opacity threshold and the difference between the depth value corresponding to the pixel and the depth value of the three-dimensional point corresponding to the pixel in the sample depth scene image is greater than a preset depth deviation threshold; and / or projecting, for each pixel, the opacity and depth values ​​corresponding to the pixel into three-dimensional space using the camera pose corresponding to the coarse-grained scene image to obtain a three-dimensional opacity point and a three-dimensional depth point corresponding to the pixel; and adjusting the parameters in the semantic map construction model if the Euclidean distance between the three-dimensional opacity point and the three-dimensional depth point is greater than a preset distance threshold.

[0131] The present application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the semantic map construction method provided in any of the aforementioned method embodiments. Since the semantic map construction method has been described in detail above, any details not fully described in this embodiment are referred to the relevant descriptions in the aforementioned embodiments and are not repeated here.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0133] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0134] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0135] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A semantic map construction method, characterized in that: include: After the target vehicle starts to move, sequentially acquiring each frame of the depth scene image captured by the preset depth camera and generating a semantic depth scene image corresponding to each frame of the depth scene image; Calculating the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle; For each frame of the semantic depth scene image, the semantic depth scene image and its corresponding camera pose are input into a pre-trained semantic map construction model; the semantic map construction model is a simultaneous localization and mapping (SLAM) model based on 3D Gaussian splattering; the semantic map construction model is used to project the semantic depth scene image into a two-dimensional space according to the camera pose corresponding to the semantic depth scene image, to obtain a volume rendered image corresponding to the semantic depth scene image; Constructing a semantic scene map according to the volume rendered image corresponding to each frame of the semantic depth scene image output by the semantic map construction model; Before inputting the semantic depth scene image and its corresponding camera pose into the pre-trained semantic map construction model, the method further includes: after the sample vehicle starts driving, sequentially collecting sample depth scene images and generating sample semantic depth scene images corresponding to each frame of the sample depth scene image; calculating the camera pose corresponding to each frame of the sample semantic depth scene image based on the parameters of the sample vehicle; using the sequentially generated multiple frames of the sample semantic depth scene images and the camera pose corresponding to each frame of the sample semantic depth scene image, training the semantic map construction model in two stages and optimizing the semantic map construction model using different loss functions in different stages; The semantic map construction model is trained in two stages and different loss functions are used to optimize the semantic map construction model in different stages, including: obtaining a part of the sample semantic depth scene images in multiple frames of the sample semantic depth scene images; generating a frame of coarse-grained scene image based on each of the obtained sample semantic depth scene images; wherein the coarse-grained scene image is formed by retaining a preset number of sampling pixels in its corresponding sample semantic depth scene image; in the first stage, the semantic map construction model is trained using multiple frames of the coarse-grained scene images and their corresponding camera poses until the semantic map construction model is initially converged according to a first loss function; in the second stage, the semantic map construction model is trained using multiple frames of the sample semantic depth scene images and their corresponding camera poses until the semantic map construction model is converged again according to a second loss function.

2. The method according to claim 1, characterized in that Generating a semantic depth scene image corresponding to each frame of the depth scene image includes: According to the acquisition order of the depth scene images of each frame, a preset semantic segmentation network is used to sequentially generate a semantic depth scene image corresponding to the depth scene image of each frame.

3. The method according to claim 1, characterized in that Calculating the camera pose corresponding to each frame of the semantic depth scene image according to the parameters of the target vehicle includes: Obtaining a preset initial camera pose, and using the initial camera pose as the camera pose corresponding to the first frame of the semantic depth scene image; Starting from the second frame of the semantic depth scene image, the camera pose corresponding to each frame of the semantic depth scene image is calculated using a preset wheel speed odometer method according to the frame interval between two adjacent frames of the semantic depth scene image and the driving information of the target vehicle.

4. The method according to claim 1, wherein The sample depth scene image includes: an RGB true value map, a depth true value map, and a semantic true value map; during the model training phase, the semantic map construction model outputs a volume rendering image corresponding to the sample depth scene image; wherein the volume rendering image includes: an RGB volume rendering map, a depth volume rendering map, and a semantic volume rendering map; The first loss function is: And / or, the second loss function is: ； in, represents the first loss value calculated by the first loss function; represents the second loss value calculated by the second loss function; represents the number of pixels in the coarse-grained scene image; Represents the number of pixels in the sample semantic depth scene image; Indicates the number of the sample semantic depth scene images used in the second stage; Indicates the depth volume rendering image The depth value of each pixel; Indicates the depth truth map with The depth value of the pixel corresponding to the pixel; Indicates the first RGB value of each pixel; Indicates the RGB true value image and the The RGB value of the pixel corresponding to the pixel; Indicates the semantic body rendering diagram Semantic RGB value of each pixel; Indicates the semantic truth graph with The semantic RGB value of the pixel corresponding to the pixel; and are all weight coefficients; represents the L1 norm.

5. The method according to claim 1, wherein The method of using multiple frames of the coarse-grained scene images and their corresponding camera poses to train the semantic map construction model further includes: For each frame of the coarse-grained scene image, obtaining a volume rendering image output by the semantic map construction model for the coarse-grained scene image; wherein the coarse-grained scene image corresponds to a frame of sample depth scene image; Reading the opacity and depth values ​​corresponding to each pixel in the volume rendered image; For each of the pixel points, if the opacity corresponding to the pixel point is less than a preset opacity threshold, and the difference between the depth value corresponding to the pixel point and the depth value of the three-dimensional point corresponding to the pixel point in the sample depth scene image is greater than a preset depth deviation threshold, then adjust the parameters in the semantic map construction model; and / or, for each of the pixel points, using the camera pose corresponding to the coarse-grained scene image, project the opacity and depth value corresponding to the pixel point into the three-dimensional space respectively to obtain the opacity three-dimensional point and depth value three-dimensional point corresponding to the pixel point; if the Euclidean distance between the opacity three-dimensional point and the depth value three-dimensional point is greater than the preset distance threshold, then adjust the parameters in the semantic map construction model.

6. A semantic map construction device, characterized in that: include: A generation module is used to sequentially acquire each frame of depth scene image captured by a preset depth camera after the target vehicle starts to move and generate a semantic depth scene image corresponding to each frame of the depth scene image; A calculation module, configured to calculate a camera pose corresponding to each frame of the semantic depth scene image based on the parameters of the target vehicle; A processing module is used to input the semantic depth scene image and its corresponding camera pose into a pre-trained semantic map construction model for each frame of the semantic depth scene image; the semantic map construction model is a simultaneous localization and map construction SLAM model based on 3D Gaussian splashing; the semantic map construction model is used to project the semantic depth scene image into a two-dimensional space according to the camera pose corresponding to the semantic depth scene image, and obtain a volume rendering image corresponding to the semantic depth scene image; wherein, before the semantic depth scene image and its corresponding camera pose are input into the pre-trained semantic map construction model, it also includes: after the sample vehicle starts driving, sequentially collecting sample depth scene images and generating sample semantic depth scene images corresponding to each frame of the sample depth scene image; calculating the camera pose corresponding to each frame of the sample semantic depth scene image according to the parameters of the sample vehicle; using the sequentially generated multiple frames of the sample semantic depth scene images and the camera pose corresponding to each frame of the sample semantic depth scene image, dividing into two The semantic map construction model is trained in two stages and different loss functions are used to optimize the semantic map construction model in different stages, including: obtaining part of the sample semantic depth scene images in multiple frames of the sample semantic depth scene images; generating a frame of coarse-grained scene image based on each of the obtained sample semantic depth scene images; wherein the coarse-grained scene image is formed by retaining a preset number of sampling pixels in the corresponding sample semantic depth scene image; in the first stage, the semantic map construction model is trained using multiple frames of the coarse-grained scene images and their corresponding camera poses until the semantic map construction model is initially converged according to a first loss function; in the second stage, the semantic map construction model is trained using multiple frames of the sample semantic depth scene images and their corresponding camera poses until the semantic map construction model is converged again according to a second loss function; A construction module is used to construct a semantic scene map according to the volume rendering image corresponding to each frame of the semantic depth scene image output by the semantic map construction model.

7. A semantic map construction device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; At least one memory connected to the at least one bus, wherein the processor is configured to: execute a semantic map construction program stored in the memory to implement the semantic map construction method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed to implement the semantic map construction method according to any one of claims 1 to 5.

Images