A grid mapping optimization method and system based on adaptive control factors

Abstract

The application discloses a grid mapping optimization method and system based on an adaptive control factor, and belongs to the technical field of artificial intelligence.The application comprises a semantic map establishing module, a point cloud map establishing module, a semantic database, a map gridding module and a post-rendering module.The application carries out image preprocessing after camera image data is collected, and establishes a semantic map; carries out filtering after laser radar point cloud data is collected, and establishes a point cloud map; according to the data information of the corresponding importance score Score, the distance Dis from the central observation position and the multi-material composition Mat in the semantic database in the semantic model for establishing the point cloud map, the adaptive control factor delta of each individual model in the point cloud map is determined; according to the determined adaptive control factor, the connection between spatial points is realized in the map gridding module, and a gridded map is formed; according to pixel normal interpolation calculation of a light source, the map is restored and reconstructed through a rendering module, and the authenticity and integrity of the reconstructed map are improved.

Description

Technical Field

[0001] This invention relates to a gridded mapping optimization method based on adaptive control factors, belonging to the field of artificial intelligence technology. Background Technology

[0002] Point cloud maps, efficiently generated by LiDAR or RGB-D cameras, provide a basic map. While point cloud maps can meet some localization needs, they lack feature point information and cannot indicate whether a space is occupied, thus failing to directly meet navigation and obstacle avoidance requirements. Furthermore, with the development of SLAM technology, visualization and interaction have become new applications in environmental mapping, requiring a higher level of understanding of reconstructed maps—that is, semantic maps. Mesh maps, building upon point clouds, add necessary topological information between vertex data to form a continuous surface. This enables the effective definition of map geometric features and efficient rendering and display of model contours, holding crucial application value in 3D reconstruction and environmental mapping.

[0003] Mesh maps are represented in file structure as a set of surface polygons surrounding the described model, most commonly triangular meshes. The number of triangular units in the model depends on the degree of mesh discretization. Generally, the higher the degree of discretization, the higher the accuracy of the mesh map, but also the higher the number of stored triangular units and the greater the file storage complexity. Finding a balance between "shape contour geometric feature information" and "the number of triangular meshes" is a pressing issue that needs to be addressed.

[0004] Map models are not just about describing simple outline information. We also hope to incorporate a series of additional information, such as semantic information of specific model individuals, distance from the observation center, and information on the composition of multiple materials, into the environmental mapping process.

[0005] Therefore, it is necessary to provide a gridded mapping optimization method based on adaptive control factors to provide necessary technical innovation for broader applications of environmental mapping. Summary of the Invention

[0006] The main objective of this invention is to provide a gridded mapping optimization method and system based on adaptive control factors. After the camera acquires image data, image preprocessing is performed to establish a semantic map; after the lidar acquires point cloud data, filtering is performed to establish a point cloud map; after determining the adaptive control factors, the connection between spatial points is realized in the map gridding module to form a gridded map; and the realism and integrity of the map reconstruction are restored through the post-rendering module.

[0007] The objective of this invention is achieved through the following technical solution.

[0008] This invention discloses a gridded mapping optimization method based on adaptive control factors, comprising the following steps:

[0009] Step 1: Perform target recognition on the image dataset to obtain a semantic model, which is then added to the semantic database. The main attributes of the semantic model include individual model type and individual model feature attributes. Based on different map scenarios, assign different importance scores to the semantic models.

[0010] Step Two: The camera acquires real-time images. Semantic segmentation and image recognition are performed on these images to obtain a semantic model. This semantic model is then compared with the semantic models in the semantic database from Step One to create a semantic map. The importance score, distance from the observation center point, and material composition information of the semantic model for the real-time images are obtained. For newly emerging map scenes and individual models, feature extraction and classification are performed to expand the semantic database.

[0011] Step 3: Simultaneously acquire point cloud data using LiDAR to construct a point cloud map. Perform region segmentation and target identification on the point cloud map to establish a semantic model of the point cloud map.

[0012] Preferably, in step three, a point cloud map is created. To achieve a better visual effect, filtering is added during the map creation process. Statistical filtering is responsible for removing invalid and isolated points, and calculating the distance between each point and its nearest neighbor. The distance values ​​of each point are distributed to remove points with excessively large mean distances; a voxel filter is used for downsampling to ensure that there is only one point within a voxel of a predetermined size, thereby reducing the number of point clouds.

[0013] Step 4: Based on the semantic model of the point cloud map built in Step 3, calculate the corresponding importance score in the semantic database from Step 1. Distance from the center observation position Composed of multiple materials Based on the data information, determine the adaptive control factor for each individual model in the point cloud map. .

[0014] Preferably, the adaptive control factor for each individual model in the point cloud map is determined by the following formula. .

[0015]

[0016] in As the overall scaling factor, Model importance score The proportion factor, Distance from the center observation position The proportion factor, Composed of multiple materials The proportion of data information.

[0017] Step 5: Based on the adaptive control factors obtained in Step 4, construct a map and connect spatial points in the map gridding module to form a gridded map.

[0018] Delaunnay local properties refer to the triangular mesh Vertex in The circumcircle of the triangle contains no vertices other than the triangle's vertices. The mapping method based on the adaptive control factor obtained in step four is as follows:

[0019] Step 5.1: The triangular network used as input data All edges in the array that do not satisfy the Delaunnay local property Add to priority queue In this context, the priority queue functions as a "stack" and stores edges... The maximum value of the sum of the two corresponding vertex angles is used as the storage standard for the "top of the stack" position;

[0020] Step 5.2: sequentially process the edges that do not satisfy the local property. The following processing is performed: Insert a new vertex at the middle position of the edge. To optimize the triangular mesh formed by this edge, and to determine the first-order neighborhood that will be affected after inserting a vertex. All triangular meshes are recalculated for their Delaunnay local properties;

[0021] Step 5.3: Determine if a new vertex is inserted. The affected set If all edge elements within the queue still satisfy the local Delaunay property, then add the edges that do not satisfy the Delaunay property back to the priority queue. Repeat step 5.2 until all edges that do not satisfy the local property have been optimized, i.e., the priority queue has been completed. The process ends when the middle element is zero.

[0022] During the optimization process, when a local region triangle is cut into elongated triangular elements, excessively elongated triangular elements are treated as straight lines. The excessive elongation is determined by the following method: [The text then abruptly shifts to a different topic:] ...adaptive control factor... The judgment formula for identifying elongated triangular units is as follows:

[0023]

[0024] in For newly inserted vertices The distance between the vertex and other vertices.

[0025] Step six: Render the gridded map obtained in step five. Use the rendering module to restore and reconstruct the map, thereby improving the realism and completeness of the reconstructed map.

[0026] The 3D point coordinates in space are projected onto the parameter space texture coordinates using a spherical projection function. Based on the point cloud map from step three, the semantic model colors are determined, and the parameter space values ​​are converted to the texture space using a mapping function. Then, virtual light sources, including parallel light sources, point light sources, and spotlights, are added to the map model according to the actual scene. The light sources are calculated based on pixel normals using a fragment shader, and the map is reconstructed using the rendering module, improving the realism and completeness of the reconstructed map.

[0027] It also includes step seven, which uses the reconstructed map obtained in step six to establish interactive scenarios. These interactive scenarios include robot localization, navigation, obstacle avoidance, and VR / AR, further improving the robot's localization and navigation capabilities and accuracy, enhancing the immersive experience of virtual environments such as VR, and solving related engineering application problems.

[0028] This invention discloses a gridded mapping optimization system based on adaptive control factors, implemented based on the aforementioned gridded mapping optimization method based on adaptive control factors. The gridded mapping optimization system based on adaptive control factors includes a semantic map building module, a point cloud map building module, a semantic database, a map gridding module, and a post-rendering module.

[0029] The semantic map building module includes performing semantic segmentation and image recognition on image data acquired by the camera, and building a semantic map by comparing it with semantic models in the semantic database. For newly emerging map scenes and individual models, feature extraction and classification are performed to expand the semantic database.

[0030] The point cloud map building module includes adding filtering processing to the point cloud data after the lidar collects the data. The main processing methods are statistical filters to remove isolated points and points with large errors, and downsampling filters to downsample the point cloud data.

[0031] The semantic database includes attributes such as individual model type, individual model feature attributes, and the importance score of the individual model in the scene map. Scenes are divided into multiple types, and models should have different importance scores in different scenes.

[0032] The map gridding module determines the adaptive control factors of each volume model in the map based on the importance score, distance from the center observation position, and multi-material composition of the semantic model in the semantic database, and uses the Delaunay triangle grid optimization algorithm to perform adaptive gridding reconstruction of the map.

[0033] The post-rendering module processes operations including global illumination or point illumination, texture mapping, and shading mapping. It renders the gridded map input by the map gridding module and restores and reconstructs the map through the rendering module, thereby improving the realism and completeness of the reconstructed map.

[0034] Beneficial effects:

[0035] 1. Compared with traditional mapping methods with fixed parameters, the present invention discloses a gridded mapping optimization method and system based on adaptive control factors. On the basis of establishing a semantic database, the adaptive control factors of each semantic model in the environment are determined by importance score, observation distance and object material. A balance is achieved between feature information and grid quantity, taking into account both mapping efficiency and mapping quality, and expanding the application scope of environmental mapping.

[0036] 2. The present invention discloses a gridded mapping optimization method and system based on adaptive control factors. Based on the Delaunay triangle grid optimization method and spherical spatial projection texture rendering, it improves the realism and integrity of the reconstructed map. It can be used for robot localization, navigation, obstacle avoidance and interactive scenarios such as VR and AR, further improving the robot's localization and navigation capabilities and accuracy, and increasing the immersive experience of virtual environments such as VR.

[0037] 3. The present invention discloses a gridded mapping optimization method and system based on adaptive control factors. The method uses a fragment shader to calculate the light source based on pixel normal interpolation for each pixel, and then uses a rendering module to restore and reconstruct the map, thereby improving the realism and completeness of the reconstructed map. Attached Figure Description

[0038] Figure 1 The flowchart shows the grid-based mapping optimization method based on adaptive control factors.

[0039] Figure 2 This is a schematic diagram of the Delaunnay triangular mesh optimization algorithm, where: Figure 2 (a) Defines the Delaunay triangle mesh. Figure 2 (b) New vertices are added during the process. Figure 2 (c) Priority queue during optimization . Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] like Figure 1 As shown in the figure, the specific implementation steps of the gridded mapping optimization method based on adaptive control factors disclosed in this embodiment are as follows:

[0042] Step 1: Perform target recognition on the image dataset to obtain a semantic model, which is then added to the semantic database. The main attributes of the semantic model include individual model type and individual model feature attributes. Based on different map scenarios, assign different importance scores to the semantic models.

[0043] Step Two: The camera acquires real-time images. Semantic segmentation and image recognition are performed on these images to obtain a semantic model. This semantic model is then compared with the semantic models in the semantic database from Step One to create a semantic map. The importance score, distance from the observation center point, and material composition information of the semantic model for the real-time images are obtained. For newly emerging map scenes and individual models, feature extraction and classification are performed to expand the semantic database.

[0044] Step 3: Simultaneously acquire point cloud data using LiDAR to construct a point cloud map. Perform region segmentation and target identification on the point cloud map to establish a semantic model of the point cloud map.

[0045] Step 4: Based on the semantic model of the point cloud map built in Step 3, calculate the corresponding importance score in the semantic database from Step 1. Distance from the center observation position Composed of multiple materials Based on the data information, determine the adaptive control factor for each individual model in the point cloud map. :

[0046]

[0047] in As the overall scaling factor, Model importance score The proportion factor, Distance from the center observation position The proportion factor, Composed of multiple materials The proportion of data information.

[0048] Step 5: Construct a graph based on the adaptive control factors obtained in Step 4;

[0049] like Figure 2 As shown, the local properties of Delaunnay refer to the triangular mesh. Vertex in The circumcircle of the triangle contains no vertices other than the vertices of the triangle itself. The specific graph construction process is as follows:

[0050] 1. The triangular network used as input data All edges in the array that do not satisfy the Delaunnay local property Add to priority queue In this context, the priority queue functions as a "stack" and stores edges... The maximum value of the sum of the two corresponding vertex angles is used as the storage standard for the "top of the stack" position;

[0051] 2. For edges that do not satisfy the local property, proceed sequentially. The following processing is performed: Insert a new vertex at the middle position of the edge. To optimize the triangular mesh formed by this edge, and to determine the first-order neighborhood that will be affected after inserting a vertex. All triangular meshes are recalculated for their Delaunnay local properties;

[0052] 3. Determine if a new vertex is inserted. The affected set If all edge elements within the queue still satisfy the local Delaunay property, then add the edges that do not satisfy the Delaunay property back to the priority queue. Repeat step 2 until all edges that do not satisfy the local property have been optimized, i.e., the priority queue has been completed. The process ends when the middle element is zero.

[0053] During the optimization process, when a local region triangle is cut into elongated triangular elements, excessively elongated triangular elements are treated as straight lines. The excessive elongation is determined by the following method: [The text then abruptly shifts to a different topic:] ...adaptive control factor... The judgment formula for identifying elongated triangular units is as follows:

[0054]

[0055] in For newly inserted vertices Distances between vertices and other vertices. See attached diagram for details. Figure 2 As shown.

[0056] Step six involves rendering the gridded map to make the map more realistic.

[0057] The 3D point coordinates in space are projected into the parameter space (texture coordinates) using a spherical projection function. Based on the point cloud map in step three, the semantic model color is determined, and the parameter space values ​​are converted to the texture space using a mapping function. Then, based on the actual scene, virtual light sources are added to the map model, including parallel light sources, point light sources, and spotlights. The light sources are calculated based on each pixel using a fragment shader and interpolated according to the pixel normal.

[0058] Step three describes the creation of a point cloud map. To achieve better visual effects, filtering is added during the map creation process. Statistical filtering removes invalid and isolated points and calculates the distance between each point and its nearest neighbor. The distance values ​​of each point are distributed to remove points with excessively large mean distances; a voxel filter is used for downsampling to ensure that there is only one point within a voxel of a certain size, thereby reducing the number of point clouds.

[0059] This embodiment discloses a gridded mapping optimization system based on adaptive control factors, implemented based on the aforementioned gridded mapping optimization method based on adaptive control factors. The gridded mapping optimization system based on adaptive control factors includes a semantic map building module, a point cloud map building module, a semantic database, a map gridding module, and a post-rendering module.

[0060] The semantic map building module includes performing semantic segmentation and image recognition on image data acquired by the camera, and building a semantic map by comparing it with semantic models in the semantic database. For newly emerging map scenes and individual models, feature extraction and classification are performed to expand the semantic database.

[0061] The point cloud map building module includes adding filtering processing to the point cloud data after the lidar collects the data. The main processing methods are statistical filters to remove isolated points and points with large errors, and downsampling filters to downsample the point cloud data.

[0062] The semantic database includes attributes such as individual model type, individual model feature attributes, and the importance score of the individual model in the scene map. Scenes are divided into multiple types, and models should have different importance scores in different scenes.

[0063] The map gridding module determines the adaptive control factors of each volume model in the map based on the importance score, distance from the center observation position, and multi-material composition of the semantic model in the semantic database, and uses the Delaunay triangle grid optimization algorithm to perform adaptive gridding reconstruction of the map.

[0064] The post-rendering module processes operations including global illumination or point illumination, texture mapping, and shading mapping. It renders the gridded map input by the map gridding module and restores and reconstructs the map through the rendering module, thereby improving the realism and completeness of the reconstructed map.

[0065] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A gridded mapping optimization method based on adaptive control factors, characterized in that: Includes the following steps, Step 1: Perform target recognition on the image dataset to obtain a semantic model, which is then added to the semantic database; the main attributes of the semantic model include individual model type and individual model feature attributes; The semantic model of the marker has different importance scores depending on the map scenario; Step 2: The camera acquires real-time images, performs semantic segmentation and image recognition on the real-time images to obtain a semantic model of the real-time images; the semantic model of the real-time images is compared with the semantic model in the semantic database of Step 1 to build a semantic map and obtain the importance score, distance from the observation center point and material composition information of the semantic model of the real-time images; for newly emerging map scenes and individual models, feature extraction and classification are performed to expand the semantic database. Step 3: The lidar synchronously collects point cloud data and constructs a point cloud map; the point cloud map is segmented into regions and targets are identified to establish a semantic model of the point cloud map; Step 4: Based on the semantic model of the point cloud map built in Step 3, calculate the corresponding importance score in the semantic database from Step 1. Distance from the center observation position Composed of multiple materials Based on the data information, determine the adaptive control factor for each individual model in the point cloud map. ; The adaptive control factor for each individual model in the point cloud map is determined by the following formula. ; Where K is the overall scaling factor. Model importance score The proportion factor, Distance from the center observation position The proportion factor, Composed of multiple materials The proportion of data information; Step 5: Based on the adaptive control factors obtained in Step 4, construct a map and connect spatial points in the map gridding module to form a gridded map; In step five, Delaunnay local properties refer to the triangular mesh Vertex in The circumcircle of the triangle contains no vertices other than the triangle's vertices; the mapping method based on the adaptive control factor obtained in step four is as follows: Step 5.1: The triangular network used as input data All edges in the array that do not satisfy the Delaunnay local property Add to priority queue In this context, the priority queue functions as a "stack" and holds edges... The maximum value of the sum of the two corresponding vertices is used as the storage standard for the "top of the stack" position; Step 5.2: sequentially process the edges that do not satisfy the local property. The following processing is performed: Insert a new vertex at the middle position of the edge. To optimize the triangular mesh formed by that edge; The first-order neighborhood that will be affected after the vertex is inserted. All triangular meshes are recalculated for their Delaunnay local properties; Step 5.3: Determine if a new vertex is inserted. The affected set If all edge elements within the queue still satisfy the local Delaunay property, then add the edges that do not satisfy the Delaunay property back to the priority queue. Repeat step 5.2 until all edges that do not satisfy the local property have been optimized, i.e., the priority queue has been completed. The process ends when the middle element is zero. During the optimization process, when a local region triangle is cut into elongated triangular units; excessively elongated triangular units are treated as straight lines; the excessive elongation is obtained by the following method: [The text then abruptly shifts to a different topic:] ...adaptive control factor... The judgment formula for identifying elongated triangular units is as follows: in For newly inserted vertices Distance between other vertices; Step six: Render the gridded map obtained in step five. Use the rendering module to restore and reconstruct the map, thereby improving the realism and completeness of the reconstructed map.

2. The gridded mapping optimization method based on adaptive control factors as described in claim 1, characterized in that: It also includes step seven, which uses the reconstructed map obtained in step six to establish interactive scenarios, including robot localization, navigation, obstacle avoidance, and VR / AR; further improving the robot's localization and navigation capabilities and accuracy, increasing the immersive experience of the VR virtual environment, and solving related engineering application problems.

3. A gridded mapping optimization method based on adaptive control factors as described in claim 1 or 2, characterized in that: Step 3 describes the creation of a point cloud map. To achieve a better visual effect, filtering is added during the map creation process. Statistical filtering is responsible for removing invalid and outlier points, and calculating the distance between each point and its nearest neighbor. The distance values ​​of each point are distributed to remove points with excessively large mean distances; a voxel filter is used for downsampling to ensure that there is only one point within a voxel of a predetermined size, thereby reducing the number of point clouds.

4. The gridded mapping optimization method based on adaptive control factors as described in claim 1, characterized in that: In step six, the coordinates of three-dimensional points in space are projected onto the texture coordinates in the parameter space using a spherical projection function. Based on the point cloud map in step three, the color of the semantic model is determined, and the parameter space values ​​are converted to the texture space using a mapping function. Then, based on the actual scenario, virtual light sources are added to the map model, including parallel light sources, point light sources, and spotlights. The light sources are calculated based on each pixel using a fragment shader and interpolated according to the pixel normal. The map is then restored and reconstructed through the rendering module, improving the realism and completeness of the reconstructed map.

5. A gridded mapping optimization system based on adaptive control factors, implemented based on the gridded mapping optimization method based on adaptive control factors as described in claim 1 or 2, characterized in that: It includes a semantic map creation module, a point cloud map creation module, a semantic database, a map gridding module, and a post-rendering module; The semantic map building module includes performing semantic segmentation and image recognition on the image data acquired by the camera, building a semantic map by comparing it with the semantic models in the semantic database, and extracting and classifying features for newly emerging map scenes and individual models to expand the semantic database. The point cloud map building module includes adding filtering processing to the point cloud data after the lidar collects the data. The main processing methods are statistical filters to remove isolated points and points with large errors, and downsampling filters to downsample the point cloud data. The semantic database includes attributes such as individual model type, individual model feature attributes, and individual model importance score in the scene map. Scenes are divided into multiple types, and models should have different importance scores in different scenes. The map gridding module determines the adaptive control factor of each volume model in the map based on the importance score of the semantic model in the semantic database, the distance from the central observation position, and the multi-material composition data information, and uses the Delaunay triangle grid optimization algorithm to perform adaptive gridding reconstruction of the map. The post-rendering module processes operations including global illumination or point illumination, texture mapping, and shading mapping. It renders the gridded map input by the map gridding module and restores and reconstructs the map through the rendering module, thereby improving the realism and completeness of the reconstructed map.

Images