A processor-implemented method and system for indoor navigation map generation with accessibility information

Abstract

A processor-implemented method for indoor navigation map generation with accessibility information is provided. The method includes collecting a plurality of colored point cloud maps by utilizing both mobile and stationary mapping systems. The map can be aligned with an outdoor map if the GNSS or the global coordinate is available. The method also includes performing semantic segmentation based on at least implicit representation and synthetic data- aided training network for generating a plurality of structural elements and a plurality of functional elements. The method also includes performing topology-aided structural modeling by generating a structural building information modeling (BIM) using topology information at a plurality of levels of topology and by automatically applying a plurality of levels of topological constraints associated with the structural elements. The method also includes performing parametric modeling by detecting a plurality of functional objects in the functional elements using one or more simulated datasets including simulations of lighting, and environmental conditions.

Description

A PROCESSOR-IMPLEMENTED METHOD AND SYSTEM FOR INDOOR NAVIGATION MAP GENERATION WITH ACCESSIBILITY INFORMATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of the provisional patent application titled “A PROCESSOR-IMPLEMENTED METHOD AND SYSTEM FOR INDOOR NAVIGATION MAP GENERATION WITH ACCESSIBILITY INFORMATION”, with application number 63 / 722,203, filed in the United States Patent and Trademark Office on November 19, 2024. The specification of the above referenced patent application is incorporated herein by reference in its entirety.BACKGROUNDTechnical Field

[0002] The present invention generally relates to the field of navigation map generation. The present invention more particularly relates to a processor-implemented method and system for indoor navigation map generation with accessibility information.Description of the Related Art

[0003] Typically, building information modeling (BIMs) can be extracted from point clouds via several steps, including room segmentation, semantic / geometric features extraction, segmentation and classification, primitives’ estimation, and modeling generation to generate building information modeling (BIM) from point clouds. The BIM generation involves estimating the geometric primitives of the buildings. The goal of room segmentation (space partitioning) is to clip the whole point clouds of the building into different rooms, hallways, and other spaces, for example, stairs and elevators. Current solutions for room segmentation project the walls into a 2D plan and estimate the connection relationship between different walls. Specifically, the first step is to project all 3D point clouds into a 2D through a vertical projection. Then the wall lines segmentation will be implemented to extract the boundaries of each room. The lines can be represented as 2D primitive lines using a liner function, or a curve line using a polynomial equation. Finally, the spaces will be labeled as empty spaces or solid spaces, respectively for permanent structures or outsides. The final output will be different separated rooms with relative topology information. The main goal of the feature extraction is to extract the features from point clouds for the clusters and segmentation. The featureextraction process mainly extracts the levels of geometric features (e.g., normal vectors and mesh surfaces) and semantic features (e.g., tables). The segmentation and classification process involves conversion of raw point clouds into different groups using geometric features or semantic features. The segmentation involves dividing the point cloud into clusters or segments that represent different surfaces or features of the building structure, while classification can classify the point clouds according to its geometric characteristics (e.g., walls, floors, beams) which assists in automating the modeling process.

[0004] For example, semantic segmentation solutions, such as PointNet and PointNet++, cluster the point clouds into different groups using a deep-1 eaming-based solution. The primitive estimation uses algorithms to identify and extract basic geometric shapes (like planes, cylinders, and spheres) from the classified segments. This is crucial for creating primitive models that represent the physical environment. For example, RANSAC and its upgraded solutions will estimate the geometric planar equation of planes by selecting inlier points. In addition, identifying edges and comers helps in defining the precise shape and position of structural elements within the model. Modeling generation and optimization estimate the 3D wireframe / 2D floorplan from the extracted primitives. For example, B1M optimization with topology constraints and detected primitives is a sophisticated area of engineering that focuses on optimizing the material layout within a given design space, for a given set of loads, boundary conditions, and constraints with the goal of maximizing the performance of the system. This kind of method is widely used in aerospace, automotive, civil engineering, and product design to create lightweight and efficient structures. The topology optimization works as follows; the objective function could be to minimize the weight, maximize stiffness, reduce vibration, or any combination of performance metrics. The topology constraints are critical and might include limitations on total volume (amount of material), stress thresholds, displacement limits, or manufacturing constraints like overhang angles in additive manufacturing.

[0005] The optimization algorithm iteratively updates the material distribution within the design space to improve the objective function. The process uses finite element analysis (FEA) to simulate the mechanical behavior of the structure under the given loads and constraints. Methods such as the solid isotropic material with penalization (SIMP), level set, and evolutionary algorithms are commonly used. Generation of a navigation map and extracting a 2D map for indoor navigation typically involves several stages, from data collection to processing and map creation. The pipeline with manual work is the normal solution for industries. For instance, Autodesk, Revit and AutoCAD are popular for drafting and refininglayouts, while specialized software like Cloud Compare or MeshLab can handle point cloud processing. For the pathfinding and navigation part, tools such as Google's Indoor Maps and ArcGIS offer frameworks to develop indoor navigation applications.

[0006] However, manual works creating navigation map from point clouds or BIMs are labor-intensive and lead to unexpected errors. Current modeling information mainly focuses on structural primitives modeling, such as walls, and ceilings. The structural information can provide geometric information and floorplan information. However, for the navigation applications, the structural information is not enough for the path planning. For functional object detection, the size and boundary of the objects is not accurate enough for indoor navigation. Further, the performance of functional objects (e.g., doors) is limited due to the complex shape, missing holes, and the inconsistency of status (e.g., Door segmentation OA <10%). Low accuracy of semantic segmentation affects the representation of topology, especially the accessibility.

[0007] The above-mentioned shortcomings, disadvantages, and problems are addressed herein and will be understood by reading and studying the following specifications.SUMMARY

[0008] This summary is provided to introduce a selection of concepts in a simplified form that are further disclosed in the detailed description. This summary is not intended to determine the scope of the claimed subject matter.

[0009] In one aspect a processor-implemented method for indoor navigation map generation with accessibility information is provided. The method includes collecting a plurality of colored point cloud maps by utilizing both mobile and stationary mapping systems. The map can be aligned with outdoor map if the GNSS or the global coordinate is available. The method also includes performing semantic segmentation based on at least implicit representation and synthetic data-aided training network for generating a plurality of structural elements and a plurality of functional elements. The method also includes performing topology-aided structural modeling by generating a structural building information modeling (BIM) using topology information at a plurality of levels of topology and by automatically applying a plurality of levels of topological constraints associated with the structural elements. The method also includes performing parametric modeling by detecting a plurality of functional objects in the functional elements using one or more simulated datasets including simulations of lighting, and environmental conditions. The method also includesimplementing a boundary optimization to refine detection boundaries of functional objects for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation map representation and 3D graph-based accessibility. The 3D graph incorporates one or more nodes and one or more edges representing physical features and paths and enables dynamic update and scaling based on real-time data and user interactions.

[0010] According to an embodiment, performing topology-aided structural modeling includes cross-referencing the BIM against a database of standards using Al and rule-based algorithms and identifying and correcting non-compliance issues during the generation process.

[0011] According to an embodiment, performing parametric modeling includes detecting at least one of doors, elevators and stairs using generative Al models and simulated data by training generative models to create varied, realistic images of at least one of the doors, elevators, and stairs under a plurality of conditions.

[0012] According to an embodiment, the method further includes generating a comprehensive simulated dataset that replicates a variety of real-world scenarios, where the comprehensive simulated dataset includes one or more images or one or more 3D models of functional objects under a plurality of conditions including at least varying lighting, and occlusions. The method also includes training one or more deep learning models using the comprehensive simulated dataset, particularly Convolutional Neural Networks (CNNs) or Deep Neural Networks (DNNs), which are designed to extract features and learn object detection from complex backgrounds. The method also includes implementing data augmentation within the simulation to artificially expand the comprehensive simulated dataset with altered images for ensuring that the deep learning model is robust to variations in object appearance. The method also includes training the deep learning models to extract one or more relevant features from the objects and classify the one or more relevant features to a plurality of predefined categories.

[0013] According to an embodiment, implementing boundary optimization includes utilizing the trained object detection model to identify potential boundaries of objects within the data, implementing advanced algorithms to optimize one or more initial boundaries by utilizing energy-minimizing splines guided by external constraint forces and image forces that pull them toward features such as lines and edges in potential object boundaries, applying graph-theoretical methods to find the optimal boundaries based on predefined criteria and performing a rule-aided optimization by designing one or more geometric rules as constraints.

[0014] According to an embodiment, collecting the colored point clouds includes utilizing both mobile and stationary mapping systems to collect colored point clouds, for enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems.

[0015] In another aspect a system for indoor navigation map generation with accessibility information is provided. The system includes a processor configured to fetch and execute computer-readable instructions stored in a memory of the system, a memory configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed to create or share data packets over a network service. The system also includes a collection module for collecting a plurality of colored point cloud maps by utilizing both mobile and stationary mapping systems, a segmentation module for performing semantic segmentation based on at least implicit representation and synthetic data-aided training network for generating a plurality of structural elements and a plurality of functional elements, a structural modeling module for performing topology-aided structural modeling by generating a structural building information modeling (BIM) using topology information at a plurality of levels of topology and by automatically applying a plurality of levels of topological constraints associated with the structural elements, a parametric modeling module for performing parametric modeling by detecting a plurality of functional objects in the functional elements using one or more simulated datasets including simulations of lighting, and environmental conditions and an optimization module for implementing a boundary optimization to refine detection boundaries of functional objects, for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation map representation and 3D graph-based accessibility, where the 3D graph incorporates one or more nodes and one or more edges representing physical features and paths and enables dynamic update and scaling based on real-time data and user interactions.

[0016] According to an embodiment, the structural modeling module is further configured for cross-referencing the BIM against a database of standards using Al and rulebased algorithms and identifying and correcting non-compliance issues during the generation process.

[0017] According to an embodiment, the parametric modeling module is further configured for detecting doors using generative Al models and simulated data by training generative models to create varied, realistic images of doors under a plurality of conditions.

[0018] According to an embodiment, the system further includes a training module forgenerating a comprehensive simulated dataset that replicates a variety of real-world scenarios, where the comprehensive simulated dataset includes one or more images or one or more 3D models of functional objects under a plurality of conditions including at least varying lighting, and occlusions, training one or more deep learning models using the comprehensive simulated dataset, to extract features and leam object detection from complex backgrounds, implementing data augmentation within the simulation to artificially expand the comprehensive simulated dataset with altered images for ensuring that the deep learning model is robust to variations in object appearance and for training the deep learning models to extract one or more relevant features from the objects and classify the one or more relevant features to a plurality of predefined categories.

[0019] According to an embodiment, the optimization module is further configured for utilizing the trained object detection model to identify potential boundaries of objects within the data, implementing advanced algorithms to optimize one or more initial boundaries by utilizing energy-minimizing splines guided by external constraint forces and image forces that pull them toward features such as lines and edges in potential object boundaries, applying graph-theoretical methods to find the optimal boundaries based on predefined criteria and performing a rule-aided optimization by designing one or more geometric rules as constraints.

[0020] According to an embodiment, the collection module is further configured for utilizing both mobile and stationary mapping systems to collect colored point clouds, for enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems.

[0021] The present technology provides a navigation map for a navigation APP (e.g., Google) to extend the accurate navigation function from outdoor to indoor environments. Traditional navigation apps excel in outdoor environments but often struggle inside complex structures like malls, airports, and large office buildings. The present technology generates accurate navigation maps by using colored point clouds and utilizing raw 3D point clouds without incorporating any floor plan information and advanced object detection allows these apps to extend their functionality indoors. The navigation maps can provide navigation information and path planning information to ground mobile mapping system, smart vehicles, and UAVs. This is crucial for maintaining user orientation in indoor settings and enhancing user experience. The present technology detects semantic objects using an extended training dataset generated by GANs. The object detection and boundary optimization techniques can significantly improve the safety features by providing more accurate, and data for obstacledetection, pedestrian safety, and navigation in complex environments like crowded urban settings or parking lots. The present technology reduces costs in surveying while increasing the accuracy and speed of BIM. The capability of the present technology to generate BIM using levels-of topology constraints can streamline and automate many processes involved in BIM creation and updates.

[0022] By utilizing a simulated dataset for training object detection algorithms, the system is capable of recognizing and categorizing functional objects under a wide range of environmental conditions. This robustness enhances the reliability and versatility of the navigation system, particularly in complex and dynamic environments. The boundary optimization method refines the detection boundaries of identified objects, ensuring precise and reliable recognition. This is critical for applications where exact object dimensions and placements are necessary, such as in automated driving systems and architectural modeling. The generation of a 3D graph for the navigation map offers a structured and intuitive method of data visualization and interaction. The structured representation makes it easier to update, scale, and manipulate the map data, supporting more dynamic and responsive navigation systems. The application of levels of topology constraints in BIM generation ensures that the structural models are not only accurate but also compliant with relevant architectural standards and building codes. The hierarchical application of constraints helps in maintaining the structural integrity and legal compliance of the models without extensive manual oversight. The boundary optimization methods improve the ability of the system to adapt dynamically to changes within an indoor environment. For instance, if furniture is moved or new barriers are added, the system can quickly adjust its navigational paths, maintaining accuracy and reliability. The generation of a 3D graph for navigation maps enables a more intuitive visualization and interaction with the indoor space. Users can view multi-level structures in a comprehensible format, making it easier to navigate through floors and complex sections. The structural BIM generation with topology constraints can be integrated with other building management systems, such as HVAC, lighting, and security systems, to provide holistic management and navigation solutions. This integration can lead to smarter, more energyefficient buildings. The boundary of detailed functional objects (e.g., door, window, and elevator) provides accessibility information that will be extracted in an accurate solution. The navigation map can provide geometric and topology information for path planning and system control.

[0023] The embodiments herein address the above-recited needs for a system and amethod for indoor navigation map generation with accessibility information.

[0024] It is to be understood that the aspects and embodiments of the disclosure described above may be used in any combination with each other. Several of the aspects and embodiments may be combined to form a further embodiment of the disclosure.

[0025] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.

[0026] These and other objects and advantages will become more apparent when reference is made to the following description and accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The other objects, features and advantages will occur to those skilled in the art from the following description of the preferred embodiment and the accompanying drawings in which:

[0028] FIG. 1A illustrates an exemplary block diagram of a system for indoor navigation map generation with accessibility information, in accordance with an embodiment of the present technology.

[0029] FIG. IB depicts a navigation map generation pipeline using colored point clouds, in accordance with an exemplary scenario.

[0030] FIG. 2 depicts a structural BIM generation using levels of topology constraints, in accordance with an exemplary scenario.

[0031] FIG. 3 illustrates a flow diagram depicting a method for indoor navigation map generation with accessibility information, in accordance with an embodiment.

[0032] FIG. 4 illustrates an exemplary computer system in which or with which embodiments of the present disclosure may be implemented.

[0033] Although the specific features of the embodiments herein are shown in some drawings and not in others. This is done for convenience only as each feature may be combined with any or all of the other features in accordance with the embodiments herein.DETAILED DESCRIPTION OF THE DRAWINGS

[0034] The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the amount of details provided herein is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.

[0035] It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.

[0036] While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0037] The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the details provided herein is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.

[0038] It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.

[0039] While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood however, it is not intended to limit the disclosure to the forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0040] The various embodiments of the present technology provide an efficient technique for the generation of navigation map pipelines using colored point clouds. Thisinvention provides the navigation map for multiple mobile mapping systems, navigation systems, and robotics systems. The generated navigation map can provide such information, including the floor plan of the indoor environments, 3D wireframe, BIM, and level accessibility information, including door- for different rooms entry, stairs, and elevator for different floor navigation. An automatic extraction of three-dimensional (3D) wireframe / 2D-floor plan with navigation information is proposed to parametrically estimate the 3D modeling as well as the 3D navigation graph from semantic point clouds. Topology-aided structural modeling is disclosed to model the 3D wireframe. In some embodiments, a parametric description of structural elements is developed and accessibility information is encoded using a 3D graph based on the room distribution and parametric modeling of the doors, center of the room, room distribution, and the floors of the building. The present technology uses 3D parametric parameters and mathematical functions to represent the shapes and sizes of functional objects (e.g., elevators and doors).

[0041] FIG. 1A illustrates an exemplary block diagram of a system 100 for indoor navigation map generation with accessibility information, in accordance with an embodiment of the present technology. FIG. 1 A illustrates an exemplary block diagram of a system 100 and a method for conversion of geodatabase mapping formats into one of the indoor navigation formats, in accordance with an embodiment of the present technology. The system 100 includes a processor 102, a memory 104, a user interface 106, a processing engine 108 including a collection module 110, a segmentation module 112, a structural modeling module 114, a parametric modeling module 116, an optimization module 118, a training module 120 and a database 122. The processors 102 and the memory 104 may be communi cably coupled to one or more other processors. The one or more processor(s) may be implemented as one or more microprocessors, microcomputers, microcontrollers, edge or fog microcontrollers, digital signal processors, central processing units, logic circuitries, and / or any devices that process data based on operational instructions. Among other capabilities, one or more processor(s) may be configured to fetch and execute computer-readable instructions stored in the memory 104 of the system 100. The processor 102 is configured to fetch and execute computer-readable instructions stored in memory 104. The memory 104 may be configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed to create or share data packets over a network service. The memory 104 may include any non-transitory storage device including, for example, volatile memory such as Random-Access Memory (RAM), or non-volatile memory such as Erasable Programmable Read-Only Memory (EPROM), flash memory, and the like.

[0042] The memory 104 may be configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, fetched and executed to create or share data packets over a network service. The collection module 110 is configured for collecting a plurality of colored point cloud maps by utilizing both mobile and stationary mapping systems. A point cloud is a collection of points in 3D space representing the surface exposure of an object. Each point has a 3D coordinate (x, y, and z). In addition to coordinates, other useful data can be associated with each point, such as Red-Green-Blue (RGB) color (colored point cloud maps) or laser beam intensity. The collection module 110 utilizes both mobile and stationary mapping systems to collect colored point clouds, for enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems. The mapping system incorporates advanced sensors and cameras that capture high- resolution, full-color 3D point clouds, which provide intricate details about the environment, including surface textures and material properties.

[0043] In some embodiments, extraction using mobile mapping systems includes using sensors to continuously record the environment while navigating through spaces. The trajectory data is analyzed for patterns indicating transitions between environments, such as doorways or hallways connecting rooms. Distinct breaks or changes in the trajectory pattern help identify separate spaces, enabling the extraction of topological relationships between them. In some embodiments, extraction using stationary mapping system includes stationary scanners performing detailed 3D scans from fixed locations, capturing comprehensive point clouds of the surroundings. The overlap and density of these scans are used to delineate spatial boundaries. By examining the continuity and breaks in scan coverage, the physical separation and connectivity between rooms can be inferred, mapping the spatial topology effectively. Further in some embodiments, for structural elements topology, semantic models or rules are applied to raw spatial data to infer connections between structural elements. The system recognizes typical architectural features and their standard relationships, such as how walls connect to floors or ceilings. This method often utilizes predefined templates or rules that dictate how different structural elements interact based on their semantic definitions. In some embodiments, the functional elements topology is performed based on geometric relations involving analyzing the spatial arrangement and geometric relationships between fixed structural elements and movable or functional components. The process identifies and categorizes these relationships based on criteria such as distance, alignment, and intersection. This helps in understanding how functional elements like furniture are integrated into the overall structure, focusing on how they occupy and interact with the space.

[0044] The segmentation module 112 is configured for performing semantic segmentation based on at least implicit representation and synthetic data-aided training network for generating a plurality of structural elements and a plurality of functional elements. The structural modeling module 114 is configured for performing topology-aided structural modeling by generating a structural building information modeling (BIM) using topology information at a plurality of levels of topology and by automatically applying a plurality of levels of topological constraints associated with the structural elements. The structural modeling module 114 is further configured for cross-referencing the BIM against a database of standards using Al and rule-based algorithms and identifying and correcting non-compliance issues during the generation process. The criteria of BIM includes IndoorGML, CityGML, KML and IFC. The parametric modeling module 116 is configured for performing parametric modeling by detecting a plurality of functional objects in the functional elements using one or more simulated datasets comprising simulations of lighting, and environmental conditions. The parametric modeling module 116 is further configured for detecting doors using generative Al models and simulated data by training generative models to create varied, realistic images of doors under a plurality of conditions.

[0045] The optimization module 118 is configured for implementing a boundary optimization to refine detection boundaries of functional objects, for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation map representation and 3D graph-based accessibility. The functional object is a category that includes objects that provide accessibility information, such as doors and stairs. These objects are crucial for navigation and accessibility applications. The structural object is a category that consists of objects like walls and floors, which are integral to understanding the physical layout of an environment.

[0046] The 3D graph incorporates one or more nodes and one or more edges representing physical features and paths and enables dynamic update and scaling based on realtime data and user interactions. The optimization module 118 is further configured for utilizing the trained object detection model to identify potential boundaries of objects within the data, implementing advanced algorithms to optimize one or more initial boundaries by utilizing energy -minimizing splines guided by external constraint forces and image forces that pull them toward features such as lines and edges in potential object boundaries. Using the trained object detection model M, identify potential boundaries Binit of objects within an image I. This step involves applying the model to the image data to extract initial approximations of object boundaries based on learned features like edges and shapes.

[0047] The optimization module 118 is further configured for applying graph- theoretical methods to find the optimal boundaries based on predefined criteria such as for example, color and texture gradients. All the initial boundaries are considered as potential boundary points for the optimization and performing a rule-aided optimization by designing one or more geometric rules (e.g., parallel and connection relationship) as constraints.

[0048] The boundary optimization algorithm can be performed using an energyminimizing splines function, for example:Bopt = argminf (a • Emt(B(s)) + 0 • Eext(B(s), I))ds B where in the case of internal energy: Encourages spline continuity and smoothness, often defined by the spline’s curvature, and in the case of external energy: Aligns the spline with image features such as intensity gradients, edges, or textures. There are optional algorithms for the model optimization, for example, graph-theoretical methods:Bgraph = MinimumSpanningTree (G, w) where Nodes V represent potential boundary points from Binit, and edges E connect pairs of nodes, Weights , ,: Defined by a function g. which computes weights based on color, texture gradients, or other predefined criteria between nodes and Vj. The optimization goal is to find a minimum spanning tree or other optimal subgraph that best represents the object boundaries according to the weighted criteria. In some embodiments, a rule-aided boundary optimization is performed using rule-aided optimization solution with designed constraints.Bfmal= argmin / (B) subject to C(B)Where J(B) is an objective function evaluating the quality or suitability of boundary B . considering factors like boundary length or fitting errors, and constraints C(B) includes geometric rules such as ensuring parallelism, connectivity, or specific angular relationships are enforced as constraints C on the optimization process.

[0049] The training module 120 is configured for generating a comprehensive simulated dataset that replicates a variety of real-world scenarios. The comprehensive simulated dataset comprises one or more images or one or more 3D models of functional objects under aplurality of conditions comprising at least varying lighting, and occlusions. In some embodiments, advanced simulation tools and generative models (like GANs) are used to create realistic scenarios that cover a wide range of possibilities. The comprehensive simulated dataset includes functional objects rendered at different point densities. This simulates various levels of detail that might be captured by different sensor qualities or distances from the object. Low- density points might represent distant or poorly captured objects, while high-density points would simulate close-up or high-quality sensor captures. The comprehensive simulated dataset is generated using a simulated environment where the geometry and spatial relationships of objects are accurately modeled. This controlled setup allows for precise manipulation of lighting, opening statuses, and point densities. Virtual sensors mimic real-world data acquisition technologies, such as LiDAR or photogrammetric cameras, to collect data from the simulated environment. This includes varying the sensor resolution and scanning patterns to create diverse point density scenarios. The simulated scenarios include instances where the objects are partially obstructed by other objects, such as a stair blocked by furniture, or a door partially hidden behind a curtain. This helps in training the model to recognize objects even in partially visible conditions. Include scenarios where similar or different object types overlap, such as a stair overlaying part of a wall. This tests the model's ability to distinguish between adjoining structures.

[0050] The training module 120 is configured for training one or more deep learning models using the comprehensive simulated dataset, particularly Convolutional Neural Networks (CNNs) or Deep Neural Networks (DNNs), to extract features and learn object detection from complex backgrounds. The comprehensive simulated dataset provides a controlled and expansive training environment, allowing the deep learning models to leam detailed features without the noise and unpredictability of real-world data. The training module 120 is configured for implementing data augmentation within the simulation to artificially expand the comprehensive simulated dataset with altered images (e.g., rotated, scaled, or color- adjusted) for ensuring that the deep learning model is robust to variations in object appearance. The training module 120 is configured for training the deep learning models to extract one or more relevant features from the objects and classify the one or more relevant features to a plurality of predefined categories. Training the deep learning models involves both supervised learning, with labeled examples of each object type, and potentially unsupervised learning, to discover new object categories or features.

[0051] FIG. IB depicts a navigation map generation pipeline 124 using colored point clouds, in accordance with an exemplary scenario. At step 126, the color point clouds are takenas input. At step 128, implicit representation-aided semantic segmentation is performed. At step 130, structural elements are extracted and at step 132 functional elements are extracted. At step 134, topology-aided structural modeling is performed using the structural elements. At step 136, functional elements parametric modeling is performed using the functional elements. At step 138, 3D wireframe and 3D graph-based accessibility are generated as an output. At step 140, implicit representation is extracted. At step 142 the implicit representation is passed through synthetic data-aided training network 142 to generate functional elements at 132. At step 144, topology analysis and parametric estimation are performed.

[0052] According to an embodiment, colored point clouds map is collected using mobile mapping system and / or stational mapping system, enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems. The mapping system incorporates advanced sensors and cameras that capture high-resolution, full-color 3D point clouds, which provide intricate details about the environment, including surface textures and material properties. The application of topology information at multiple levels aids in the structural integrity and detail of building information modeling, ensuring models are both accurate and compliant with architectural standards. The dataset includes simulations of different lighting, and environmental conditions, providing a robust training platform for the Al algorithms responsible for object detection. Implementing a boundary optimization method to refine the detection boundaries of functional objects, enhancing the precision and reliability of object recognition. The 3D graph incorporates nodes and edges representing physical features and paths, respectively, allowing for dynamic updating and scaling based on real-time data and user interactions.

[0053] FIG. 2 depicts a structural BIM generation 200 using levels of topology constraints, in accordance with an exemplary scenario. The color point clouds 202 is collected using a mobile mapping system or stational mapping system. Utilizing both mobile and stationary mapping systems to collect colored point clouds enhances the detail and accuracy of data collected over traditional grayscale point cloud systems. The structural elements 204 are generated using the color point clouds 202 via semantic segmentation 201 and structural modeling 206 is performed. The local topology information 208 is generated from the structural elements 204 via primitive detection 203 and the local topology information 208 is used to generate topology graph 210 via graph generation 205. To generate structural BIM using levels of topology constraints the system uses Al and rule-based algorithms to cross-reference the BIM against a database of standards, identifying and correcting non-compliance issues during the generation process.

[0054] The embodiments herein detect functional objects using simulated dataset. Detecting doors using generative Al models and simulated data is an innovative application that can be particularly useful in fields such as robotics, autonomous vehicles, and augmented reality. The process involves training generative models to create varied, realistic images of doors under different conditions, which can then be used to train a door detection system. A comprehensive simulated dataset is generated that replicates a variety of real-world scenarios. This dataset includes images or 3D models of functional objects under different conditions, such as varying lighting, and occlusions. Advanced simulation tools and generative models (like GANs) are used to create realistic scenarios that cover a wide range of possibilities. The dataset is used to train deep learning models, particularly Convolutional Neural Networks (CNNs) or Deep Neural Networks (DNNs), which are designed to extract features and leam object detection from complex backgrounds. The simulated dataset provides a controlled yet expansive training environment, allowing the models to leam detailed features without the noise and unpredictability of real-world data. Implement data augmentation techniques within the simulation to artificially expand the dataset with altered images (e.g., rotated, scaled, or color- adjusted), ensuring that the model is robust to variations in object appearance. The models are trained to extract relevant features from the objects and classify them according to predefined categories. This involves both supervised learning, with labeled examples of each object type, and potentially unsupervised learning, to discover new object categories or features. During the boundary optimization process, the trained object detection model is utilized to identify potential boundaries of objects within the data. This step is called initial boundary detection. Subsequently, advanced algorithms are implemented to optimize these initial boundaries. This might involve utilizing energy-minimizing splines guided by external constraint forces and image forces that pull them toward features such as lines and edges in potential object boundaries. Also, graph-theoretical methods are applied to find the optimal boundaries based on predefined criteria like color and texture gradients. All the initial boundaries are considered as potential boundary points for the optimization. This step is called boundary optimization algorithm. Geometric rules (e.g., parallel and connection relationship) are designed as constraints for the rule-aided optimization.

[0055] FIG. 3 illustrates a flow diagram depicting a method for indoor navigation map generation with accessibility information, in accordance with an embodiment. At step 302, a plurality of colored point cloud maps is collected by utilizing both mobile and stationary mapping systems. According to an embodiment, collecting the colored point clouds includes utilizing both mobile and stationary mapping systems to collect colored point clouds, forenhancing the detail and accuracy of data collected over traditional grayscale point cloud systems. At step 304, semantic segmentation is performed based on at least implicit representation and synthetic data-aided training network for generating a plurality of structural elements and a plurality of functional elements. At step 306, topology-aided structural modeling is performed by generating a structural building information modeling (BIM) using topology information at a plurality of levels of topology and by automatically applying a plurality of levels of topological constraints associated with the structural elements. According to an embodiment, performing topology-aided structural modeling includes cross-referencing the BIM against a database of standards using Al and rule-based algorithms and identifying and correcting non-compliance issues during the generation process.

[0056] At step 308, parametric modeling is performed by detecting a plurality of functional objects in the functional elements using one or more simulated datasets including simulations of lighting, and environmental conditions. According to an embodiment, performing parametric modeling includes detecting doors using generative Al models and simulated data by training generative models to create varied, realistic images of doors under a plurality of conditions.

[0057] At step 310, a boundary optimization is implemented to refine detection boundaries of functional objects, for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation map representation and 3D graphbased accessibility, where the 3D graph incorporates one or more nodes and one or more edges representing physical features and paths and enables dynamic update and scaling based on realtime data and user interactions. According to an embodiment, implementing boundary optimization includes utilizing the trained object detection model to identify potential boundaries of objects within the data, implementing advanced algorithms to optimize one or more initial boundaries by utilizing energy-minimizing splines guided by external constraint forces and image forces that pull them toward features such as lines and edges in potential object boundaries, applying graph-theoretical methods to find the optimal boundaries based on predefined criteria and performing a rule-aided optimization by designing one or more geometric rules as constraints.

[0058] Creating an indoor navigation map using a 3D graph representation involves a detailed and systematic approach that leverages the geometry and semantic information of an environment. This 3D graph represents a comprehensive model where nodes and edges correspond to physical features and connectivity paths, respectively. The functional objects represent nodes in the graph represent functional objects like doors, stairs, and elevators. Theseare key points where user navigation decisions are made. The structural elements include elements such as walls, columns, and floors also form nodes. These are crucial for defining the physical constraints of the environment and aiding in accurate localization. The edges in the graph represent feasible paths that users can take to navigate between nodes. These include hallways, open spaces, and transitions between floors. Each edge carries attributes such as length, width, accessibility features (like ramps for wheelchairs), and directional constraints, which are essential for path planning and user guidance. In some embodiments a 3D occupancy grid is used. The 3D occupancy grid is a spatial representation that divides the environment into a three-dimensional grid of cells. Each cell in the grid contains information about the state of occupancy within that specific volume of space — whether it is empty, occupied, or uncertain. These grids are typically populated using real-time data from various sensors such as LiDAR, cameras, or ultrasound, providing up-to-date information about the environment's physical state. The 3D occupancy can be linked to the 3D navigation graph map. Each node and edge in the 3D graph corresponds to specific regions in the 3D occupancy grid. This linkage ensures that any changes in occupancy data directly influence the graph’s navigational paths and nodes. As occupancy information changes, the corresponding nodes and edges in the 3D graph can be dynamically updated. For example, if a particular area becomes obstructed, the connected paths in the graph can be adjusted to reflect reduced accessibility or complete blockage. The dynamic layer represents the spatial occupancy generated by the dynamic objects (e.g., desks).

[0059] According to an embodiment, the method further includes generating a comprehensive simulated dataset that replicates a variety of real-world scenarios, where the comprehensive simulated dataset includes one or more images or one or more 3D models of functional objects under a plurality of conditions including at least varying lighting, and occlusions. The method also includes training one or more deep learning models using the comprehensive simulated dataset, particularly Convolutional Neural Networks (CNNs) or Deep Neural Networks (DNNs), which are designed to extract features and leam object detection from complex backgrounds. The method also includes implementing data augmentation within the simulation to artificially expand the comprehensive simulated dataset with altered images for ensuring that the deep learning model is robust to variations in object appearance. The method also includes training the deep learning models to extract one or more relevant features from the objects and classify the one or more relevant features to a plurality of predefined categories.

[0060] FIG. 4 illustrates an exemplary computer system 400 in which or with which embodiments of the present disclosure may be implemented. The computer system 400 mayinclude an external storage device 410, a bus 420, a main memory 430, a read-only memory 440, a mass storage device 450, a communication port(s) 460, and a processor 470. A person skilled in the art will appreciate that the computer system 400 may include more than one processor and communication ports. The processor 470 may include various modules associated with embodiments of the present disclosure. The communication port(s) 460 may be any of an RS-232 port for use with a modem-based dialup connection, a 10 / 100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. The communication ports(s) 460 may be chosen depending on a network, such as a Local Area Network (LAN), Wide Area Network (WAN), or any network to which the computer system 300 connects.

[0061] In an embodiment, the main memory 430 may be Random Access Memory (RAM), or any other dynamic storage device commonly known in the art. The read-only memory 440 may be any static storage device(s) e.g., but not limited to, a Programmable Read Only Memory (PROM) chip for storing static information e.g., start-up or basic input / output system (BIOS) instructions for the processor 470. The mass storage device 450 may be any current or future mass storage solution, which can be used to store information and / or instructions. Exemplary mass storage solutions include, but are not limited to, Parallel Advanced Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and / or Firewire interfaces).

[0062] In an embodiment, x the bus 420 may communicatively couple the processor(s) 470 with the other memory, storage, and communication blocks. The bus 420 may be, e.g. a Peripheral Component Interconnect PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), USB, or the like, for connecting expansion cards, drives, and other subsystems as well as other buses, such a front side bus (FSB), which connects the processor 470 to the computer system 400.

[0063] Various embodiments of the present technology integrate colored point clouds for the generation of high-definition, highly detailed navigation maps. Unlike traditional mapping techniques that often rely on grayscale point clouds or 2D imaging, this system utilizes full-color data capturing subtle environmental cues. This enhances object recognition and environmental understanding, crucial for autonomous navigation systems. This technology integrates RGB data into point clouds as well as simulated data to enhance the detailed boundary and objects in generated maps. The real-time adjustment of map details based on environmental changes captured in color variations improves navigation reliability undervarying conditions. The aspect of functional object detection using a simulated dataset, implicit feature space, and boundary modeling optimization leverages a simulated dataset to train a detection system that operates within an implicit feature space, optimized through boundary modeling techniques. This approach allows the system to detect functional objects with high precision and adaptability, surpassing conventional explicit feature-based systems in complexity and detection capabilities. To simulate training environment a richly varied simulated dataset is used that provides extensive coverage of potential environmental scenarios, reducing the need for extensive real-world data collection. The simulated dataset is generated to enhance the entry boundary detection. The present technology also employs an innovative optimization of model boundaries based on real-time data, thereby enhancing the ability of the system to detect and interact with newly encountered or rare functional objects in dynamic environments. The present technology introduces an advanced method for generating structural building information modeling (BIM) using levels-of-topology constraints. The present method systematically applies topological constraints to ensure that generated models are not only geometrically accurate but also structurally sound and compliant with design codes, which is a significant advancement over traditional BIM systems that may require manual adjustments for such compliance. The automatic application of multiple levels of topological constraints during the BIM generation process enhances the structural integrity and accuracy of the models. The incorporation of a hierarchical modeling approach allows for the detailed development of BIM at various complexity levels, from basic structural forms to intricate architectural details. The integration of compliance verification with architectural and engineering standards directly into the BIM generation process ensures that all models meet the required specifications and reduces the need for post-processing.

[0064] The present technology provides a navigation map for a navigation APP (e.g., Google) to extend the accurate navigation function from outdoor to indoor environments. Traditional navigation apps excel in outdoor environments but often struggle inside complex structures like malls, airports, and large office buildings. The present technology generates accurate navigation maps using colored point clouds and advanced object detection allows these apps to extend their functionality indoors. This is crucial for maintaining user orientation in indoor settings and enhancing user experience. The present technology detects semantic objects using an extended training dataset generated by GANs and other generative Al solutions. The object detection and boundary optimization techniques can significantly improve the safety features by providing more accurate and data for obstacle detection, pedestrian safety, and navigation in complex environments like crowded urban settings or parking lots. The presenttechnology reduces costs in surveying while increasing the accuracy and speed of BIM. The capability of the present technology to generate BIM using levels-of topology constraints can streamline and automate many processes involved in BIM creation and updates.

[0065] By utilizing a simulated dataset for training object detection algorithms, the system is capable of recognizing and categorizing functional objects under a wide range of environmental conditions. This robustness enhances the reliability and versatility of the navigation system, particularly in complex and dynamic environments. The boundary optimization method refines the detection boundaries of identified objects, ensuring precise and reliable recognition. This is critical for applications where exact object dimensions and placements are necessary, such as in automated driving systems and architectural modeling. The generation of a 3D graph for the navigation map offers a structured and intuitive method of data visualization and interaction. The structured representation makes it easier to update, scale, and manipulate the map data, supporting more dynamic and responsive navigation systems. The application of levels of topology constraints in BIM generation ensures that the structural models are not only accurate but also compliant with relevant architectural standards and building codes. The hierarchical application of constraints helps in maintaining the structural integrity and legal compliance of the models without extensive manual oversight. The boundary optimization methods improve the ability of the system to adapt dynamically to changes within an indoor environment. For instance, if furniture is moved or new barriers are added, the system can quickly adjust its navigational paths, maintaining accuracy and reliability. The generation of a 3D graph for navigation maps enables a more intuitive visualization and interaction with the indoor space. Users can view multi-level structures in a comprehensible format, making it easier to navigate through floors and complex sections. The structural BIM generation with topology constraints can be integrated with other building management systems, such as HVAC, lighting, and security systems, to provide holistic management and navigation solutions. This integration can lead to smarter, more energyefficient buildings. The boundary of detailed functional objects (e.g., door, window, and elevator) provides accessibility information that will be extracted in an accurate solution. The navigation map can provide geometric and topology information for path planning and system control.

[0066] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and / or adapt for various applications such as specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications shouldand are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modifications. However, all such modifications are deemed to be within the scope of the claims. The scope of the embodiments will be ascertained by the claims to be submitted at the time of filing a complete specification.

Claims

CLAIMSWhat is claimed is:

1. A processor-implemented method for indoor navigation map generation with accessibility information, the method comprising: collecting a plurality of colored point cloud maps using both mobile and stationary mapping systems, wherein the map is aligned with an outdoor map when GNSS or a global coordinate is available; performing semantic segmentation based on at least implicit representation and synthetic data-aided training networks to generate a plurality of structural elements and a plurality of functional elements; performing topology-aided structural modeling by generating a structural building information model (BIM) using topology information at multiple levels and automatically applying topological constraints associated with the structural elements; performing parametric modeling by detecting a plurality of functional objects within the functional elements using one or more simulated datasets comprising simulations of lighting and environmental conditions; and implementing boundary optimization to refine the detection boundaries of functional objects, for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation and accessibility representation, wherein the 3D graph comprises one or more nodes and one or more edges representing physical features and paths, and enables dynamic updates and scaling based on real-time data and user interactions.

2. The method of claim 1 , wherein performing topology-aided structural modeling further comprises: cross-referencing the BIM against a database of standards using Al and rule-based algorithms; and identifying and correcting non-compliance issues during the BIM generation process.

3. The method of claim 1, wherein performing parametric modeling comprises: detecting at least one of doors, elevators, or stairs using generative Al models trained with simulated data, wherein the generative models create varied, realistic images of at least one of doors, elevators or stairs under a plurality of conditions.

4. The method of claim 3, further comprising: generating a comprehensive simulated dataset replicating a variety of real-world scenarios, the dataset comprising one or more images or one or more 3D models of functional objects under varying lighting and occlusion conditions; training one or more deep learning models, including Convolutional Neural Networks (CNNs) or Deep Neural Networks (DNNs), using the simulated dataset to extract features and enable object detection in complex environments; implementing data augmentation within the simulation to artificially expand the dataset with modified images, thereby improving model robustness to variations in object appearance; and training the deep learning models to extract one or more relevant features from the objects and classify them into predefined categories.

5. The method of claim 1, wherein implementing boundary optimization comprises: utilizing the trained object detection model to identify initial object boundaries within real-world data; implementing advanced algorithms to optimize identified initial object boundaries by utilizing energy-minimizing splines guided by external constraints and image forces that align with lines or edges in object boundaries. applying graph-theoretical methods to determine optimal object boundaries based on predefined criteria; and performing rule-aided optimization using geometric constraints.

6. The method of claim 1, wherein collecting colored point clouds comprises: utilizing both mobile and stationary mapping systems to collect colored point clouds, for enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems.

7. A system for indoor navigation map generation with accessibility information, the system comprising: a processor configured to fetch and execute computer-readable instructions stored in a memory of the system;a memory configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed to create or share data packets over a network service; a collection module for collecting a plurality of colored point cloud maps using both mobile and stationary mapping systems; a segmentation module for performing semantic segmentation based on at least implicit representation and synthetic data-aided training networks to generate a plurality of structural elements and a plurality of functional elements; a structural modeling module for performing topology-aided structural modeling by generating a structural building information model (BIM) using topology information at multiple levels and automatically applying topological constraints associated with the structural elements; a parametric modeling module for performing parametric modeling by detecting a plurality of functional objects within the functional elements using one or more simulated datasets comprising simulations of lighting and environmental conditions; and an optimization module for implementing boundary optimization to refine the detection boundaries of functional objects, for enhancing the precision and reliability of object recognition and generating a 3D graph for indoor navigation and accessibility representation, wherein the 3D graph comprises one or more nodes and one or more edges representing physical features and paths, and enables dynamic updates and scaling based on real-time data and user interactions.

8. The system of claim 7, wherein the structural modeling module is further configured for: cross-referencing the BIM against a database of standards using Al and rule-based algorithms; and identifying and correcting non-compliance issues during the BIM generation process.

9. The system of claim 7, wherein the parametric modeling module is further configured for detecting doors, windows, stairs, or any other elements which can provide accessibility information, using generative Al models trained with simulated data, wherein the generative models create varied, realistic images of doors, windows, stairs, or any other elements under a plurality of conditions.

10. The system of claim 7, further comprising a training module for: generating a comprehensive simulated dataset replicating a variety of real-world scenarios, the dataset comprising one or more images or one or more 3D models of functional objects under varying lighting and occlusion conditions; training one or more deep learning models, using the simulated dataset to extract features and enable object detection in complex environments; implementing data augmentation within the simulation to artificially expand the dataset with modified images, thereby improving model robustness to variations in object appearance; and training the deep learning models to extract one or more relevant features from the objects and classify them into predefined categories.

11. The system of claim 7, wherein the optimization module is further configured for: utilizing the trained object detection model to identify initial object boundaries within real-world data; implementing advanced algorithms to optimize identified initial object boundaries by utilizing energy-minimizing splines guided by external constraints and image forces that align with lines or edges in object boundaries. applying graph-theoretical methods to determine optimal object boundaries based on predefined criteria; and performing rule-aided optimization using geometric constraints.

12. The system of claim 7, wherein the collection module is further configured for: utilizing both mobile and stationary mapping systems to collect colored point clouds, for enhancing the detail and accuracy of data collected over traditional grayscale point cloud systems.

Images