System and method for augmented reality wayfinding assistant applications

Abstract

The invention in various embodiments includes systems, computer implemented methods, computer program products and a device worn or carried by a user related to directional wayfinding for use in assisting a user in conducting activities within an indoor space, such as within a building. The invention more specifically includes a computer supported application that enables a user to use their mobile device to take images or video of the indoor space in which an augmented reality (AR) image is overlayed or superimposed on the images and video being taken. The AR image provides specific information the user is looking for, such as the location of specific products or groups of products within the indoor space.

Description

[0001] SYSTEM AND METHOD FOR AUGMENTED REALITY WAYFINDING ASSISTANT APPLICATIONS CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Provisional Application No. 63 / 557,171, filed February 23, 2024. The entire specification and figures of the above-referenced application are hereby incorporated, in their entirety by reference. FIELD OF THE INVENTION The invention comprises systems, computer implemented methods and computer program products related to directional wayfinding for use in assisting a user in conducting activities within an enclosed environment, such as within a building. More specifically, the invention relates to the systems, methods and products that incorporate augmented reality (AR) in which overlayed virtual environments enhance the accessibility and user-friendliness of the activity undertaken by the user. BACKGROUND OF THE INVENTION AR generally refers to the technology that overlays virtual environments on top of the real world. AR technology currently allows for customizable models, information, and visual cues to users through diverse types of equipment, including Head-Mounted Displays (HMD) and handheld devices, such as smartphones and tablet computers. The extensive user base of smartphones and tablet computers facilitates the deployment of AR applications. AR-enabled wayfinding applications are known for use in complex building environments. These applications require precise building information models of the building environments and therefore, these applications based on building information models are not applicable to the majority of existing buildings in the U.S., such as grocery stores. Accordingly, there is a need for AR-enabled indoor wayfinding applications that can be created by non- professional personnel in which data to create virtual mapping of an indoor space can be achieved by generating relative spatial relationships between locations within the indoor space. Non-professionals can collect the necessary data regarding the spatial relationships with minimal training and easily accessible tools. One method to create virtual mapping of indoor spaces according to the present invention includes a crowdsourcing approach in which standardized data can be maintained in a cloud database. This method of the invention enables economical indoor wayfinding solutions for existing buildings that do not have accessible and up-to-date models. There are two major challenges that obstruct the development and use of AR wayfinding apps for enclosed environments. First, the majority of existing AR-enabled indoor wayfinding solutions derive context from detailed and up-to-date building models, yet most buildings do not have building models. As mentioned, building models may be available for some newly constructed, well-maintained, and complex building environments, such as airports and office buildings; however, the majority of other existing buildings and infrastructure systems do not have accessible building models. Even if an original building model exists, it is a rare occurrence that the models are timely updated to reflect changes in the arrangement of indoor spaces within the existing buildings. The limitation on available building models and current, accurate data highlights the need for a model-independent AR wayfinding application that can be created independent from building models. Indoor localization poses the second major challenge to AR wayfinding applications. Since the Global Positioning System (GPS) cannot provide accurate locations for indoor environments due to several factors including pervasive wireless connectivity issues, the best practice of AR indoor localization relies on signals from sensing devices distributed in the building, such as Bluetooth Low Energy beacons. An AR application detects these signals and estimates distances from each beacon based on the strength of the signal received. The application then calculates the sensing device(s) relative positions in three-dimensional space using trilateration and recommends routes to a destination by referring to an existing building information model. A user’s location can also be determined by comparing the building information model and an image captured by a video camera. Although this type of solution provides relatively accurate tracking and routes for wayfinding, equipment costs and maintenance costs can be financially prohibitive for broad deployment in the vast number of existing indoor spaces, most notably, retail spaces and grocery stores. Considering drawbacks of current wayfinding applications and the lack of building information model accessibility, there is a need for a new wayfinding application that does not rely on tracking devices, can be independently derived and deployed without the use of building information models, and can be customized and shared directly by users in a crowdsourcing approach in retail or other public spaces. As set forth below, the present invention provides a three-dimensional spatial map using spatial relationships between target locations, which is thereafter visualized in an AR prototype. The invention takes advantage of minimum data input requirements to facilitate AR-enabled wayfinding assistance. The invention significantly expands the possible deployment of AR- enabled wayfinding because of the simplified methodology in creating an AR application that does not require building models or expensive data gathering equipment. The invention provides a clear advantage for deploying AR-enabled wayfinding in all types of existing buildings that do not have three-dimensional models which is a required framework for existing or traditional AR- wayfinding applications.

[0002] SUMMARY OF THE INVENTION In accordance with corresponding embodiments of the invention, it comprises a system and computer implemented method for directional wayfinding. The wayfinding environment of the invention is the interior space of man-made structures which will be referred to generically herein as “buildings”. It should therefore be understood that the term “building” may encompass a wide range of man-made facilities including, but not limited to, commercial buildings, government or municipal buildings, shopping centers, retail locations, office buildings, hotels, industrial buildings, warehouses, and multifamily dwellings. In the system and method of the invention, the preferred embodiments incorporate AR in which overlayed virtual environments enhance the accessibility and user-friendliness of an activity undertaken by a user in the building. According to other embodiments of the invention, it comprises computer program products that facilitate AR-enabled directional wayfinding within a building. The computer program products comprise non-transitory computer-readable storage mediums having computer-executable instructions including various sets of computer readable codes that cause one or more connected computers to execute instructions relating to AR-enabled wayfinding functions. For purposes of understanding the embodiments of the invention, an AR-enabled wayfinding computer application or “App” of the invention is described with respect to the interior space of a commercial building, namely, a grocery store. An AR-enabled computer application or App of the invention therefore comprises an application that overlays the location of grocery store items on top of a camera captured grocery store environment. To develop the application, initially a data collecting individual or group of individuals are responsible for creating spatial maps of a grocery store using easily available measurement tools such as laser measuring devices including cell phones, or a cell phone’s accelerometer and gyroscope. A database of grocery store spatial maps may then be developed with collaborative efforts. Algorithms are provided to automatically convert spatial maps into visual cues that depict the locations of selected grocery locations. The development and deployment of activities of the wayfinding application may begin with creating a standard spatial map by collecting spatial data on grocery items. The spatial data, for example, can be distances measured between individual target destinations or between groups of target destinations. The target destinations can be, for example, the locations of specific products. The measured distances can be obtained by easily available measurement tools, such as laser measuring devices, by video cameras or mobile phones that have distance measurement capabilities, such as LiDAR. The database of grocery store spatial maps can then be developed with collaborative efforts. Algorithms are applied to the data to automatically convert spatial maps into visual cues that depict the relative locations of selected groceries. With respect to the development of an AR-enabled wayfinding application, the method or process may be more specifically described as including the following steps or actions: within an existing building such as a grocery store, spatial data is collected on selected grocery store items or selected categories of groceries. Through the use of one or more algorithms or mathematical equations, the measured distances are converted into a spatial map for the grocery store location relative to a known point, such as the building’s entrance. As explained in further detail below, the indoor environment mapping is achieved using a simplified trilateration method in two or three dimensions. After creation of the spatial map, the AR-enabled wayfinding application is created by computer programming techniques which include use of measured distances and images taken of the grocery store location are combined with AR overlay capabilities that enables the application to show the AR overlays at the retail location in real time. More specifically, when the user is within the grocery store location, the user can use the camera function on the user’s mobile device and when viewing the image taken by the camera, the user will see not only the real-world environment but also the AR overlay which provides the necessary information for the user to conduct a desired activity within the grocery store location, such as shopping for a specific product or group of products. It should be understood that other devices can be used to view the AR overlays within a building location in real time. These other devices can include HMDs and tablets as previously mentioned. There are a variety of HMDs that could incorporate the wayfinding application of the invention such as monocular HMDs, binocular HMDs, Virtual Reality (VR) HMDs, and optical head mounted displays (OHMDs). As necessary, the spatial data on grocery store items is updated using the latest or cumulative data in order to provide a more detailed and accurate application for display to a user. In addition to utilizing trilateration techniques, it is also contemplated that indoor environment mapping can be achieved by generating pathways from an analysis of aggregated pathway data from a collection of users. For example, users may be willing to share their individual localization data within a specific building (e.g. GPS data). The users’ movement data within a building supplemented with comments from the users as to what purchases may have occurred or what other interactions took place within the building can be used to generate spatial mapping elements to supplement trilateration elements of a special map. With respect to the deployment of the AR-enabled wayfinding application, the deployment can be more specifically described as first providing a shopping list of items that a user is looking to purchase at the retail location. Once the list of items on the shopping list is entered in the application, a display is provided as to the locations within the retail location as to where each item may be found. The display that can be provided by the application is real-time view of the retail location in which the user uses the camera function on the mobile device and the view provided to the user includes the actual image of the retail location as taken by the camera and a virtual overlay with queues or symbols that indicate where shopping list items are found within the location. As the user moves about the location, the camera of the mobile device will continue to show where the shopping list items are found according to the latest location of the user. In this way, easy tracking is provided to the user so that the user can continue to move about the location without losing any information on the shopping list items. For users that may be visually impaired or have difficulty reading words, the application provides an audio assistant that provides an audio output as to the location of the shopping list items. As the user moves about the location the audio assistant also has the capability to update the user as to the general distance and direction in which the user must travel in order to approach and find each specified shopping list item. Another capability of the application is that it can provide audio instructions and printed or visible instructions to the user in multiple languages. At the beginning of the shopping experience, a user may select a desired language for which to interact with the application. The mention of a camera of a mobile device herein is not limited to just cell phones or other similar handheld devices. Cameras for mobile devices can also include all types of HMDs and any other device that has camera or video functionality that is transported by a user while using the wayfinding app. After the user selects and picks up an item, the user clicks or checks off the item in the app in order to confirm pick-up and recalibrate the location of the user. In this way, the application can maintain accurate information as to the locations of the other items and locations of other functional areas within the location. In order to generate the indoor environment mapping with spatial relationships as opposed to using a building information model, one method of the invention is the use of mathematical trilateration to create spatial maps of indoor locations which simplifies mapping tools and the training required for personnel to obtain and record data to create the spatial maps. Prior to deploying the application at a desired indoor location, the data gatherer documents the location of known positions, such as the entrance of the location, and unknown positions, such as the specific positions of grocery items. As the application displays grocery item locations in two dimensions, at least two known locations are required to calculate the coordinates of one unknown location. The user enters the two known locations in data entry fields of the application and then the application has mathematical equations or algorithms that automatically generate the one unknown location. As the user continues to enter data in the application, additional locations are generated by the application. After the desired number of locations are created in the application, the relative spatial map of the indoor location can be made available to the user. As time progresses, recurring visitations to the indoor location by the user can be used to provide greater detail for the spatial map in which the user continues to enter data in the application upon each or selected visitations. According to one feature of the invention, spatial maps can be stored in a cloud database and made accessible to selected personnel from one or many different organizations. This crowdsourcing approach allows spatial maps to be created without burdening one individual or organization in recording application data. Particularly for indoor locations that are commonly visited by most people, such as grocery stores, the crowdsourcing approach allows for detailed mapping of the locations in an incremental approach and thereby also provides an improved and updated mapping capability over time. According to another feature of the invention, it is contemplated that the application can incorporate machine learning or artificial intelligence (AI) to filter scattered crowd-sourcing data that is collected by, for example, users using their mobile devices to therefore interpolate pathways and create model-independent maps of indoor spaces. According to another feature of the invention, in addition to or separate from creating spatial maps by use of the trilateration technique, it is also contemplated that the layout of a building can be created by use of data captured by a user’s mobile device that has gyroscopes and accelerometers and Lidar technologies. In the context of current “smart” phone products, many of these have gyroscope sensors and accelerometers. Gyroscopes are used in smart phones for a variety of functions including screen orientation, navigation, and gaming. Accelerometers in smart phones are used to measure constant forces such as gravity, time varying forces such as vibrations and acceleration forces such as tilt variations in the position of a phone. These forces can affect the orientation of a smart phone in space and can be measured in the three-dimensional axes and in linear acceleration, such as feet or meters per second squared. The application of the invention may include three core modules that are used to create the spatial maps by use of smart phones with the gyroscopes and accelerometers. A first module can be a data collection module that has a function of gathering data on (1) the locations of the entrance and exit used in each data collection trip; (2) user-specified destinations and navigation errors; and (3) gyroscope and accelerometer data that depicts user’s movements between the user-specified destinations. A second module may be a data processing module that can “de- noise” collected location data samples and merge the location data for each data collection trip. From the merged location data, a spatial map can be created for the indoor location. Mitigating cumulative errors inherent in location data samples is one challenge in creating an accurate spatial map. There can be cumulative errors associated with gyroscopes and accelerometers, such as time, distance, and turning frequency. These factors can be used as reliability indicators for the location data samples obtained by the smart phones. Two approaches can be taken to validate the accuracy of spatial maps created by use of data obtained from devices such as smart phones. One approach is using one or more AI algorithms to identify passageways or aisles in a mapped indoor location based on recorded location data and corresponding reliability indicators. A second approach is training AI algorithms to de-noise merged location data samples and generating passageway maps via a rule-based algorithm. A third module may be a maintenance and updating module for existing passageway or spatial maps. A default function of this third module may include regenerating spatial maps of a building by assigning a greater weight to newer data samples. This module has the ability to prioritize user input on user input of navigation errors when available so that the module can promptly re-generate spatial maps in response to recent building interior changes. When a user enters an enclosed environment, it is necessary to establish and confirm the user’s location or “anchor the origin” from which subsequent relative geometry can be used to create the spatial maps. According to the invention, it is contemplated that multi-modality data inputs can be used to calibrate the user’s location and orientation. More specifically, the application of the invention may be adapted to accept different data formats including but not limited to computer vision-enabled building environment recognition, QR codes, GPS signals, AI- video recognition from Augmented Reality (AR) glasses and other types of localization signals. Considering the foregoing, according to one aspect of the invention, it may be considered a system for directional wayfinding for use in assisting a user in conducting activities within an enclosed environment, such as within a building. The system includes one or more data gathering devices to obtain data corresponding to distance measurements within the enclosed environment, a computer processor for receiving and recording the data, computer coded instructions executed by said computer processor for developing a spatial map that shows a layout of the enclosed environment, the map comprising passageways and locations of target destinations, such as products stored within the building, a user interface associated with the computer processor for displaying information obtained by execution of said computer processor, the user interface including an image of the interior of the building as imaged by a camera of a mobile device and an augmented reality (AR) image that is overlaid on the image of the interior of the building, and wherein the user interface further includes visual symbols that are viewable by a user to indicate locations of products or services (i.e. restrooms or elevators) being sought within the interior of the building. There are a number of optional features that may be added to the above-described system. These features may include any one or all of the following: wherein the computer coded instructions include artificial intelligence (AI) functionality to de-noise merged location data samples or to filter and prioritize scattered crowd-sourcing data; wherein the target destinations are the location of consumer products stored in the building; and wherein the target destinations are the location of consumer products stored in the building. According to another aspect of the invention, it may be considered a computer program product that incorporates AR for use in conducting activities within an enclosed environment such as a building. The product comprises a non-transitory computer-readable medium containing computer executable instructions, wherein, when executed by a computer processor, the instructions cause the computer processor to execute a method of the product which includes a wayfinding application to navigate the interior space of a building. The executable instructions may include: instructions to receive and store data corresponding to spatial data of the interior space; instructions to convert spatial data to a user viewable map showing locations of passageways and product locations; instructions to convert spatial data and product specific data to an AR overlay that is viewable by a user when the user is located within the interior space and the user views a video camera image of the interior space or views another type of image of the interior space, such as an AR image, wherein the AR overlay is aligned with the video camera image as the user may move about the interior space. Optionally, the computer program product may incorporate artificial intelligence (AI) functionality in order to de-noise merged location data samples or to filter and prioritize scattered crowd-sourcing data. There are optional features that may also be added to the computer program product, and these may include any one or all of : wherein the computer readable instructions have artificial intelligence (AI) functionality to de-noise merged location data samples or to filter and prioritize scattered crowd-sourcing data; and wherein the target destinations are consumer product locations. According to another aspect of the invention, it may be considered a computer implemented method for directional wayfinding for use in assisting a user in conducting activities within the enclosed environment, such as a building. The method includes providing one or more data gathering devices to obtain data corresponding to distance measurements within the enclosed environment, providing a computer processor for receiving and recording the data, creating computer coded instructions and executed by the computer processor for developing a spatial map that shows a layout of the enclosed environment, generating a user interface associated with the computer processor for displaying information obtained by execution of said computer processor, the user interface including an image of the interior of the building as imaged by a video camera of a mobile device and an augmented reality (AR) image that is overlaid on the image of the interior of the building, generating visual symbols that are viewable by the user to indicate locations of target destinations, such as products being sought within the interior of the building. In one concept, the spatial map includes the AR image that is overlaid on the images taken by the video camera and wherein the AR image continues to maintain alignment with images such that when the user moves throughout the building, the AR image accurately indicates the locations of the products being sought. Again, there are a number of optional features that can be added to the method, including any one or all of: wherein the computer readable instructions have artificial intelligence (AI) functionality to de-noise merged location data samples or to filter and prioritize scattered crowd- sourcing data; wherein the AR image maintains alignment with images taken by the video camera such that when the user moves throughout the building, the AR image accurately indicates the locations of the target destinations being sought; and wherein the target destinations are consumer product locations. According to another aspect of the invention, it may be considered a device that can be worn by the user for directional wayfinding in conducting activities within an enclosed environment, wherein the device is worn by the user such as an eyewear product that contains all of the functionality of a mobile device that is used for the above-described computer implemented method. More specifically, the device can be generally characterized as eyewear with a virtual heads-up display that is seen by the user which includes real-time images of the interior of the building in which the user is located along with the overlaid AR images. The overlaid AR images may include the visual symbols that indicate locations of target destinations, such as products or services within the building. As the user passes through the interior of the building, the user will view the actual interior of the building along with the corresponding overlaid AR images. The basic functionality of the eyewear device of the invention may be similar to some existing commercial products such as Google Glass® or Apple Vision®. In this regard, navigation control by the user is achieved through a combination of eye, voice or hand gestures. With respect to the basic structure of the device, it may include those elements of standard eyeglasses including an eyeglass frame structure that is used to secure a heads-up display which occupies the visual space in front of the user. Alternatively, the device could incorporate headgear which is used to support the heads-up display that is placed in front of the eyes of the user. The heads-up display allows the user to view the surrounding environment and the special features of the device including the AR images and other symbols that enable the user to navigate within the building. One optional feature of the device may be that the desired target destinations are the locations of products or services within the building. In yet another embodiment of the invention, the device can be generally characterized as an HMD and the components of the HMD may be those shown in FIG. 6. Other features and advantages of the invention will become apparent with a review of the following drawings and detailed description.

[0003] BRIEF DESCRIPTION OF THE FIGURES FIG. 1 is a flow diagram showing steps in the development of and deployment of the computer- based App of the invention; FIG. 2 is a sample video image of the interior of an indoor location, the image showing various target destinations, specifically products within a grocery store, along with the open spaces or passageways between the products mounted on the store shelves; FIG. 3 is the sample video image of FIG. 2 further showing elements of an AR overlay in which types of products within the store are shown as to their location according to the video image; FIG. 4 is a schematic diagram illustrating the trilateration method of the invention in determining the location of an unknown location as derived from the known location of two known locations and distances between the two known locations and the unknown location; FIG. 5 is a schematic diagram of a data processing system that can support the App of the present invention; and FIG. 6 is a schematic diagram of a head mounted display that can be used to visualize the images of the App of the invention and to control the App of the invention

[0004] DETAILED DESCRIPTION OF THE INVENTION FIG. 1 is a flow diagram showing steps in the development of and deployment of the application of the invention. More specifically, FIG. 1 shows basic steps or considerations in how to develop the application of the invention and how the invention is used, such as withing a grocery store. In development of the App, one consideration or goal is to generate a standard spatial map format that can be used as a repeating template for other locations. Spatial data is collected on grocery store items that are of interest to the user. For example, the user may generate a shopping list that is reflective of common items in which the user may wish to purchase within the grocery store. The spatial data is obtained, for example, by a leisure measuring device in which distances are recorded between marked locations within the grocery store. These marked locations could be the location of other products within the shopping list or prominent locations within the grocery store such as the entrance, exit, bakery, deli, etc., and these structures serve as the base location for that particular product as the App is developed. Of course, as product locations may change within a grocery store, in which case location data will be updated by the next user who locates the item. A spatial map is ultimately created from the collected spatial data. The application of the invention is then created which functions to help navigate a user based upon the spatial map. The spatial data, grocery store items must continually be updated over time to reflect new products and the change in location of grocery store items as the indoor location may be arranged over time. With respect to deployment of the App, the user may start with the desired shopping list in which each of the products within the shopping list are displayed to the user in two ways. First, an overall store layout or spatial map may be provided to the user in a first user display which helps the user obtain an understanding about the overall locations of the products within the store. This overall store layout may assist the user in determining how the user may wish to navigate through the store most efficiently. The second user display can be described as an active user display in which the user activates the video camera on their mobile device and as the user travels through the store, the user is able to see the actual interior of the store and overlaid augmented reality image showing where specific products or groups of products may be found. As the user moves through the store, the AR image is able to maintain alignment with the actual image of the store such that there is always an accurate representation of the location of the desired product despite the user moving throughout the store. Once a particular grocery store item is picked up, checks off the item, resulting in data that can be used to help recalibrate the location relative to the other products within the store. Ultimately, the user will pick up the desired remaining items in the shopping list and then can complete the grocery shopping. It is important to note that the user has autonomy of pathway through the interior space and that the App’s guidance adjusts to user location in real time. FIG. 2 is a sample video image of the interior of an indoor location, the image showing various products within a grocery store along with the open spaces or passageways between the products mounted on the store shelves. As shown, the grocery store may include a plurality of product shelves separated by aisles or passageways along with a plurality of corresponding overhead visual signs that may be consecutively numbered and may contain short descriptions of the products found within those aisles. The grocery store may include other products such as products in a refrigerated case, the example in FIG. 2 showing a refrigerated case on the lower right side of the image. FIG. 3 is the sample video image of FIG. 2 further showing simplified elements of an AR overlay in which types of products within the store are shown as to their location according to the video image. More specifically, FIG. 3 shows four AR elements, namely, “milk”, “broccoli”, “pasta”, and “cashier”. Each of these AR elements are shown in the English language and with directional arrows pointing specifically to the locations within the store as to where the respective products or the cashier are located. It should be understood that this FIG. 3 is a simplified AR image and that there are a number of additional AR elements that could be illustrated. For example, both general product groupings and specific products could be shown in the AR overlay. Other directional assistance elements could be provided such as the location of the entrance, exit, bathrooms, service desk, etc. Further, the AR elements are not limited to the English language and a user can selectively display a desired foreign language. Yet further, color coding can be provided to indicate various categories of products as well as various categories of interior structures such as the entrance and exit. For those users that are visually impaired or those users that may not be able to read effectively, other means may be provided on the AR overlay to include visual symbols of products as well as audio instructions as to where the user is currently located within the store based upon the captured video image. For those who are visually impaired, such users could generate a voice command to the mobile device in which the App is then able to provide audio instructions as to how to move to the desired product or group of products. For example, the user could issue the command: “please direct me to the bakery” and upon issuing the command, the Application would provide verbal instructions to the user in terms of the number of steps and distance to the bakery along with any necessary turning movements the user may have to undertake. The AR overlay can be provided to the user in various presentations so that the user has detailed visual cues and audio cues as to how to travel within the building based on the user’s instructions for finding one or more targeted destinations. The visual cues can be in the form of written words or symbols. The audio cues can be in the form of a specified language or a group or series of preconfigured sounds that would indicate how the travel of the user is progressing through the building. FIG. 4 is a schematic diagram illustrating the trilateration method of the invention in determining the location of an unknown location as derived from the known location of two known locations and distances between the two known locations and the unknown location. More specifically, FIG. 4 demonstrates two typical measuring scenarios. The two trilateration examples are illustrated with respect to two unknown points or locations, (x1,y1) and (x4,y4). The first trilateration example involves a data gatherer, such as a caregiver, who labels the first unknown location (x1, y1). From the unknown location, the user records the location of two known locations or reference points. The first reference point could be the location of milk (0,a) that was previously established, and the second reference point could be the entrance of the store (0,0) that was also previously established. The user then measures and records the distance between the reference points and the unknown location (distances a, b and c). From this data entered regarding the reference points and the three distances, the application of the invention calculates the location of the first unknown location. Similar to the first trilateration example, the second unknown location (x4,y4) is derived by measuring the distances between two reference points and the second unknown location and entering data on the location of the two reference points. The application of the invention automatically applies trilateration mathematics, such as the below equations, in order to establish the positions of the unknown locations. Equations 1-3 can be used with respect to the first trilateration example while equations 4 and 5 can be used with respect to the second trilateration example. This trilateration mathematics should be understood to be exemplary only, and accordingly the invention should not be interpreted as limited by this exemplary mathematics. Eq. 1 As mentioned, spatial maps can be stored in a cloud database and can be easily accessed when needed. With respect to data entry and generation of a spatial map, a spatial map of a given indoor location can be first imported as a Comma-Separated Values (CSV) document into an AR template. A C# script can be used to create a new grocery item class for every item in the CSV document. Each CSV document could include the item name, coordinate, and coordinate. When deployed, a data gatherer or caregiver imports a grocery shopping list containing only the item names in the CSV format. Users can use GPS data available the entrance of the grocery store to set up their initial locations, and the Application then generates the spatial relationships between items with the entrance to display the name of all included items at their actual locations. One of the advantages of the invention is that the wayfinding application of the invention does not have to be derived from any existing building information model and rather, the invention generates a fully functional and detailed “pseudo model” derived from independent data collection and analysis. The functionality associated with Fig.4 provides a method by which a pseudo model can be created for the wayfinding app of the invention. FIG.5 is a schematic diagram of a data processing system that can support the Application of the present invention. More specifically, FIG. 5 shows an example computer processing and communication network that may be used in connection with the systems and methods of the invention disclosed herein. More specifically, according to a computer processing and communication network 100, one or more crowdsource computers 102, 112 and 114 may be provided. One or more of these computers may be integrated into a specified internal network 104. The computers 102, 112 and 114 can be, for example, those computers used by a group of individuals or entities that provide updates for the Application by way of data inputs for creation and revisions to spatial maps. These computers may comprise general purpose personal computers (including, merely by way of example, personal computers and / or laptop computers running various versions of Microsoft’s Windows® and / or Apple® operating systems) or smart phones (such as Android or iPhones). The computers 102, 112 and 114 may have any of a variety of applications, including for example, database client and / or server applications, and web browser applications. Each of the computers 102, 112 and 114 are capable of communicating via a network and / or displaying and navigating web pages or other types of electronic documents. The system 100 may further include a network / cloud provider 106 that can support data communications using any of a variety of commercially available protocols, including without limitation TCP / IP, SNA, IPX, AppleTalk®, and the like. Merely by way of example, the communications network 110 maybe a local area network (“LAN”), such as an Ethernet network, a Token-Ring network and / or the like; a wide-area network; a virtual network, including without limitation a virtual private network (“VPN”); the Internet; an intranet; an extranet; an infra-red network; a wireless network (e.g., a network operating under any of the IEEE 802.11 suite of protocols, the Bluetooth™ protocol known in the art, and / or any other wireless protocol); and / or any combination of these and / or other networks. The system may also include one or more server computers 108. One type of server may include a web server used to process requests for web pages or other electronic documents from user computers 102, 112 and 114. The web server can run an operating system including any of those discussed above, as well as any commercially available server operating systems. The web server can also run a variety of server applications, including HTTP servers, FTP servers, CGI servers, database servers, Java servers, and the like. In some instances, the web server may publish operations available as one or more web services. The computer server 108 may also represent one or more file and / or application servers, which can, in addition to an operating system, include one or more applications accessible by a client running on one or more of the user computers 102, 112 and 114. The file / application server(s) may be one or more general purpose computers capable of executing programs or scripts in response to the user computers 102, 112 and 114. As one example, the server may execute one or more web applications. The web application may be implemented as one or more scripts or programs written in any programming language, such as Java®, C, C#™ or C++, and / or any scripting language, such as Perl, Python, or TCL, as well as combinations of any programming / scripting languages. The application server(s) may also include database servers, including without limitation those commercially available from Oracle®, Microsoft, Sybase®, IBM® and the like, which can process requests from database clients running on a user computer. The system 100 may also include a database 130. The database 130 may reside in a variety of locations. By way of example, database 130 may reside on a storage medium local to (and / or resident in) one or more of the computers 102, 112 and 114. In a particular set of embodiments, the database 130 may reside in a storage-area network (“SAN”). Similarly, any necessary files for performing the functions attributed to the computers 102, 112 and 114, may be stored locally on the respective computer and / or remotely, as appropriate. The database 130 may be a relational database, such as Oracle® database, which is adapted to store, update, and retrieve data in response to SQL-formatted commands. The system further includes one or more mobile devices 120 such as smart phones. These mobile devices 120 may communicate with other computers of the system or servers by a web interface. The mobile devices have their own internal computer processing capabilities with integral computer processors and other supporting hardware and software. The mobile devices may be specially configured to run a mobile software application(s) in order to view user interfaces and to view and update system data. More specifically, the mobile devices are configured to run the application of the invention. All of the functionality associated with the system as applied to the computers 102, 112 and 114 may be incorporated in the mobile devices 120 as modified by mobile software applications especially adapted for the mobile device hardware and operating systems. In connection with operating systems, it should therefore be understood that the mobile devices are not limited to any particular operating system, Apple iOS and Android-based systems being two examples. The mobile devices 120 are primarily operated by users of the invention to conduct indoor activities such as shopping in a grocery store. It should also be understood that the depicted mobile devices 120 can be any device that is capable of being carried or worn by a user to include tablets, HMD’s, VR glasses or goggles, or any other electronic device that has the capability to run the App of the invention. FIG. 5 also schematically illustrates one or more field devices such as measuring devices 130 and video recording units 140 in which each may have supporting sensor circuitry (not shown) for recording and transmitting measurement and video signals to the computers or mobile devices. As explained, a laser measuring device can be used to measure distances within an indoor location. The video recording unit can be used to generate templates for the AR overlays that are displayed on the mobile devices 120. In accordance with any of the computers 102, 112, 114 and the mobile devices 120, these may be generally described as general-purpose computers with elements that cooperate to achieve multiple functions normally associated with general purpose computers. For example, the hardware elements may include one or more central processing units (CPUs) for processing data. The computers may further include one or more input devices (e.g., a mouse, a keyboard, etc.); and one or more output devices (e.g., a display device, a printer, etc.). The computers may also include one or more storage devices. By way of example, storage device(s) may be disk drives, optical storage devices, solid-state storage device such as a random-access memory (“RAM”) and / or a read-only memory (“ROM”), which can be programmable, flash-updateable and / or the like. In one functional aspect, an application server may create web pages dynamically for displaying the functionality associated with the system to include user interfaces that show the AR overlays. Each of the computers and mobile devices described herein may include a computer- readable storage media reader; a communications peripheral (e.g., a modem, a network card (wireless or wired), an infra-red communication device, etc.); working memory, which may include RAM and ROM devices as described above. The computers and mobile devices may also comprise various software elements and an operating system and / or other programmable code such as program code implementing a web service connector or components of a web service connector. It should be appreciated that alternate embodiments of a computer may have numerous variations from that described above. For example, customized hardware might also be used and / or particular elements might be implemented in hardware, software (including portable software, such as applets), or both. Further, connection to other computing devices such as network input / output devices may be employed. It should also be appreciated that the method described herein may be performed by hardware components or may be embodied in sequences of machine-executable instructions, which may be used to cause a machine, such as a general-purpose or special-purpose processor or logic circuits programmed with the instructions to perform the methods. These machine- executable instructions may be stored on one or more machine readable mediums, such as CD- ROMs or other type of optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or other types of machine-readable mediums suitable for storing electronic instructions. Alternatively, the methods may be performed by a combination of hardware and software. The application of the invention may be configured to run software to achieve the desired functionality described herein. The term “software” shall be broadly interpreted to include all information processed by a computer processor, a microcontroller, or processed by related computer executed programs communicating with the software. Software therefore includes computer programs, libraries, and related non- executable datahttps: / / en.wikipedia.org / wiki / Data_(computing), such as online documentation or digital media. Executable code makes up definable parts of the software and is embodied in machine language instructions readable by a corresponding data processor such as a central processing unit of the computer. The software may be written in any known programming language in which a selected programming language is translated to machine language by a compile, interpreter or assembler element of the associated computer. Considering the foregoing exemplary computer and communications network and elements described therein, in connection with one embodiment of the invention, it may be considered a software program or software platform with computer coded instructions that enable execution of the functionality associated with creating visual displays on the video camera of a user’s mobile device. More specifically, the invention may be considered a software program or software platform that executes programmed instructions based on data inputs as described herein. In connection with yet another embodiment of the invention, it may be considered a sub- combination including one or more user interfaces generated by the software and the field devices 130 and 140 that provide inputs to a data processor of a computer that runs the software for purposes of generating the AR overlays. Integrating AI into the application of the invention can allow for streamlining both data collection and information delivery processes. Indoor photogrammetry and AI-enabled object recognition as used in the present invention can automate spatial map development tasks. Furthermore, an AI-enabled real-time assistant as provided by the current invention allows for more detailed and precise instructions to users while operating the application, such as managing and reading a checklist, finding target items when close to the destination, and requesting a spatial map update. FIG. 6 is a schematic diagram of a generic or common head mounted display (HMD) that can be used to visualize the images of the App of the invention and to control the App of the invention. This figure is therefore intended to only show basic components of an HMD which could also encompass VR goggles or glasses. As shown, the HMD 200 may include one or more optic lenses 210, one or more mirrors 220, a combiner 230, a projector 240, an eyeglass frame or headgear 250, a computer processor 260, and a communications module 270. Optic lenses 210 represent any lenses that the HMD may use to focus images for viewing or recording. One advantage of an HMD is that the lenses can be used to focus images therefore allowing the user to perceive them at a comfortable distance, thereby enhancing the virtual environment that may be otherwise more difficult to perceive with the naked eye. The mirrors 220 may be used to redirect and precisely direct projected digital images from the display onto the user's eyes. Accordingly, the mirrors may act as a light path that reflects generated images to the correct angle to reach the user's pupils in which the images incorporate both the virtual content and the real-world view content. The combiner 230 is the optical component that merges the digitally generated image from the display with the user's real-world view. The combiner 230 is a semi- transparent optical element that allows the user to see both the real world and the virtual content generated by the display simultaneously. This simultaneous display provides overlayed digital information onto the user's view of the environment. The combiner has attributes of a partial mirror in which the digital image is reflected while the real-world light passes through. The key function of the combiner is to integrate two visual streams into a single image for viewing. The projector 240 may be required to project images onto the mirror 220 so images can be directed by the mirror to the correct location for viewing. The eyeglass frame or headgear 250 is the structure that attached the HMD to the user. In the case of eyeglass frames, these function in the same way as traditional eyeglass frames that each have a bridge, rim and opposing hinged temples. In the case of headgear, these function in the same way as traditional headgear that may be used for attaching a medical / dental device or instrument to the head of a user. The computer processor 260 is used to control functions of the HMD which could include control by voice command recognition. The processing capability of the HMD could be like that of the processor for a mobile communication device or smartphone. The communications module 270 can be provided to allow the HMD to also act as a mobile communication device with the same functionality as a smartphone. FIG. 6 is a simplified schematic view of an HMD and it should be understood that not all of the components shown in FIG. 6 may be required for an HMD. For example, a particular HMD may not have a communications module. Therefore, this figure will not be construed as limiting the description of a basic HMD as requiring each of the illustrated components. While the invention is described herein with respect to multiple preferred embodiments, it should be understood that the invention is not strictly limited to these embodiments and therefore, the invention in totality should be considered commensurate with the scope of the claims appended hereto.

Claims

What is claimed is:

1. A system of directional wayfinding for use in assisting a user in conducting activities within an enclosed environment such as a building, said system comprising: at least one data gathering device to obtain data corresponding to distance measurements within the enclosed environment; a computer processor for receiving and recording the data; computer coded instructions executed by said computer processor for developing a spatial map that shows a layout of the enclosed environment, the map comprising passageways and locations of target destinations within the building; a user interface associated with the computer processor for displaying information obtained by execution of said computer processor, the user interface including an image of the interior of the building as imaged by a video camera of a mobile device and an augmented reality (AR) image that is overlaid on the image of the interior of the building; and wherein the user interface further includes visual symbols that are viewable by a user to indicate locations of the target destinations being sought within the interior of the building.

2. The system of claim 2, further wherein: said computer coded instructions include artificial intelligence (AI) functionality to de- noise merged location data samples or to filter and prioritize scattered crowd-sourcing data.

3. The system of claim 1, wherein the target destinations are the location of consumer products stored in the building.

4. A computer program product that incorporates augmented reality (AR) for use in conducting activities within an enclosed environment such as a building, the computer program product comprising: a non-transitory computer-readable medium containing computer executable instructions, wherein, when executed by a computer processor, the instructions cause the computer processor to execute a method of the computer program product which includes a wayfinding application to navigate the interior space of a building; computer readable instructions to receive and store data corresponding to spatial data of the interior space; computer readable instructions to convert spatial data to a user viewable map showing locations of passageways and target destinations; computer readable instructions to convert spatial data and target destinations specific data to an AR overlay that is viewable by a user when the user is located within the interior space and the user views a video camera image of the interior space; and wherein the AR overlay is aligned with the video camera image as the user may move about the interior space.

5. The computer program product of claim 4, further wherein: said computer readable instructions have artificial intelligence (AI) functionality to de- noise merged location data samples or to filter and prioritize scattered crowd-sourcing data.

6. The computer program product of claim 4, wherein: the target destinations are consumer product locations7. A computer implemented method for directional wayfinding for use in assisting a user in conducting activities within an enclosed environment, such as a building, said method comprising: providing one or more data gathering devices to obtain data corresponding to distance measurements within the enclosed environment; providing a computer processor for receiving and recording the data; creating computer coded instructions and executed by the computer processor for developing a spatial map that shows a layout of the enclosed environment; generating a user interface associated with the computer processor for displaying information obtained by execution of said computer processor, the user interface including an image of the interior of the building as imaged by a video camera of a mobile device and an augmented reality (AR) image that is overlaid on the image of the interior of the building; and generating visual symbols that are viewable by the user to indicate locations of target destinations being sought within the interior of the building.

8. The computer implemented method of claim 7, further wherein: said computer readable instructions have artificial intelligence (AI) functionality to de- noise merged location data samples or to filter and prioritize scattered crowd-sourcing data.

9. The computer implemented method of claim 7, wherein: the AR image maintains alignment with images taken by the video camera such that when the user moves throughout the building, the AR image accurately indicates the locations of the target destinations being sought.

10. The computer implemented method of claim 7, wherein: the target destinations are consumer product locations.

11. A device worn by a user for directional wayfinding for use in assisting a user in conducting activities within an enclosed environment, such as a building, said device comprising: a wearable eyeglass structure including at least one of an eyeglass frame or headgear; a heads-up display supported by said at least one eyeglass frame or headgear; said heads-up display including a viewable area through which the user can see the surrounding environment when the device is worn; augmented reality images superimposed upon the viewable area seen by the user, the augmented reality images generated by: a computer processor that has received and recorded data corresponding to distance measurements within the enclosed environment and data corresponding to physical locations of objects within the enclosed environment; computer coded instructions executed by the computer processor for generating viewable objects within the augmented reality images, said viewable objects including at least one of (a) a spatial map that shows a layout of the enclosed environment, (b) visual symbols viewable by the user to indicate locations of target destinations being sought within the interior of the building, and (c) directional symbols viewable by the user to provide directional assistance to the user in finding a desired target destination within the building.

12. The device of claim 11, wherein: The desired target destinations are the locations of products or services within the building.

Images